I am trying to run this for-loop and just seeking some feedback on what it is supposed to do:
for FF in `cat bams`
do
F=$(basename $FF)
F_PREFIX=${F/.bam/}
angsd -i $F -anc $GENOME_REF $FILTERS -GL 1 -doSaf 1 -doCounts 1 -out ${F_PREFIX} && realSFS ${F_PREFIX}.saf.idx >${F_PREFIX}.ml | awk -v file=$F '{print file\"\t\"(\$1+\$2+\$3)\"\t\"\$2/(\$1+\$2+\$3)}' ${F_PREFIX}.ml >> goodbams.goodsites.het
done
The issue I have is piping the output to awk.
Can someone comment if the input file (file=$F) is the original file under
F/.bam
or
{F_PREFIX}.saf.idx
or
${F_PREFIX}.ml
I am also unclear what the second ${F_PREFIX}.ml is supposed to do in the awk script.
This to me looks like the .saf.idx output should be stored in the ${F_PREFIX}.ml file, and then awk should execute the command using the ${F/.bam/} file and also put the output into ${F_PREFIX}.ml.
Then those total ${F_PREFIX}.ml output should go into goodbams.goodsites.het
Would someone be able to confirm that interpretation?
To get this to work, I had to break it into multiple chunks:
First the full script, which only the angsd command worked -
for FF in `cat bams`
do
F=$(basename $FF)
F_PREFIX=${F/.bam/}
angsd -i $F -anc $GENOME_REF $FILTERS -GL 1 -doSaf 1 -doCounts 1 -out ${F/.bam/} && realSFS ${F/.bam/}.saf.idx | awk -v file=$F '{print file\"\t\"(\$1+\$2+\$3)\"\t\"\$2/(\$1+\$2+\$3)}' ${F/.bam/}.ml >> goodbams.goodsites.het
done
Then
for FF in `cat bams`;
do
F=$(basename $FF)
F_PREFIX=${F/.bam/}
realSFS ${F/.bam/}.saf.idx > ${F_PREFIX}.ml
done
then
for FF in `cat bams`;
do
F=$(basename $FF)
F_PREFIX=${F/.bam/}
eval "awk -v file=$F '{print file\"\t\"(\$1+\$2+\$3)\"\t\"\$2/(\$1+\$2+\$3)}' ${F/.bam/}.ml" >> goodbams.goodsites.het
done
All performed in indev mode
Related
Struggling with this...
for i in `cat services.txt`
do
if ! grep -q $i; then
echo " $i Is NOT Running"
else
echo " Checking $i on `hostname`..."
ps aux | grep -i $i | awk '{print $1, $11}'| cut -d' ' -f1-2| sort
echo -e " "
sleep 4
fi
done
The block just hangs - Ive not been able to capture the success/failure of grep
If a string in services.txt is NOT found ... the script hangs... Id like for grep to skip it if not found
services.txt contain just single words
thanks!
The reason your script hangs is beacuse the command grep -q $i is waiting for an input. You can try running that command separately in a shell and verify that it prompts for an input.
Changing your command to ps aux | grep -i $i in the if statement should fix your issue.
NOTE: ps aux | grep -i $i lists grep also as one of the process. Make sure you exclude that process by piping it to another grep ps aux | grep -i $i | grep -v 'grep'
here is the working code
checkservices() {
cat >$HOME/services.txt <<EOF
ganglia
hbase
hdfs
hive
hue
livy
mapred
test-missing-service
mysql
oozie
presto
spark
yarn
zeppelin
EOF
for i in `cat $HOME/services.txt`
do
if `ps -ef | grep ^$i | grep -v grep >/dev/null`
then
i=$(echo "$i" | awk '{print toupper($0)}')
echo "$i -- is running on `hostname`"
echo ""
sleep 2
else
i=$(echo "$i" | awk '{print tolower($0)}')
echo "$i -- IS NOT running on `hostname` error"
echo ""
fi
done
}
Hi I have written a c program that takes 3 integers as input:
./myprogram 1 2 3
and I am aiming to pipe data from a csv file into the input of the c program. I grab each line from the c program using:
for i in $(seq 1 `wc -l "test.csv" | awk '{print $1}'`); do sed -n $i'p' "test.csv"; done;
and then would like to pipe the output of this into my c program. I have tried doing:
for i in $(seq 1 `wc -l "test.csv" | awk '{print $1}'`); do sed -n $i'p' "test.csv"; done; | ./myprogram
however I get:
Line
bash: syntax error near unexpected token `|'
how do I pipe the output into my c program?
Thanks
It helps when you really try to understand error messages the shell gives you:
Line
bash: syntax error near unexpected token `|'
If you think about it, when you chain commands together in a pipeline, there is never a ; before a |, for example:
ls | wc -l
# and not: ls; | wc -l
Whatever comes after a ; is like an independent new command, as if you typed it on a completely new, clear command line. If you type | hello on a clear command line, you'll get the exact same error, because that's the exact same situation as ; | ... in your script, for example:
$ | hello
-bash: syntax error near unexpected token `|'
Others already answered this, but I also wanted to urge you to make other improvements in your script:
Always use $() instead of backticks, for example:
for i in $(seq 1 $(wc -l "test.csv" | awk '{print $1}')); ...
You didn't need the awk there, this would work just as well:
for i in $(seq 1 $(wc -l "test.csv")); ...
You could reduce your entire script to simply this, for the same effect:
./myprogram < test.csv
In the shell, it doesn't like an explicit line termination followed by a pipe (|). The pipe already delimits the commands. So you want:
for i in $(seq 1 `wc -l "test.csv" | awk '{print $1}'`); do sed -n $i'p' "test.csv"; done | ./myprogram
i am still writing a bash script and need your help !
I want to grep with two parameters, but it doesn't work.
This script stands in an until queue so it runs x times.
Please help me:
e=1
g=db_$((e++))
for call in $(cat /home/palyground/foo.xml |grep $HOME|grep db_$g| awk -F \" '{print $2}')
do
:
done
echo $call
My Problem is, that the $g is still standing by 1 and dont move to 2 3 eg.
I want to grep |grep db_1 in the first time in the second time |grep db_2 e.g.
Do you have any soulutions for this Problem ?
Try
g=$((++e))
for call in $(awk -F \" '$0~home && $0~db {print $2}' home="$HOME" db="db_$g" /home/palyground/foo.xml)
By changing to ++e it will starts with 1 instead of 0
So i get the right code now..
First time thank you very much Jotne and chepner !
The right code:
f=1
q=/home/playground/foo.xml
g=db_$((f++))
HOME_GREP=${DB_HOME}/${g}
var1=$(cat $q | grep ${HOME_GREP} )
var2=$(echo $var1| awk -F \" '{print $2}' )
echo $var2
#!/bin/bash
#OLDIFS=$IFS
IFS=$'\r'
fortune_lines=($(fortune | fold -w 90))
#Screen_Session=$"{mainscreen}"
Screen_Session=`screen -ls|grep "\."|grep "("|awk '{print $1}'`
Screen_OneLiner=$(screen -p 0 -S ${Screen_Session} -X stuff "`printf "${fortune_lines[#]}\r"`")
#IFS=$OLDIFS;
for var in "${Screen_OneLiner[#]}"
do
echo "${var}"
done
ok this script works (sorta). I need to at the string "say " to front of the entire array index. Currently I can only get it to print out "say " to the first line.
We need not complicate things with contraptions like arrays - we can let good old sed do the job.
#!/bin/bash
fortune_lines=$(fortune | fold -w 90 | sed 's/^/say /')
Screen_Session=`screen -ls|grep "\."|grep "("|awk '{print $1}'`
screen -p 0 -S $Screen_Session -X stuff "$fortune_lines"
First, I am not experienced in scripting, so be gentle with me
Anyway, I tried making a script for finding files by mime-type ( audio, video, text...etc), and here's the poor result I came up with.
#!/bin/bash
FINDPATH="$1"
FILETYPE="$2"
locate $FINDPATH* | while read FILEPROCESS
do
if file -bi "$FILEPROCESS" | grep -q "$FILETYPE"
then
echo $FILEPROCESS
fi
done
It works, but as you could guess, the performance is not so good.
So, can you guys help me make it better ? and also, I don't want to rely on files extensions.
Update:
Here's what I am using now
#!/bin/bash
FINDPATH="$1"
find "$FINDPATH" -type f | file -i -F "::" -f - | awk -v FILETYPE="$2" -F"::" '$2 ~ FILETYPE { print $1 }'
Forking (exec) is expensive. This runs the file command only once, so it is fast:
find . -print | file -if - | grep "what you want" | awk -F: '{print $1}'
or
locate what.want | file -if -
check man file
-i #print mime types
-f - #read filenames from the stdin
#!/bin/bash
find $1 | file -if- | grep $2 | awk -F: '{print $1}'
#!/usr/bin/env bash
mimetypes=$(sed -E 's/\/.*//g; /^$/d; /^#/d' /etc/mime.types | uniq)
display_help(){
echo "Usage: ${0##*/} [mimetype]"
echo "Available mimetypes:"
echo "$mimetypes"
exit 2
}
[[ $# -lt 1 ]] && display_help
ext=$(sed -E "/^${1}/!d; s/^[^ \t]+[ \t]*//g; /^$/d; s/ /\n/g" /etc/mime.types | sed -Ez 's/\n$//; s/\n/\\|/g; s/(.*)/\.*\\.\\(\1\\)\n/')
find "$PWD" -type f -regex "$ext"