I am using tcsh. I want to output:
mkdir dir1 dir2 dir3 dir4
Where of course the maximum number of dirs is variable. The point is they contain the incrementing variable from a loop in their names. I have a feeling I should use xargs. I just want to be able to use a simple loop, and do it all in one line if possible. Is this possible? My feeling is it will be something like:
loop here | xargs mkdir
but I am just not able to make my syntax work.
edit: I figured out how to do it with multiple lines of input. Anyone know how to make the following into a single line input?
for i in {1..5}
do
echo -n " dir$i"
done \
| xargs mkdir
Code:
foreach a ( `seq 1 1 10` )
mkdir dir$a
end
or
seq -f 'dir%.0f' -s ' ' 1 1 10 | xargs mkdir
Related
I have a problem with “basename” command as follow:
In my host directory I have two samples’ fastq.gz files, named as:
A29_WES_S3_R1_001.fastq.gz
A29_WES_S3_R2_001.fastq.gz
A30_WES_S1_R1_001.fastq.gz
A30_WES_S1_R2_001.fastq.gz
Now I need to have their basename without suffix like:
A29_WES_S3_R1_001
A29_WES_S3_R2_001
A30_WES_S1_R1_001
A30_WES_S1_R2_001
I used the bash pipeline as follow:
#!/bin/bash
FILES1=(*R1_001.fastq.gz)
FILES2=(*R2_001.fastq.gz)
read1="${FILES1[#]}"
read2="${FILES2[#]}"
Ffile=$read1
Ffileprevix=$(basename "$Ffile" .fastq.gz)
Mfile=$read2
Mfileprevix=$(basename "$Mfile" .fastq.gz)
echo $Ffileprevix
echo $Mfileprevix
exit;
But every time I just get this output:
A29_WES_S3_R1_001.fastq.gz A30_WES_S1_R1_001
A29_WES_S3_R2_001.fastq.gz A30_WES_S1_R2_001
Only the last file (A30) would be included in the command!
I checked my pipeline in this way:
echo $read1
echo $read2
The result:
A29_WES_S3_R1_001.fastq.gz A30_WES_S1_R1_001.fastq.gz
A29_WES_S3_R2_001.fastq.gz A30_WES_S1_R2_001.fastq.gz
Then I did:
echo $Ffile
echo $Mfile
The result:
A29_WES_S3_R1_001.fastq.gz A30_WES_S1_R1_001.fastq.gz
A29_WES_S3_R2_001.fastq.gz A30_WES_S1_R2_001.fastq.gz
So $read1, $read2, $Ffile, and $Mfile work well.
Then I put “-a” in my basename command as it will take multiple files:
Ffileprevix=$(basename -a "$Ffile" .fastq.gz)
Mfileprevix=$(basename -a "$Mfile" .fastq.gz)
But it got worse! The result was like:
A29_WES_S3_R1_001.fastq.gz A30_WES_S1_R1_001.fastq.gz .fastq.gz
A29_WES_S3_R2_001.fastq.gz A30_WES_S1_R2_001.fastq.gz .fastq.gz
Finally, I tried “for ..... do ....” command to make a loop for basename command. Again, nothing changed!!
Is there anybody can help me to obtain what I want:
A29_WES_S3_R1_001
A29_WES_S3_R2_001
A30_WES_S1_R1_001
A30_WES_S1_R2_001
I'd leave basename out of this entirely, but that's entirely personal preference. You could do something more like:
FILES_PATTERN_1=".*R1_001.fastq.gz"
FILES_PATTERN_2=".*R2_001.fastq.gz"
# Get FILE PATTERN 1
echo "Pattern 1:"
for FILE in $(find . | grep "${FILES_PATTERN_1}" | cut -d. -f2 | tr -d /); do
echo $FILE
done
# Get FILE PATTERN 2
echo "Pattern 2:"
for FILE in $(find . | grep "${FILES_PATTERN_2}" | cut -d. -f2 | tr -d /); do
echo $FILE
done
Output should be:
Pattern 1:
A30_WES_S1_R1_001
A29_WES_S3_R1_001
Pattern 2:
A29_WES_S3_R2_001
A30_WES_S1_R2_001
You could also play with awk to parse things instead:
# Get FILE PATTERN 1
echo "Pattern 1:"
for FILE in $(find . | grep "${FILES_PATTERN_1}" | awk -F '[/.]' '{print $3}'); do
echo $FILE
done
There are a number of ways to approach this. If you had a lot more patterns to test you could make more use of functions here to reduce code duplication.
Also note, I'm doing this from a shell on Mac OSX, so if you're doing this from a Linux box some of these commands may need to be tweaked due to differences in output for some commands, like find. (ex: print $1 instead of print $3)
I have the following problem. Let´s assume that $# contains only valid files. Variable file contains the name of the current file (the file I'm currently "on"). Then variable element contains data in the format file:function.
Now, when variable element is not empty, it should be put into the array. And that's the problem. If I echo element, it contains exactly what I want, although it is not stored in array, so for cycle doesn't print out anything.
I have written two ways I try to insert element into array, but neither works. Can you tell me, What am I doing wrong, please?
I'm using Linux Mint 16.
#!/bin/bash
nm $# | while read line
do
pattern="`echo \"$line\" | sed -n \"s/^\(.*\):$/\1/p\"`"
if [ -n "$pattern" ]; then
file="$pattern"
fi
element="`echo \"$line\" | sed -n \"s/^U \([0-9a-zA-Z_]*\).*/$file:\1/p\"`"
if [ -n "$element" ]; then
array+=("$element")
#array[$[${#array[#]}+1]]="$element"
echo element - "$element"
fi
done
for j in "${array[#]}"
do
echo "$j"
done
Your problem is that the while loop runs in a subshell because it is the second command in a pipeline, so any changes made in that loop are not available after the loop exits.
You have a few options. I often use { and } for command grouping:
nm "$#" |
{
while read line
do
…
done
for j in "${array[#]}"
do
echo "$j"
done
}
In bash, you can also use process substitution:
while read line
do
…
done < <(nm "$#")
Also, it is better to use $(…) in place of back-quotes `…` (and not just because it is hard work getting back quotes into markdown text!).
Your line:
element="`echo \"$line\" | sed -n \"s/^U \([0-9a-zA-Z_]*\).*/$file:\1/p\"`"
could be written:
element="$(echo "$line" | sed -n "s/^U \([0-9a-zA-Z_]*\).*/$file:\1/p")"
or even:
element=$(echo "$line" | sed -n "s/^U \([0-9a-zA-Z_]*\).*/$file:\1/p")
It really helps when you need them nested. For example, to list the lib directory adjacent to where gcc is found:
ls -l $(dirname $(dirname $(which gcc)))/lib
vs
ls -l `dirname \`dirname \\\`which gcc\\\`\``/lib
I know which I find easier!
I am struggling with passing several grep patterns that are contained within a variable. This is the code I have:
#!/bin/bash
GREP="$(which grep)"
GREP_MY_OPTIONS="-c"
for i in {-2..2}
do
GREP_MY_OPTIONS+=" -e "$(date --date="$i day" +'%Y-%m-%d')
done
echo $GREP_MY_OPTIONS
IFS=$'\n'
MYARRAY=( $(${GREP} ${GREP_MY_OPTIONS} "/home/user/this path has spaces in it/"*"/abc.xyz" | ${GREP} -v :0$ ) )
This is what I wanted it to do:
determine/define where grep is
assign a variable (GREP_MY_OPTIONS) holding parameters I will pass to grep
assign several patterns to GREP_MY_OPTIONS
using grep and the patterns I have stored in $GREP_MY_OPTIONS search several files within a path that contains spaces and hold them in an array
When I use "echo $GREP_MY_OPTIONS" it is generating what I expected but when I run the script it fails with an error of:
/bin/grep: invalid option -- ' '
What am I doing wrong? If the path does not have spaces in it everything seems to work fine so I think it is something to do with the IFS but I'm not sure.
If you want to grep some content in a set of paths, you can do the following:
find <directory> -type f -print0 |
grep "/home/user/this path has spaces in it/\"*\"/abc.xyz" |
xargs -I {} grep <your_options> -f <patterns> {}
So that <patterns> is a file containing the patterns you want to search for in each file from directory.
Considering your answer, this shall do what you want:
find "/path\ with\ spaces/" -type f | xargs -I {} grep -H -c -e 2013-01-17 {}
From man grep:
-H, --with-filename
Print the file name for each match. This is the default when
there is more than one file to search.
Since you want to insert the elements into an array, you can do the following:
IFS=$'\n'; array=( $(find "/path\ with\ spaces/" -type f -print0 |
xargs -I {} grep -H -c -e 2013-01-17 "{}") )
And then use the values as:
echo ${array[0]}
echo ${array[1]}
echo ${array[...]}
When using variables to pass the parameters, use eval to evaluate the entire line. Do the following:
parameters="-H -c"
eval "grep ${parameters} file"
If you build the GREP_MY_OPTIONS as an array instead of as a simple string, you can get the original outline script to work sensibly:
#!/bin/bash
path="/home/user/this path has spaces in it"
GREP="$(which grep)"
GREP_MY_OPTIONS=("-c")
j=1
for i in {-2..2}
do
GREP_MY_OPTIONS[$((j++))]="-e"
GREP_MY_OPTIONS[$((j++))]=$(date --date="$i day" +'%Y-%m-%d')
done
IFS=$'\n'
MYARRAY=( $(${GREP} "${GREP_MY_OPTIONS[#]}" "$path/"*"/abc.xyz" | ${GREP} -v :0$ ) )
I'm not clear why you use GREP="$(which grep)" since you will execute the same grep as if you wrote grep directly — unless, I suppose, you have some alias for grep (which is then the problem; don't alias grep).
You can do one thing without making things complex:
First do a change directory in your script like following:
cd /home/user/this\ path\ has\ spaces\ in\ it/
$ pwd
/home/user/this path has spaces in it
or
$ cd "/home/user/this path has spaces in it/"
$ pwd
/home/user/this path has spaces in it
Then do what ever your want in your script.
$(${GREP} ${GREP_MY_OPTIONS} */abc.xyz)
EDIT :
[sgeorge#sgeorge-ld stack1]$ ls -l
total 4
drwxr-xr-x 2 sgeorge eng 4096 Jan 19 06:05 test tesd
[sgeorge#sgeorge-ld stack1]$ cat test\ tesd/file
SUKU
[sgeorge#sgeorge-ld stack1]$ grep SUKU */file
SUKU
EDIT :
[sgeorge#sgeorge-ld stack1]$ find */* -print | xargs -I {} grep SUKU {}
SUKU
I want to write a shell script to show a list of directories entered by a user and then for a user to select one of the directories with an index number based on how many directories there are
I'm thinking this is some kind of array operation, but im not sure how to do this in shell script
example:
> whichdir
There are 3 dirs in the current path
1 dir1
2 dir2
3 dir3
which dir do you want?
> 3
you selected dir3!
$ ls -a
./ ../ .foo/ bar/ baz qux*
$ shopt -s dotglob
$ shopt -s nullglob
$ array=(*/)
$ for dir in "${array[#]}"; do echo "$dir"; done
.foo/
bar/
$ for dir in */; do echo "$dir"; done
.foo/
bar/
$ PS3="which dir do you want? "
$ echo "There are ${#array[#]} dirs in the current path"; \
select dir in "${array[#]}"; do echo "you selected ${dir}"'!'; break; done
There are 2 dirs in the current path
1) .foo/
2) bar/
which dir do you want? 2
you selected bar/!
Array syntax
Assuming you have the directories stored in an array:
dirs=(dir1 dir2 dir3)
You can get the length of the array thusly:
echo "There are ${#dirs[#]} dirs in the current path"
You can loop through it like so:
let i=1
for dir in "${dirs[#]}"; do
echo "$((i++)) $dir"
done
And assuming you've gotten the user's answer, you can index it as follows. Remember that arrays are 0-based so the 3rd entry is index 2.
answer=2
echo "you selected ${dirs[$answer]}!"
Find
How do you get the file names into an array, anyways? It's a bit tricky. If you have find that might be the best way:
readarray -t dirs < <(find . -maxdepth 1 -type d -printf '%P\n')
The -maxdepth 1 stops find from looking through subdirectories, -type d tells it to find directories and skip files, and -printf '%P\n' tells it to print the directory names without the leading ./ it normally likes to print.
#! /bin/bash
declare -a dirs
i=1
for d in */
do
dirs[i++]="${d%/}"
done
echo "There are ${#dirs[#]} dirs in the current path"
for((i=1;i<=${#dirs[#]};i++))
do
echo $i "${dirs[i]}"
done
echo "which dir do you want?"
echo -n "> "
read i
echo "you selected ${dirs[$i]}"
Update: my answer is wrong
Leaving it here to address a common misunderstanding, below the line is erroneous.
To put the directories in an array you can do...
array=( $( ls -1p | grep / | sed 's/^\(.*\)/"\1"/') )
This will capture the dir names, including those with spaces.
Extracting from comments:
literal quotes don't have any effect on string-splitting, so array=( echo '"hello world" "goodbye world"' ) is an array with four elements, not two
#Charles Duffy
Charles also supplied the following link Bash FAQ #50 which is an extended discussion on this issue.
I should also draw attention to the link posted by #Dennis Williamson - why I shouldn't have used ls
Need to process files in current directory one at a time. I am looking for a way to take the output of ls or find and store the resulting value as elements of an array. This way I can manipulate the array elements as needed.
To answer your exact question, use the following:
arr=( $(find /path/to/toplevel/dir -type f) )
Example
$ find . -type f
./test1.txt
./test2.txt
./test3.txt
$ arr=( $(find . -type f) )
$ echo ${#arr[#]}
3
$ echo ${arr[#]}
./test1.txt ./test2.txt ./test3.txt
$ echo ${arr[0]}
./test1.txt
However, if you just want to process files one at a time, you can either use find's -exec option if the script is somewhat simple, or you can do a loop over what find returns like so:
while IFS= read -r -d $'\0' file; do
# stuff with "$file" here
done < <(find /path/to/toplevel/dir -type f -print0)
for i in `ls`; do echo $i; done;
can't get simpler than that!
edit: hmm - as per Dennis Williamson's comment, it seems you can!
edit 2: although the OP specifically asks how to parse the output of ls, I just wanted to point out that, as the commentators below have said, the correct answer is "you don't". Use for i in * or similar instead.
You actually don't need to use ls/find for files in current directory.
Just use a for loop:
for files in *; do
if [ -f "$files" ]; then
# do something
fi
done
And if you want to process hidden files too, you can set the relative option:
shopt -s dotglob
This last command works in bash only.
Depending on what you want to do, you could use xargs:
ls directory | xargs cp -v dir2
For example. xargs will act on each item returned.