I am running into problems when interating over files using a for-loop. For simplicity, I created a small loop which should explain which problem I have at the moment.
Starting point: files in a folder which have a file-specific one to three digit number at a defined position in their filename.
Goal: Iterate over some of these files (not all) using a for-loop.
Problem: I created an array containing these one to three digit numbers specific for each file. The files are called at the beginning of the for-loop and I would like to use the array to reference to the specific files. But: The array is not expanding correctly.
Hope someone can help!
(There might be several good alternative ways to do this. Maybe some of them do not need an array, but I would be interested in knowing the solution to my specific problem since I think this might be a fundamental missunderstanding in how to expand a variable as part of filenames at the beginning of a for-loop.)
This is the code:
declare -a SOME_SAMPLES=(37 132 253 642 242 42)
for d in prmrp_*_${SOME_SAMPLES[#]}_S*_L00?_R1_001.fastq.gz; do
INPUT_FILE1=$(echo $d | sed 's/_L00._R1_001.fastq.gz//')
echo ${INPUT_FILE1}
done
Again, this is just an example code. The problem is the ${SOME_SAMPLES[#]} part which is not expanding correctly so the loop fails.
Thanks!
I think the problem is that in
prmrp_*_${SOME_SAMPLES[#]}_S*_L00?_R1_001.fastq.gz
it doesn't duplicate the entire expression for each element of the array, it just blindly inserts the array's elements in the middle, giving the equivalent of this:
prmrp_*_37 132 253 642 242 42_S*_L00?_R1_001.fastq.gz
... which is a bunch of separate items (prmrp_*_37 as a wildcard expression, followed by 132 as a simple string, followed by 253 etc). AIUI you want to expand the array's contents, and then for each element use a wildcard expression to get all matching files. The best way to do this is to use two loops, one to expand the array, and another to find matching files:
for sample in "${SOME_SAMPLES[#]}"; do
for d in prmrp_*_"${sample}"_S*_L00?_R1_001.fastq.gz; do
...
BTW, I'd also recommend using lowercase or mixed-case variable names (e.g. sample above) to avoid possible conflicts with the many all-caps variables with special meanings/functions. Also, I'd use a parameter expansion to remove the filename's suffix (instead of sed):
input_file1=${d%_L00?_R1_001.fastq.gz}
Also, you should generally put double-quotes around variable references (e.g. echo "${input_file1}" instead of echo ${input_file1}). (Assignments like input_file1=${d... are an exception, although double-quotes don't hurt there; they just aren't needed.) Note that in the for loop above, I put double-quotes around the array and variable references, but not around the wildcards; this means the shell will expand the wildcards (as you want) but not mess with the variable's contents.
Try:
array=( 37 132 253 642 242 42 );
for d in ${array[#]}; do
INPUT_FILE1="prmrp_*_"$d"_S*_L00?_R1_001.fastq.gz";
echo ${INPUT_FILE1}
done
Related
I'm not an expert in bash coding and I'm trying to do one interative-like code to help me in my work.
I have a file that contains some numbers (coordinates), and I'm trying to make a code to read some specific numbers from the file and then store them in an array. Modify that array using some arithmetic operation and then replace the numbers in the original file with the modified array. So far I've done everything except replacing the numbers in the file, I tried using sed but it does not change the file. The original numbers are stored in an array called "readfile" and the new numbers are stored in an array called "d".
I'm trying to use sed in this way: sed -i 's/${readfile[$j]}/${d[$k]}/' file.txt
And I loop j and k to cover all the numbers in the arrays. Everything seems to work but the file is not being modified. After some digging, I'm noticing that sed is not reading the value of the array, but I do not know how to fix that.
Your help is really appreciated.
When a file isn't modified by sed -i, it means sed didn't find any matches to modify. Your pattern is wrong somehow.
After using " instead of ' so that the variables can actually be evaluated inside the string, look at the contents of the readfile array and check whether it actually matches the text. If it seems to match, look for special characters in the pattern, characters that would mean something specific to sed (the most common mistake is /, which will interfere with the search command).
The fix for special characters is either to (1) escape them, e.g. \/ instead of just /, or (2) (and especially for /) to use another delimiter for the search/replace command (instead of s/foo/bar/ you can use s|foo|bar| or s,foo,bar, etc - pretty much any delimiter works, so you can pick one that you know isn't in the pattern string).
If you post data samples and more of your script, we can look at where you went wrong.
I'm trying to create a for-loop to go through all the items from an array, and add the items to a string. The tags are given as a single string with format "tag1 tag2 tag3", and the tagging parameter can be given as many times as I want with the single command with syntax "-tag tag1 -tag -tag2 -tag tag3". I'm unable to create a for loop for the job, and I'm a little confused what is wrong with my code.
TAGS="asd fgh jkl zxc bnm" # Amount of tags varies, but there is always at least one
ARRAY=($TAGS)
TAGSTOBEADDED=""
for i in "$ARRAY[#]"
do
STRINGTOBEADDED="-tag ${ARRAY[$i]}"
$TAGSTOBEADDED=$TAGSTOBEADDED+$STRINGTOBEADDED
done
command $TAGSTOBEADDED
First, your array sintax is wrong as #oguz ismail said. To iter through array items you shold use this:
for i in "${ARRAY[#]}"; { echo $i;}
Second $TAGSTOBEADDED=$TAGSTOBEADDED+$STRINGTOBEADDED this is also fail.
Variables are set like so var="$var 123" you don't need $ in front of var name if you want to change it. Back to code. In this example you dont even need an array, just use TAGS var(without ""):
for i in $TAGS; { TAGSTOBEADDED+="-tag $i"; }
First: avoid storing lists of things in space-delimited strings (as you're currently doing with TAGS and TAGSTOBEADDED) -- there are a bunch of things that can go wrong if they have any "funny" characters (or if IFS gets changed). Use an array instead. Storing them as a string and then converting doesn't help; all of the same potential problems apply during the conversion.
I also recommend using lower- or mixed-case variable names in scripts, since there are a bunch of all-caps names with special meanings, and accidentally using one of those for something else can have weird effects. So, to define the array of tags, I'd just use this:
tags=(asd fgh jkl zxc bnm)
You also have a number of syntax errors in the script. In this line:
for i in "$ARRAY[#]"
... the shell will try to expand $ARRAY as a plain variable (not an array), and then treat "[#]" as just some unrelated characters that go after it. You need braces around the variable refence (like "${ARRAY[#]}") any time you're doing anything nontrivial with a variable reference. BTW, this idiom -- including double-quotes, braces, square-brackets and at-sign -- is what you almost always want when getting the contents of an array.
In this line:
STRINGTOBEADDED="-tag ${ARRAY[$i]}"
$i will expand to one of the array elements, not its index. That is, it'll expand to something like:
STRINGTOBEADDED="-tag ${ARRAY[asd]}"
...which doesn't make any sense. You just want
STRINGTOBEADDED="-tag $i"
...except you don't want that either, because (as I said before) storing lists of things space-delimited in a string is a bad idea. But I'll get to that because fixing it will involve the next line:
$TAGSTOBEADDED=$TAGSTOBEADDED+$STRINGTOBEADDED
There are two problems here: you don't want a dollar sign on the variable being assigned to ($varname gets the value of a variable; anytime you're setting it, don't use the $). Also the + isn't needed to add strings, you just stick them end to end. Well, you'd need to add a space in between, something like one of these:
TAGSTOBEADDED=$TAGSTOBEADDED" "$STRINGTOBEADDED
TAGSTOBEADDED="$TAGSTOBEADDED $STRINGTOBEADDED"
(Generally, you should have double-quotes around all variable references; on the right side of a plain assignment is one of the few places it's safe to leave them unquoted, but I tend to prefer to just double-quote always rather than try to remember all of the exceptions about where it's safe and where it isn't. Plus, quoting just the space looks weird.)
But you don't want to do that either, because (again) space-delimited strings are a bad way to do things. Use an array. So before the loop, create an empty array instead of an empty string:
tagstobeadded=()
...and then inside the loop, append to it with +=( ):
tagstobeadded+=(-tag "$i")
...and then at the end, use it with all the appropriate quotes, braces, etc:
command "${tagstobeadded[#]}"
So, with all of these changes, here's what I'd recommend:
tags=(asd fgh jkl zxc bnm)
tagstobeadded=()
for i in "${tags[#]}"
do
tagstobeadded+=(-tag "$i")
done
command "${tagstobeadded[#]}"
I want to make an array with string values that have square brackets. but every time I keep getting output unexpected.
selections=()
for i in $choices
do
selections+=("role[${filenames[$i]}]")
done
echo ${selections[#]}
If choices were 1 and 2, and the array filenames[1] and filenames[2] held the values 'A', 'B' I want the selections array to hold the strings role[A], and role[B]
instead the output I get is just roles.
I can make the code you presented produce the output you wanted, or not, depending on the values I assign to variables filenames and choices.
First, I observe that bash indexed arrays are indexed starting at 0, not 1. If you are using the values 1 and 2 as indices into array filenames, and if that is an indexed array with only two elements, then it may be that ${filenames[2]} expands to nothing. This would be the result if you initialize filenames like so:
# NOT WHAT YOU WANT:
filenames=(A B)
Instead, either assign array elements individually, or add a dummy value at index 0:
# Could work:
filenames=('' A B)
Next, I'm suspicious of choices. Since you're playing with arrays, I speculate that you may have initialized choices as an array, like so:
# NOT CONSISTENT WITH YOUR LATER USAGE:
choices=(1 2)
If you expand an array-valued variable without specifying an index, it is as if you specified index 0. With the above initialization, then, $choices would expand to just 1, not 1 2 as you intend. There are two possibilities: either initialize choices as a flat string:
# Could work:
choices='1 2'
or expand it differently:
# or expand it this way:
for i in "${choices[#]}"
. Do not overlook the quotes, by the way: that particular form will expand to one word per array element, but without the quotes the array elements would be subject to word splitting and other expansions (though that's moot for the particular values you're using in this case).
The quoting applies also, in general, to your echo command: if you do not quote the expansion then you have to analyze the code much more carefully to be confident that it will do what you intend in all cases. It will be subject not only to word splitting, but pathname expansion and a few others. In your case, there is a potential for pathname expansion to be performed, depending on the names of the files in the working directory (thanks #CharlesDuffy). It is far safer to just quote.
Anyway, here is a complete demonstration incorporating your code verbatim and producing the output you want:
#!/bin/bash
filenames=('' 'A' 'B')
choices="1 2"
selections=()
for i in $choices
do
selections+=("role[${filenames[$i]}]")
done
echo ${selections[#]}
# better:
# echo "${selections[#]}"
Output:
role[A] role[B]
Finally, as I observed in comments, there is no way that your code could output "roles", as you claim it does, given the inputs (variable values) you claim it has. If that's in fact what you see, then either it is not related to the code you presented at all, or your inputs are different than you claim.
I'm trying to get an array containing the full current directory path in zsh. I'm currently using
local pwd="${PWD/#$HOME/~}"
pwd_list=(${(s:/:)pwd})
Which works except for one problem, it treats the starting / as a directory split too. I'd like my array to be like
/
usr
lib
php
instead of
usr
lib
php
I can see 2 ways of doing this but I'm unaware of how to do either in zsh. The first idea is to simple do a push and force a new element to the beginning (after the split).
The second, would be to alter the split to ignore the first / when parsing.
How can I resolve this to get an accurate directory path with minimal overhead into an array?
do you really need the first /? Assuming you're using a script to use the results of that, can't you just cd / to just start from there?
Anyways... is this what you want?
local pwd="${PWD/#$HOME/~}"
pwd_list=(${(s:/:)pwd})
pwd_list=('/' $pwd_list)
I think you're thinking about it slightly wrong. If you split "/usr/lib/php" on "/", you should get four elements, the first of which is an empty string. If you join those array elements back together with "/", you get the original path. Trying to think of the first element of "/" means you're treating the splitting inconsistently, which will make everything else harder.
So the problem really is that you're only getting three elements instead of four: the empty first element is getting dropped. You can fix that by quoting, like this:
local pwd="${PWD/#$HOME/~}"
pwd_list=( "${(s:/:)pwd}" )
(The extra space next to the outer parentheses isn't necessary, but it makes it a little easier to read.) You can even combine that into one expression:
pwd_list=( "${(s:/:)${PWD/#$HOME/~}}" )
I have an array which contains
commName[0]="ls"
commName[1]="date"
commName[2]="crontab"
commName[3]="uname"
commName[4]="hostname"
Now the array doesn't always contain these. Sometimes it can have more indices sometimes less. And the values are not always ls,date,... They can be different. Bottom line, I don't know the size nor the values of the array when I'm coding.
Every array value ls,date,... has its own unique address. So for example, ls would have /home/test/ and date would have /home/test/test2/ etc... These addresses need to be stored into a variable which will be used later on in the code. So I should have following variables according to the given array
$lsAddress
$dateAddress
$crontabAddress
$unameAddress
$hostnameAddress
Therefore, I need a way to make these variables (have in mind that I don't know ls,date,uname,....)
My approach was this
for ((j=0 ; j<${#commName[#]} ; j++))
do
set commName[$j]Nick="hi"
echo $(${commName[$j]}Nick)
done
What I expected this to do was to create new variables for every index of the array and set them equal to hi (just for test purposes) and then access those new variables.
Also, The new created variables Must be accessible anywhere. So, I can't have a temporary variable that keeps getting replaced.
However, this method isn't working... Is there any other way I can do this?
Use eval. Try this:
for ((j=0 ; j<${#commName[#]} ; j++))
do
param=`echo ${commName[$j]}Nick`
eval "$param=hi1"
eval "echo \$$param"
done
Use two parallel arrays, so that the entry in the command array matches with the corresponding entry in the address array.
commName[0]="ls"
commName[1]="date"
commName[2]="crontab"
commName[3]="uname"
commName[4]="hostname"
commAddress[0]="/home/test/" # ls
commAddress[1]="/home/test/test2" # date
# etc
Then, when you have a particular value of i, you know that ${commName[i]} and ${commAddress[i]} go together.
I recommend the two arrays, but you might also consider using bash's indirect parameter expansion instead.
$ commName[0]="ls"
$ lsAddress="/home/test"
$ name="${commName[0]}Address"
$ echo "${!name}"
/home/test