In bash I have an array with path names, and I would like to replace each of them with different ones using sed, like so:
sed 's#^(.*?)master_repo(.*?)#\1"${SOME_REPO_NAME}"\2#g' <<< ${FULL_TGT_DIRS[${i}]}
A sample path name which is an element of the array would be:
/Volumes/munki/master_repo/pkgs/apps
I would like to replace the path name "master_repo" with for example "somedir", which is stored in $SOME_REPO_NAME, so I get:
/Volumes/munki/somedir/pkgs/apps
Or with built in string substitution:
for i in ${FULL_TGT_DIRS[#]}
do
FULL_TGT_DIRS[$i]=${FULL_TGT_DIRS[$i]/master_repo/$SOME_REPO_NAME}
#sed 's#^(.*?)master_repo(.*?)#\1"${SOME_REPO_NAME}"\2#g' <<< ${FULL_TGT_DIRS[${i}]}
done
I always get the following error when running my script:
> /usr/local/bin/repomgr: line 135:
> /Volumes/munki/master_repo/pkgs/apps: syntax error: operand expected
> (error token is "/Volumes/munki/master_repo/pkgs/apps")
I've tried using different separaters and sed options, as well as shuffling through different quote constellations. I don't write bash scripts on a daily basis so perhaps I'm missing something?
BTW, I run this on a Mac and therefore only have bash 3.2 at my disposal.
There's no need to use sed for this, bash has built-in string replacement in its parameter expansion.
var=/Volumes/munki/master_repo/pkgs/apps
$SOME_REPO_NAME=somedir
newvar=${var/master_repo/$SOME_REPO_NAME}
In a for-in loop, the variable gets set to the array elements, not the array indexes, so you shouldn't be using FULL_TGT_DIRS[$i] -- $i contains the pathname. So the loop should be:
for file in ${FULL_TGT_DIRS[#]}
do
file=${file/master_repo/$SOME_REPO_NAME}
# Do something with $file here
done
If you need to modify the array in place, you need a different loop for the indexes:
for ((i = 0; i < ${#FULL_TGT_DIRS[#]}; i++))
do
FULL_TGT_DIRS[$i]=${FULL_TGT_DIRS[$i]/master_repo/"$SOME_REPO_NAME"}
done
You can even go a step further using bashes own replacement:
for file in "${FULL_TGT_DIRS[#]/master_repo/somedir}"
do
...work on file variable here...
done
Related
am in a need of using an array to set the variable value for further manipulations from the output file.
scenario:
> 1. fetch the list from database
> 2. trim the column using sed to a file named x.txt (got specific value as that is required)
> 3. this file x.txt has the below output as
10000
20000
30000
> 4. I need to set a variable and assign the above values to it.
A=10000
B=20000
C=30000
> 5. I can invoke this variable A,B,C for further manipulations.
Please let me know how to define an array assigning to its variable from the output file.
Thanks.
In bash (starting from version 4.x) and you can use the mapfile command:
mapfile -t myArray < file.txt
see https://stackoverflow.com/a/30988704/10622916
or another answer for older bash versions: https://stackoverflow.com/a/46225812/10622916
I am not a big proponent of using arrays in bash (if your code is complex enough to need an array, it's complex enough to need a more robust language), but you can do:
$ unset a
$ unset i
$ declare -a a
$ while read line; do a[$((i++))]="$line"; done < x.txt
(I've left the interactive prompt in place. Remove the leading $
if you want to put this in a script.)
While I've handled this task in other languages easily, I'm at a loss for which commands to use when Shell Scripting (CentOS/BASH)
I have some regex that provides many matches in a file I've read to a variable, and would like to take the regex matches to an array to loop over and process each entry.
Regex I typically use https://regexr.com/ to form my capture groups, and throw that to JS/Python/Go to get an array and loop - but in Shell Scripting, not sure what I can use.
So far I've played with "sed" to find all matches and replace, but don't know if it's capable of returning an array to loop from matches.
Take regex, run on file, get array back. I would love some help with Shell Scripting for this task.
EDIT:
Based on comments, put this together (not working via shellcheck.net):
#!/bin/sh
examplefile="
asset('1a/1b/1c.ext')
asset('2a/2b/2c.ext')
asset('3a/3b/3c.ext')
"
examplearr=($(sed 'asset\((.*)\)' $examplefile))
for el in ${!examplearr[*]}
do
echo "${examplearr[$el]}"
done
This works in bash on a mac:
#!/bin/sh
examplefile="
asset('1a/1b/1c.ext')
asset('2a/2b/2c.ext')
asset('3a/3b/3c.ext')
"
examplearr=(`echo "$examplefile" | sed -e '/.*/s/asset(\(.*\))/\1/'`)
for el in ${examplearr[*]}; do
echo "$el"
done
output:
'1a/1b/1c.ext'
'2a/2b/2c.ext'
'3a/3b/3c.ext'
Note the wrapping of $examplefile in quotes, and the use of sed to replace the entire line with the match. If there will be other content in the file, either on the same lines as the "asset" string or in other lines with no assets at all you can refine it like this:
#!/bin/sh
examplefile="
fooasset('1a/1b/1c.ext')
asset('2a/2b/2c.ext')bar
foobar
fooasset('3a/3b/3c.ext')bar
"
examplearr=(`echo "$examplefile" | grep asset | sed -e '/.*/s/^.*asset(\(.*\)).*$/\1/'`)
for el in ${examplearr[*]}; do
echo "$el"
done
and achieve the same result.
There are several ways to do this. I'd do with GNU grep with perl-compatible regex (ah, delightful line noise):
mapfile -t examplearr < <(grep -oP '(?<=[(]).*?(?=[)])' <<<"$examplefile")
for i in "${!examplearr[#]}"; do printf "%d\t%s\n" $i "${examplearr[i]}"; done
0 '1a/1b/1c.ext'
1 '2a/2b/2c.ext'
2 '3a/3b/3c.ext'
This uses the bash mapfile command to read lines from stdin and assign them to an array.
The bits you're missing from the sed command:
$examplefile is text, not a filename, so you have to send to to sed's stdin
sed's a funny little language with 1-character commands: you've given it the "a" command, which is inappropriate in this case.
you only want to output the captured parts of the matches, not every line, so you need the -n option, and you need to print somewhere: the p flag in s///p means "print the [line] if a substitution was made".
sed -n 's/asset\(([^)]*)\)/\1/p' <<<"$examplefile"
# or
echo "$examplefile" | sed -n 's/asset\(([^)]*)\)/\1/p'
Note that this returns values like ('1a/1b/1c.ext') -- with the parentheses. If you don't want them, add the -r or -E option to sed: among other things, that flips the meaning of ( and \(
I am trying to make a array/list from a bash output and then I want to for loop it. I keep on getting Syntax error: "(" unexpected (expecting "done"). If I had to put it in python term, I want to break the string up by \n and then for loop it.
IFS=$'\n'
DELETE = ($($MEGACOPY --dryrun --reload --download --local $LOCALDIR --remote $REMOTEDIR | sed 's|F '$LOCALDIR'|'$REMOTEDIR'|g'))
unset IFS
# And remove it
for i in $DELETE; do
$MEGARM $i
done
First, shell is not python. Spaces around equal signs don't work:
DELETE = ($($MEGACOPY --dryrun --reload --download --local $LOCALDIR --remote $REMOTEDIR | sed 's|F '$LOCALDIR'|'$REMOTEDIR'|g'))
When the shell sees the above, it interprets DELETE as a program name and = as its first argument. The error that you see is because the shell was unable to parse the second argument.
Replace the above with:
DELETE=($("$MEGACOPY" --dryrun --reload --download --local "$LOCALDIR" --remote "$REMOTEDIR" | sed 's|F '"$LOCALDIR"'|'"$REMOTEDIR"'|g'))
Second, regarding the for loop, DELETE is an array and arrays have special syntax:
for i in "${DELETE[#]}"; do
"$MEGARM" "$i"
done
Notes:
Unless you want word splitting and pathname expansion, all shell variables should be inside double-quotes.
It is best practices to use lower or mixed case for variable names. The system uses all upper case variables for its name and you don't want to accidentally overwrite one of them.
I have two files:
correct.txt
the sky is blue
I like eat apple
.
.
and wrong.txt
the sky are blue
I like eat apple
.
.
.
There are a lot of lines in both files.
Now, I want to correct a third file using my search in the "wrong.txt"
to correct it using the "correct.txt".
I have created two files:
readarray -t correct_array < correct.txt
readarray -t wrong_array < wrong.txt
The file to be corrected is to_be_corrected.txt
This works:
for c in "${correct_array[#]}"
do
echo "$c"
done
I tried this
for e in "${correct_array[#]}"
do
sed -i.bak 's/$wrong_array[#]/$correct_array[#]/' to_be_corrected.txt
done
But this did not work.
How can I use sed with arrays?
You are using single quotes (') for your sed command, so the shell is not evaluating the variables $wrong_array[#] and $correct_array[#]. Try double quotes and braces on the variables. Also, you are using the entire array with ${correct_array[#]}. You need to pair the elements together, perhaps with an index:
for ((e=0; e<"${#correct_array[#]}"; ++e)); do
sed -i.bak "s/${wrong_array[$e]}/${correct_array[$e]}/" to_be_corrected.txt
done
This iterates e over the indexes of the array (${#correct_array[#]} gives the size of the array) then e is used to index the corresponding elements of wrong_array and correct_array. Hopefully you don't have any quotes (single or double) in your text files.
You should always use {} with arrays. This doesn't work:
$array[1]
But this will:
${array[1]}
As pointed out by e0k you should also use double quoted otherwise the variable won't be expanded to it's actual value.
Don't know what exaclty your array has, but I think you want to iterate it instead of use the whole thing. Try this approach:
for i in `seq 0 $((${#correct_array[#]}-1))`; do
sed -i.bak "s/${wrong_array[$i]}/${correct_array[$i]}/" to_be_corrected.txt
done
I am processing some folders that each represent a page of a book. E.g. "Iliad-001" would be Book=Iliad, Page=001.
I want to iterate through all of the folders, create an array for each book and add an entry to that array for each page that is found, so that I can echo ${Iliad[#]} at the end of my script and it will give me a nice list of all the pages it found.
The catch I'm having is adding values to an array with a dynamic name. Here's the code that I think is intuitive (but clearly not right):
for j in */; do
vol_name=$(basename "$j" | sed 's/\(.*\)-[0-9]*/\1/')
page_name=$(basename "$j" | sed 's/.*-\([0-9]*\)/\1/')
$vol_name+=( "$page_name" )
done
This returns:
syntax error near unexpected token `"$page_name"'
If I change the variable assignment to this $vol_name+="( "$page_name" )" I get a little closer:
Iliad+=( 001 ): command not found
I was able to make it work using eval.
BTW, you do not need to run sed.
#! /bin/bash
for j in */; do
j=$(basename "$j")
vol_name=${j%-*}
page_name=${j#*-}
eval "$vol_name+=('$page_name')"
done
echo ${Iliad[#]}
try this
declare $vol_name+=( "$page_name" )