i want to print all array data of foo line by line but this loop is
printing last line of string of array it is not printing all line of
array variable please help.
foo=( $(grep name emp.txt) )
while read -r line ; do echo "$line"
done <<< ${foo[#]}
While David C. Rankin presented a working alternative, he chose to not explain why the original approach didn't work. See the Bash Reference Manual: Word Splitting:
The shell scans the results of parameter expansion, command
substitution, and arithmetic expansion that did not occur within
double quotes for word splitting.
So, you can make your approach work by using double quotes around the command substitution as well as the parameter expansion:
foo=("$(grep name emp.txt)")
while read -r line; do echo "$line"
done <<<"${foo[#]}"
Note that this assigns the whole grep output to the sole array element ${foo[0]}, i. e., we don't need an array at all and could use a simple variable foo just as well.
If you do want to read the grep output lines into an array with one line per element, then there's the Bash Builtin Command readarray:
< <(grep name emp.txt) readarray foo
This uses the expansion Process Substitution.
i want to replace some text can i use sed command in echo "$line"
Of course you can use echo "$line" | sed ….
Related
I'm pretty new to Linux and I've been trying some learning recently. One thing I'm struggling is Within a log file I would like to grep for all the unique IDs that exist and store them in an array.
The format of the ids are like so id=12345678,
I'm struggling though to get these in to an array. So far I've tried a range of things, the below however
a=($ (grep -HR1 `id=^[0-9]' logfile))
echo ${#a[#]}
but the echo count is always returned as 0. So it is clear the populating of the array is not working. Have explored other pages online, but nothing seems to have a clear explanation of what I am looking for exactly.
a=($(grep -Eow 'id=[0-9]+' logfile))
a=("${a[#]#id=}")
printf '%s\n' "${a[#]}"
It's safe to split an unquoted command substitution here, as we aren't printing pathname expansion characters (*?[]), or whitespace (other than the new lines which delimit the list).
If this were not the case, mapfile -t a <(grep ...) is a good alternative.
-E is extended regex (for +)
-o prints only matching text
-w matches a whole word only
${a[#]#id=} strips the id suffix from each array element
Here is an example
my_array=()
while IFS= read -r line; do
my_array+=( "$line" )
done < <( ls )
echo ${#my_array[#]}
printf '%s\n' "${my_array[#]}"
It prints out 14 and then the names of the 14 files in the same folder. Just substitute your command instead of ls and you started.
Suggesting readarray command to make sure it array reads full lines.
readarray -t my_array < <(grep -HR1 'id=^[0-9]' logfile)
printf "%s\n" "${my_array[#]}"
I am very much new to bash and shell scripting. I have a string like
string="new string added -value:ABC-10"
Now what I want here is to get the string after -value:.
I am using bash version 5. List of Things tried
IFS methods but that is syntax error: redirection unexpected
array=(${string//:/ }) this kind of thing
while read -r line; do lines+=("$line"); done <<<"$string"
string='Paris, France, Europe';
readarray -td, a <<<"$string"; declare -p a;
Apart from the IFS every other solution is giving syntax error "(" expected "}" and I had tried every possible combination for it but no luck.
Any help is appreciated.
Thanks in advance.
This is direct with bash.
Give a try to :
printf "%s\n" "${string##*-value:}"
the man of bash:
${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is expanded to produce a pattern just as in pathname expansion, and matched against the expanded value of parameter using the rules described under
Pattern Matching below. If the pattern matches the beginning of the value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern
(the ``#'' case) or the longest matching pattern (the ``##'' case) deleted. If parameter is # or *, the pattern removal operation is applied to each positional parameter in turn, and the ex‐
pansion is the resultant list. If parameter is an array variable subscripted with # or *, the pattern removal operation is applied to each member of the array in turn, and the expansion is
the resultant list.
How about using cut or awk?
echo "$string" | awk -F '-value:' '{print $2}'
With cut you can only split on a single character, so with cut it would like like:
echo "$string" | cut -d ":" -f 2
Arrays are a nice feature, but don't use them if you don't need to. IMO, a simpler way (which also does error checking) is
if [[ $string =~ \ -value:(.*)$ ]]
then
part_after_value=${BASH_REMATCH[1]}
else
echo String does not contain a value parameter 1>&2
fi
If you want to treat the case that -value is present, but nothing comes after the colon, as an error, replace .* by .+.
With IFS and read
string="new string added -value:ABC-10"
IFS=: read -r junk value <<< "$string"
echo "$value"
With mapfile aka readarray which is a bash4+ feature.
mapfile -td: array <<< "$string"
echo "${array[#]:1}"
Which is also equivalent to
echo "${array[1]}"
I have a text file with with the following lines of text:
https://i.imgur.com/9iLSS35.png
Delete page: https://imgur.com/delete/SbmZQDht9Xk2NbW
https://i.imgur.com/9iLSS336.png
Delete page: https://imgur.com/delete/SbmZQDht9Xk2NbW
https://i.imgur.com/9iLSS37.png
Delete page: https://imgur.com/delete/SbmZQDht9Xk2NbW
I need to read the odd lines and add to a array in bash. I have tried the following code but it generates a variable and not an array as I need.
A=$(sed -n 1~2p ./upimagen.txt)
How can I read the odd lines of the text file and add each line to an array and then access separately?
The result would be:
${A[0]}
https://i.imgur.com/9iLSS35.png
${A[1]}
https://i.imgur.com/9iLSS336.png
...
Array assignment requires an outer set of parentheses.
A=($(sed -n 1~2p ./upimagen.txt))
This will split on all whitespace, though, not just newlines. It will also expand any globs that use * or ?. It's better to use readarray plus process substitution to ensure that each line is preserved as is:
readarray -t A < <(sed -n 1~2p ./upimagen.txt)
Just a slight variation on John's answer using awk. Using either readarray or the synonymous mapfile reading from a process substitution is probably your best bet, e.g.
readarray -t A < <(awk 'FNR % 2 {print $1}' ./upimagen.txt)
While there is nothing wrong with the direct initialization using command substitution, e.g. A=( $(...) ), if your files are large, or if you only need part of the range of lines, or say you have a custom function you want to run on every X number of elements read, mapfile/readarray offers a wide set of features over straight command substitution.
Using awk :
readarray -t A < <(awk 'NR%2' ./upimagen.txt)
I'm trying to read the lines of output from a subshell into an array, and I'm not willing to set IFS because it's global. I don't want one part of the script to affect the following parts, because that's poor practice and I refuse to do it. Reverting IFS after the command is not an option because it's too much trouble to keep the reversion in the right place after editing the script. How can I explain to bash that I want each array element to contain an entire line, without having to set any global variables that will destroy future commands?
Here's an example showing the unwanted stickiness of IFS:
lines=($(egrep "^-o" speccmds.cmd))
echo "${#lines[#]} lines without IFS"
IFS=$'\r\n' lines=($(egrep "^-o" speccmds.cmd))
echo "${#lines[#]} lines with IFS"
lines=($(egrep "^-o" speccmds.cmd))
echo "${#lines[#]} lines without IFS?"
The output is:
42 lines without IFS
6 lines with IFS
6 lines without IFS?
This question is probably based on a misconception.
IFS=foo read does not change IFS outside of the read operation itself.
Thus, this would have side effects, and should be avoided:
IFS=
declare -a array
while read -r; do
array+=( "$REPLY" )
done < <(your-subshell-here)
...but this is perfectly side-effect free:
declare -a array
while IFS= read -r; do
array+=( "$REPLY" )
done < <(your-subshell-here)
With bash 4.0 or newer, there's also the option of readarray or mapfile (synonyms for the same operation):
mapfile -t array < <(your-subshell-here)
In examples later added to your answer, you have code along the lines of:
lines=($(egrep "^-o" speccmds.cmd))
The better way to write this is:
mapfile -t lines < <(egrep "^-o" speccmds.cmd)
Are you trying to store the lines of the output in an array, or the words of each line?
lines
mapfile -t arrayname < <(your subshell)
This does not use IFS at all.
words
(your subshell) | while IFS=: read -ra words; do ...
The form var=value command args... puts the var variable into the environment of the command, and does not affect the current shell's environment.
I have a file called failedfiles.txt with the following content:
failed1
failed2
failed3
I need to use grep to return the content on each line in that file, and save the output in a list to be accessed. So I want something like this:
temp_list=$(grep "[a-z]" failedfiles.txt)
However, the problem with this is that when I type
echo ${temp_list[0]}
I get the following output:
failed1 failed2 failed3
But what I want is when I do:
echo ${temp_list[0]}
to print
failed1
and when I do:
echo ${temp_list[1]}
to print
failed2
Thanks.
#devnull's helpful answer explains why your code didn't work as expected: command substitution always returns a single string (possibly composed of multiple lines).
However, simply putting (...) around a command substitution to create an array of lines will only work as expected if the lines output by the command do not have embedded spaces - otherwise, each individual (whitespace-separated) word will become its own array element.
Capturing command output lines at once, in an array:
To capture the lines output by an arbitrary command in an array, use the following:
bash < 4 (e.g., on OSX as of OS X 10.9.2): use read -a
IFS=$'\n' read -rd '' -a linesArray <<<"$(grep "[a-z]" failedfiles.txt)"
bash >= 4: use readarray:
readarray -t linesArray <<<"$(grep "[a-z]" failedfiles.txt)"
Note:
<<< initiates a so-called here-string, which pipes the string to its right (which happens to be the result of a command substitution here) into the command on the left via stdin.
While command <<< string is functionally equivalent to echo string | command in principle, the crucial difference is that the latter creates subshells, which make variable assignments in command pointless - they are localized to each subshell.
An alternative to combining here-strings with command substitution is [input] process substitution - <(...) - which, simply put, allows using a command's output as if it were an input file; the equivalent of <<<"$(command)" is < <(command).
read: -a reads into an array, and IFS=$'\n' ensures that every line is considered a separate field and thus read into its own array element; -d '' ensures that ALL lines are read at once (before breaking them into fields); -r turns interpretation of escape sequence in the input off.
readarray (also callable as mapfile) directly breaks input lines into an array of lines; -t ensures that the terminating \n is NOT included in the array elements.
Looping over command output lines:
If there is no need to capture all lines in an array at once and looping over a command's output line by line is sufficient, use the following:
while IFS= read -r line; do
# ...
done < <(grep "[a-z]" failedfiles.txt)
IFS= ensures that each line is read unmodified in terms of whitespace; remove it to have leading and trailing whitespace trimmed.
-r ensures that the lines are read 'raw' in that substrings in the input that look like escape sequences - e.g., \t - are NOT interpreted as such.
Note the use of [input] process substitution (explained above) to provide the command output as input to the read loop.
You did not create an array. What you did was Command Substitution which would simply put the output of a command into a variable.
In order to create an array, say:
temp_list=( $(grep "[a-z]" failedfiles.txt) )
You might also want to refer to Guide on Arrays.
The proper and portable way to loop over lines in a file is simply
while read -r line; do
... something with "$line"
done <failedfiles.txt