How can I get just the filenames into an array using the cat command?
How I've been trying:
array=()
while IFS= read -r -d $'\0'; do
array+=("$REPLY")
done < <(cat /proc/swaps | grep "swap")
This either grabs all the information from the output into an array, or just doesn't work. How can I successfully get my expected output of [/swapfile, /dev/hda1, /some/other/swap] into an array form using the cat command?
readarray array < <(awk '/swap/{print $1}' /proc/swaps)
Bash introduced readarray in version 4 which can take the place of the while read loop. readarray is the solution you want.
here is the syntax
readarray variable < inputfile
echo "${variable[0]}" ' to print the first element in array
Related
This is how I am grabbing all the NVME volumes:
all_nvme_volumes=$(sudo nvme list -o json | jq .Devices[].DevicePath)
This how the output looks like:
"/dev/nvme0n1" "/dev/nvme1n1" "/dev/nvme2n1" "/dev/nvme3n1" "/dev/nvme4n1" "/dev/nvme6n1"
How do I loop thru them process them individually?
I tried for r in "${all_nvme_volumes[#]}"; do echo "Device Name: $r"; done but the output is Device Name: "/dev/nvme0n1" "/dev/nvme1n1" "/dev/nvme2n1" "/dev/nvme3n1" "/dev/nvme4n1" "/dev/nvme6n1"
which is one string instead of each element of array:
Populating a bash array with mapfile from null delimited raw output from jq:
#!/usr/bin/env bash
mapfile -d '' all_nvme_volumes < <(
sudo nvme list --output-format=json |
jq --join-output '.Devices[].DevicePath + "\u0000"'
)
A solution for bash < 4.4:
#!/bin/bash
IFS=$'\t' read -r -a all_nvme_volumes < <(
sudo nvme list -o json | jq -r '[ .Devices[].DevicePath ] | #tsv'
)
note: device paths shouldn't be escaped by #tsv, so you won't need to unescape the values, but in case you use this trick for other purposes, you can unescape a value with printf -v value '%b' "$value"
How do I loop thru them process them individually?
Well, once you have the array, you can loop though its elements with:
for nvme_volume in "${all_nvme_volumes[#]}"
do
# process "$nvme_volume"
done
But, if you only need to loop though the nvme volumes without storing them then you can use #LéaGris null delimiter method with a while loop:
#!/bin/bash
while IFS='' read -r -d '' nvme_volume
do
# process "$nvme_volume"
done < <(sudo nvme list -o json | jq -j '.Devices[].DevicePath + "\u0000"')
This question already has answers here:
How to split a string into an array in Bash?
(24 answers)
Closed 1 year ago.
Can somebody help me out. I want to split TEXT(variable with \n) into array in bash.
Ok, I have some text-variable:
variable='13423exa*lkco3nr*sw
kjenve*kejnv'
I want to split it in array.
If variable did not have new line in it, I will do it by:
IFS='*' read -a array <<< "$variable"
I assumed the third element should be:
echo "${array[2]}"
>sw
>kjenve
But with new line it is not working. Please give me right direction.
Use readarray.
$ variable='13423exa*lkco3nr*sw
kjenve*kejnv'
$ readarray -d '*' -t arr < <(printf "%s" "$variable")
$ declare -p arr
declare -a arr=([0]="13423exa" [1]="lkco3nr" [2]=$'sw\nkjenve' [3]="kejnv")
mapfile: -d: invavlid option
Update bash, then use readarray.
If not, replace separator with zero byte and read it element by element with read -d ''.
arr=()
while IFS= read -d '' -r e || [[ -n "$e" ]]; do
arr+=("$e")
done < <(printf "%s" "$variable" | tr '*' '\0');
declare -p arr
declare -a arr=([0]="13423exa" [1]="lkco3nr" [2]=$'sw\nkjenve' [3]="kejnv")
You can use the readarray command and use it like in the following example:
readarray -d ':' -t my_array <<< "a:b:c:d:"
for (( i = 0; i < ${#my_array[*]}; i++ )); do
echo "${my_array[i]}"
done
Where the -d parameter defines the delimiter and -t ask to remove last delimiter.
Use a ending character different than new line
end=.
read -a array -d "$end" <<< "$v$end"
Of course this solution suppose there is at least one charecter not used in your input variable.
In Bash, how can I get the strings between acolades (without the '_value' suffix) from for example
"\\*\\* ${host_name_value}.${host_domain_value} - ${host_ip_value}\\*\\*"
and put them into an array?
The result for the above example should be something like:
var_array=("host_name" "host_domain")
The string could also contain other stuff such as:
"${package_updates_count_value} ${package_updates_type_value} updates"
The result for the above example should be something like:
var_array=("package_updates_count" "package_updates_type")
All variables end with _value. There could 1 or more variables in the string.
Not sure what would be the most efficient way and how I'd best handle this. Regex? Sed?
input='\\*\\* ${host_name_value}.${host_domain_value} \\*\\*'
# would also work with cat input or the like.
myarray=($(echo "$input" | awk -F'$' \
'{for(i=1;i<=NF;i++) {match($i, /{([^}]*)_value}/, a); print a[1]}}'))
Split your line(s) on $. Check if a column contains { }. If it does, print what's after { and before _value}. (If not, it will print out the empty string, which bash array creation will ignore.)
If there are only two variables, this will work.
input='\\*\\* ${host_name_value}.${host_domain_value} \\*\\*'
first=$(echo $input | sed -r -e 's/[}].+//' -e 's/.+[{]//')
last=$(echo $input | sed -r -e 's/.+[{]//' -e 's/[}].+//')
output="var_array=(\"$first\" \"$last\")"
Maybe not very efficient and beautiful, but it works well.
Starting with a string variable:
$ str='\\*\\* ${host_name_value}.${host_domain_value} - ${host_ip_value}\\*\\*'
Use grep -o to print all matching words.
$ grep -o '\${\w*_value}' <<< "$str"
${host_name_value}
${host_domain_value}
${host_ip_value}
Then remove ${ and _value}.
$ grep -o '\${\w*_value}' <<< "$str" | sed 's/^\${//; s/_value}$//'
host_name
host_domain
host_ip
Finally, use readarray to safely read the results into an array.
$ readarray -t var_array < <(grep -o '\${\w*_value}' <<< "$str" | sed 's/^\${//; s/_value}$//')
$ declare -p var_array
declare -a var_array=([0]="host_name" [1]="host_domain" [2]="host_ip")
I am trying to store values of the first line of a text file into an array. Here is what I have so far:
arr_values=()
awk '
NR==1 {
for (i=0; i<=NF; i++)
'arr_values[i]'=$i
}' file.txt
for ((i=0; i<${#arr_values[#]}; i++))
do
echo arr_values[i]
done
I am getting an error with initializing the array mainly because I don't know how to use awk to initialize an external array. Any suggestions (only with awk)? Thanks.
You can do this:
read -a array <<< $(head -n 1 file)
echo ${array[0]}
echo ${array[1]}
You probably can simply just do
read -ra arr_values < file.txt
Which would only process the first line and split it uniformly like awk does; saving it into arr_values. No need to fork with external binary commands.
I am trying to remove certain lines from a huge file, getting line numbers to be deleted from an array. The file is at least 2GB in size and the my array size can be large as well. Can I do this without a for loop? What is fastest way?
Example:
input:
>1
>2
>3
>4
>5
declare -a A=(2 3 5);
output:
>1
>4
... getting line numbers to be deleted from an array.
If I understand it correct, your array A contains line numbers to be deleted from the input.
You could use sed:
sed $(printf "%dd;" "${A[#]}") inputfile
Use the -i option to modify the file in-place.
If the array is too large, consider using process substitution instead:
sed -f <(printf "%dd;" "${A[#]}") inputfile
I wouldn't to this in plain shell code. sed is the tool for editing/transforming files.
On-The-Fly create a sed-programm from your array and edit the INPUTFILE in-place (-i)
for line in ${A[#]}; do
echo ${line}d
done| sed -i -f /dev/stdin $INPUTFILE
You can use grep -vf to get this array differential:
declare -a O=(1 2 3 4 5)
declare -a A=(2 3 5)
B=( $(grep -vf <(printf "%s\n" "${A[#]}") <(printf "%s\n" "${O[#]}")) )
OUTPUT:
declare -p B
declare -a B='([0]="1" [1]="4")'
printf "%s\n" "${B[#]}"
1
4
awk -v n=2,3,5 'BEGIN{split(n,nn,",")} !(NR in nn) {print}' input >output
In the above, the list of lines to be deleted is provided as the variable n. (I have it shown as a comma-separated format but other formats are possible.) In the BEGIN block, this list is converted to an awk array called nn. The remainder of the awk program simply prints all lines whose line number, NR, is not in the array of lines to be excluded, nn.
If awk implements its membership testing in a properly hashed fashion, the way python does it, then the above should be fast. If not, not.