I have this as a csv file:
Column1,Column2,Column3,Column4
wow,this,is,awesomeee
we,are,going,to
what,is,the,name
This is by script:
names=($(awk < example_list.csv -F, '{print $2}'))
lines=($(awk < example_list.csv -F, '{print $0 $1 $2 $3 }'))
for i in "${names[#]}"
do
if [ ${names[i]}=="is" ]
then
echo ${lines[i]}
fi
done
My goal is to find a match in the third column (index 2) and print the whole row out based on that matching index.
In this case:
I save the whole third column to an array (names)
I save all columns and rows to a second array (lines)
I then iterate through array names, looking for a matching word "is"
If it is found, I print out the whole row from array list based on that matching index.
Unfortunately, this does not work. All this prints out is:
Column1,Column2,Column3,Column4Column1Column2Column3
Column1,Column2,Column3,Column4Column1Column2Column3
Column1,Column2,Column3,Column4Column1Column2Column3
Column1,Column2,Column3,Column4Column1Column2Column3
You're making it too complicated, you can just use this awk to print full record when 3rd field is "is":
awk -F, '$3=="is"' file
wow,this,is,awesomeee
Related
I put the output of find command into array like this:
pathList=($(find /foo/bar/ -type d))
How to extract the longest paths found in the array if the array contains several equal-length longest paths?:
echo ${pathList[#]}
/foo/bar/raw/
/foo/bar/raw/2020/
/foo/bar/raw/2020/02/
/foo/bar/logs/
/foo/bar/logs/2020/
/foo/bar/logs/2020/02/
After extraction, I would like to assign /foo/bar/raw/2020/02/ and /foo/bar/logs/2020/02/ to another array.
Thank you
Could you please try following. This should print the longest array(could be multiple in numbers same maximum length ones), you could assign it to later an array to.
echo "${pathList[#]}" |
awk -F'/' '{max=max>NF?max:NF;a[NF]=(a[NF]?a[NF] ORS:"")$0} END{print a[max]}'
I just created a test array with values provided by you and tested it as follows:
arr1=($(printf '%s\n' "${pathList[#]}" |\
awk -F'/' '{max=max>NF?max:NF;a[NF]=(a[NF]?a[NF] ORS:"")$0} END{print a[max]}'))
When I see new array's contents they are as follows:
echo "${arr1[#]}"
/foo/bar/raw/2020/02/
/foo/bar/logs/2020/02/
Explanation of awk code: Adding detailed explanation for awk code.
awk -F'/' ' ##Starting awk program from here and setting field separator as / for all lines.
{
max=max>NF?max:NF ##Creating variable max and its checking condition if max is greater than NF then let it be same else set its value to current NF value.
a[NF]=(a[NF]?a[NF] ORS:"")$0 ##Creating an array a with index of value of NF and keep appending its value with new line to it.
}
END{ ##Starting END section of this program.
print a[max] ##Printing value of array a with index of variable max.
}'
I have a text file that has 8000 lines, here is an example
00122;IL;Chicago;Router;;1496009459
00133;IL;Chicago;Router;0;6.651;1496009460
00166;IL;Chicago;Router;0;5.798;1496009460
00177;IL;Chicago;Router;0;5.365;1496009460
00188;IL;Chicago;Router;0;22.347;1496009460
As you can see the file has different count of delimiter, I need to insert all columns separated by ';' to an array no matter when the the delimiter occurs
So the first line would have 6 fields and the second line would have 7.
When I tried do it through the below command
Number=( $(awk '{print $1}' $FileName.txt) ) with different array name and field for each columns, I am getting strange behavior which not all fields are printed for some lines when I echo them all in one line
Performance is very important (need to do it in a matter of seconds )and I found using awk is the fastest approach so far, unless someone has better approach.
An ideas why this is happening ?
To dump the entire text file into an array, I would use the following. In this example, we use the two arrays ${finalarray[]} and ${subarray[]} (though the latter is unset at the end) and the variable $line. We assume the file name is file.txt.
#!/bin/bash
finalarray=()
while read line; do #For each cycle, the variable $line is the next line of the file
if [[ -z $line ]]; then continue; done #If the line is empty, skip this cycle
IFS=";" read -r -a subarray <<< "$line" #Split $line into ${subarray[]} using : as delim
finalarray+=( "$subarray[#]}" ) #Add every element from ${subarray[]} to ${finalarray[]}
unset subarray #clears the array
done <file.txt
If your empty lines are, in fact, populated by spaces or other whitespace characters, the empty line catch won't work. Instead, you could use something like the following to skip any lines not containing semicolons.
if [[ $(echo "$line" | grep -c ";") -eq 0 ]]; then continue; fi
On the other hand, this would skip all lines without a semicolon, even if you intended some of those lines to be a single array entry.
I have an issue where a user gives the file, column, value, and id of the line. I am trying to change the value of the line
The format of the file is:
F1|F2|F3|F4|F5|F6|F7|F8
My thought of doing that is reading the file and put the values of each field in an array. Then I will find the line I want to change using if and I will use awk
while IFS=$'|t' read -r -a myArray
do
if [ $4 == ${myArray[0]} ]; then
echo "${myArray[1]} ${myArray[2]} ${myArray[4]}"
awk -v column="$5" -v value="$6"-F '{ ${myArray[column]} = value }'
echo "${myArray[1]} ${myArray[2]} ${myArray[4]}"
echo "${column} ${value}"
fi
done < $2
However, when I do that nothing changes: the column and value arguments don't print anything.
Any ideas?
You didnt give too much information. Assume you want to change specific column which field2 is F2 you can do as below:
$2=="F2" is checking field 2 is matching your specific string.
$2="Hello" is assigning "Hello" to field 2
$1=$1 reassign the whole record(line)
print print out the whole record
awk -F"|" 'BEGIN{OFS="|"} ($2=="F2"){$2="Hello";$1=$1; print}' sample.csv
See my example:
$cat sample.csv
F1|F2|F3|F4|F5|F6|F7|F8
$awk -F"|" 'BEGIN{OFS="|"} ($2=="F2"){$2="Hello";$1=$1; print}' sample.csv
F1|Hello|F3|F4|F5|F6|F7|F8
Im trying to put the contents of a awk command in to a bash array however im having a bit of trouble.
>>test.sh
f_checkuser() {
_l="/etc/login.defs"
_p="/etc/passwd"
## get mini UID limit ##
l=$(grep "^UID_MIN" $_l)
## get max UID limit ##
l1=$(grep "^UID_MAX" $_l)
awk -F':' -v "min=${l##UID_MIN}" -v "max=${l1##UID_MAX}" '{ if ( $3 >= min && $3 <= max && $7 != "/sbin/nologin" ) print $0 }' "$_p"
}
...
Used files:
Sample File: /etc/login.defs
>>/etc/login.defs
### Min/max values for automatic uid selection in useradd
UID_MIN 1000
UID_MAX 60000
Sample File: /etc/passwd
>>/etc/passwd
root:x:0:0:root:/root:/usr/bin/zsh
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
admin:x:1000:1000:Administrator,,,:/home/admin:/bin/bash
daniel:x:1001:1001:Daniel,,,:/home/daniel:/bin/bash
The output looks like:
admin:x:1000:1000:Administrator,,,:/home/admin:/bin/bash
daniel:x:1001:1001:User,,,:/home/user:/bin/bash
respectively (awk ... print $1 }' "$_p")
admin
daniel
Now my problem is to save the awk output in an Array to use it as variable.
>>test.sh
...
f_checkuser
echo "Array items and indexes:"
for index in ${!LOKAL_USERS[*]}
do
printf "%4d: %s\n" $index ${array[$index]}
done
It could/should look like this example.
Array items and indexes:
0: admin
1: daniel
Specially i would become all Users of a System (not root,bin,sys,ssh,...) without blocked users in an array.
Perhaps someone has another idea to solve my Problem?
Are you trying to set the output of one script to an array? There is a bash has a way of doing this. For example,
a=( $(seq 1 10) ); echo ${a[1]}
will populate the array a with elements 1 to 10 and will print 2, the second line generated by seq (array index starts at zero). Simply replace the contents of $(...) with your script.
For those coming to this years later ...
bash 4 introduced readarray (aka mapfile) exactly for this purpose.
See also Bash capturing output of awk into array
One solution that works:
array=()
f_checkuser(){
...
...
tempfile="localuser.tmp"
touch ${tempfile}
awk -F':'...'{... print $1 }' "$_p" > ${HOME}/${tempfile}
getArrayfromFile "${tempfile}"
}
getArrayfromFile() {
i=0
while read line # Read a line
do
array[i]=$line # Put it into the array
i=$(($i + 1))
done < $1
}
f_checkuser
echo "Array items and indexes:"
for index in ${!array[*]}
do
printf "%4d: %s\n" $index ${array[$index]}
done
Output:
Array items and indexes:
0: daniel
1: admin
But I like more to observe without a new temp-file.
So, have someone any another idea without a temp-file?
I'm trying to extract those lines from data text-file:
first
second
third
fourth
fifth
sixth
based on their lines numbers pre-saved in nums text-file:
2
6
3
and I ended up by the following awk solution but it gives me the lines which are sorted based on their line-numbers and not according to orders in nums file.
$ awk 'NR==FNR{lines[$0];next } FNR in lines' nums data
second
third
sixth
But what I want to achieve is below output:
second
sixth
third
So, is there any option for awk to prevent/disable it from sorting the array?
Handle the files the other way around:
awk 'NR == FNR { line[NR] = $0; next } { print line[$1] }' data nums