I have to write a script, which gets a text file which contains 9 line and 9 number (1-9) in the lines.
For example:
123456789
234567891
345678912
456789123
567891234
678912345
789123456
891234567
912345678
I have to check lines and the rows based on the sudoku rules.
Can't be same number in a line or in a row.
So the example is a correct solution.
How to write it in bash shell script?
I cannot use arrays, so what can I do?
A solution without awk, combining different other tools.
EDIT: This solution works for an inputfile called input and spaces between the digits. See comments about changing this behaviour.
echo "Checking 9 lines"
if [ $(wc -l <input ) -ne 9 ]; then
echo "Wrong number of lines"
exit 1
fi
echo "Check for correct layout"
if [ $(grep -cv '^[1-9] [1-9] [1-9] [1-9] [1-9] [1-9] [1-9] [1-9] [1-9]$' input ) -ne 0 ]; then
echo "Not all lines are correct, maybe spaces at the end of a line?"
grep -v '^[1-9] [1-9] [1-9] [1-9] [1-9] [1-9] [1-9] [1-9] [1-9]$' input
exit 1
fi
i=0
while read -r line; do
((i++))
if [ $(echo "${line}" | tr " " "\n" | sort -u | wc -l ) -ne 9 ]; then
echo "Wrong nr of unique numbers in row $i"
fi
done <input
for j in {1..9}; do
if [ $(cut -d" " -f${j} input | sort -u | wc -l ) -ne 9 ]; then
echo "Wrong nr of unique numbers in column $j"
fi
done
Related
In bash, I know to be able to find the unique values between two arrays can be found by:
echo "${array1[#]} ${array2[#]}" | tr ' ' '\n' | sort | uniq -u
However, this gives the unique values between BOTH arrays. What if I wanted something about the elements that are unique only to array1 and elements that are unique only to array2? For example:
array1=(1 2 3 4 5)
array2=(2 3 4 5 6)
original_command_output = 1 6
new_command_output1 = 1
new_command_output2 = 6
You could use the comm command.
To get elements unique to the first array:
comm -23 \
<(printf '%s\n' "${array1[#]}" | sort) \
<(printf '%s\n' "${array2[#]}" | sort)
and elements unique to the second array:
comm -13 \
<(printf '%s\n' "${array1[#]}" | sort) \
<(printf '%s\n' "${array2[#]}" | sort)
Or, more robust, allowing for any character including newlines to be part of the elements, split on the null byte:
comm -z -23 \
<(printf '%s\0' "${array1[#]}" | sort -z) \
<(printf '%s\0' "${array2[#]}" | sort -z)
comm is probably the way to go but if you're running bash >= 4 then you can do it with associative arrays:
#!/bin/bash
declare -a array1=(1 2 3 4 5) array2=(2 3 4 5 6)
declare -A uniq1=() uniq2=()
for e in "${array1[#]}"; do uniq1[$e]=; done
for e in "${array2[#]}"; do
if [[ ${uniq1[$e]-1} ]]
then
uniq2[$e]=
else
unset "uniq1[$e]"
fi
done
echo "new_command_output1 = ${!uniq1[*]}"
echo "new_command_output2 = ${!uniq2[*]}"
new_command_output1 = 1
new_command_output2 = 6
BASH builtins can handle this cleaner and quicker. This will read both arrays for each element, comparing if they exist in either. If no match is found, output unique elements
arr1=(1 2 3 4 5)
arr2=(1 3 2 4)
for i in "${arr1[#]}" "${arr2[#]}" ; do
[[ ${arr2[#]} =~ $i ]] || echo $i
[[ ${arr1[#]} =~ $i ]] || echo $i
done
output: 5
If one of yours arrays have multiple character elements, e.g. 152, then you must convert the arrays, adding a literal character before and after. This way regex can identify an exact match
arr1=(1 2 3 4 5)
arr2=(1 3 2 4 152)
for i in "${arr1[#]}" ; do
var1+="^$i$"
done
for i in "${arr2[#]}" ; do
var2+="^$i$"
done
for i in "${arr1[#]}" "${arr2[#]}" ; do
[[ $var1 =~ "^$i$" ]] || echo $i
[[ $var2 =~ "^$i$" ]] || echo $i
done
output: 5 152
Let's say we have a shell variable $x containing a space separated list of numbers from 1 to 30:
$ x=$(for i in {1..30}; do echo -n "$i "; done)
$ echo $x
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
We can print the first three input record fields with AWK like this:
$ echo $x | awk '{print $1 " " $2 " " $3}'
1 2 3
How can we print all the fields starting from the Nth field with AWK? E.g.
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
EDIT: I can use cut, sed etc. to do the same but in this case I'd like to know how to do this with AWK.
Converting my comment to answer so that solution is easy to find for future visitors.
You may use this awk:
awk '{for (i=3; i<=NF; ++i) printf "%s", $i (i<NF?OFS:ORS)}' file
or pass start position as argument:
awk -v n=3 '{for (i=n; i<=NF; ++i) printf "%s", $i (i<NF?OFS:ORS)}' file
Version 4: Shortest is probably using sub to cut off the first three fields and their separators:
$ echo $x | awk 'sub(/^ *([^ ]+ +){3}/,"")'
Output:
4 5 6 7 8 9 ...
This will, however, preserve all space after $4:
$ echo "1 2 3 4 5" | awk 'sub(/^ *([^ ]+ +){3}/,"")'
4 5
so if you wanted the space squeezed, you'd need to, for example:
$ echo "1 2 3 4 5" | awk 'sub(/^ *([^ ]+ +){3}/,"") && $1=$1'
4 5
with the exception that if there are only 4 fields and the 4th field happens to be a 0:
$ echo "1 2 3 0" | awk 'sub(/^ *([^ ]+ +){3}/,"")&&$1=$1'
$ [no output]
in which case you'd need to:
$ echo "1 2 3 0" | awk 'sub(/^ *([^ ]+ +){3}/,"") && ($1=$1) || 1'
0
Version 1: cut is better suited for the job:
$ cut -d\ -f 4- <<<$x
Version 2: Using awk you could:
$ echo -n $x | awk -v RS=\ -v ORS=\ 'NR>=4;END{printf "\n"}'
Version 3: If you want to preserve those varying amounts of space, using GNU awk you could use split's fourth parameter seps:
$ echo "1 2 3 4 5 6 7" |
gawk '{
n=split($0,a,FS,seps) # actual separators goes to seps
for(i=4;i<=n;i++) # loop from 4th
printf "%s%s",a[i],(i==n?RS:seps[i]) # get fields from arrays
}'
Adding one more approach to add all value into a variable and once all fields values are done with reading just print the value of variable. Change the value of n= as per from which field onwards you want to get the data.
echo "$x" |
awk -v n=3 '{val="";for(i=n; i<=NF; i++){val=(val?val OFS:"")$i};print val}'
With GNU awk, you can use the join function which has been a built-in include since gawk 4.1:
x=$(seq 30 | tr '\n' ' ')
echo "$x" | gawk '#include "join"
{split($0, arr)
print join(arr, 4, length(arr), "|")}
'
4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30
(Shown here with a '|' instead of a ' ' for clarity...)
Alternative way of including join:
echo "$x" | gawk -i join '{split($0, arr); print join(arr, 4, length(arr), "|")}'
Using gnu awk and gensub:
echo $x | awk '{ print gensub(/^([[:digit:]]+[[:space:]]){3}(.*$)/,"\\2",$0)}'
Using gensub, split the string into two sections based on regular expressions and print the second section only.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 2 years ago.
Improve this question
so for reading the list of file, I use this code here below:
IFS=$'\n' read -d '' -r -a data < ./somefolder/mytext.txt
for i in {0..9} #i know that i have 10 items, thats why i use 0..9
do
echo "${data[$i]}"
done
lets say i have 1-10 in the txt file, so it should print like below:
1
2
3
4
5
6
7
8
9
10
Questions:
is there any simpler way to read/write the text list than this?
how to save/update/overwrite data of mytext.txt? lets say change 4 to 88 for example.
Full example:
#!bin/bash
IFS=$'\n' read -d '' -r -a data < ./somefolder/mytext.txt
for i in {0..9} #i know that i have 10 items, thats why i use 0..9
do
echo "${data[$i]}"
done
echo "change 4 to anything"
read any
update(){
for n in {0..9}
do
if [[ n == 3 ]]; then
echo any
else
echo "${data[$n]}"
fi
done
}
update > ./somefolder/mytext.txt
#i dont know what i should do, it throws some errors saying syntax error
echo "saved"
exit 0
This is the code and output of the code, it is not the same as you describe in the comments.
printf '%s\n' {a..z} > file.txt
cat file.txt
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
A quick way of showing line numbers by using grep
grep -n . file.txt
A function to loop through an array.
func() {
n=1
for f; do
if (( n == 3 )); then
printf '%d %s\n' "$n" foo
else
printf '%d %s\n' "$n" "$f"
fi
((n++))
done
}
mapfile -t array < file.txt
func "${array[#]}"
Output
1 a
2 b
3 foo
4 d
5 e
6 f
7 g
8 h
9 i
10 j
11 k
12 l
13 m
14 n
15 o
16 p
17 q
18 r
19 s
20 t
21 u
22 v
23 w
24 x
25 y
26 z
On the other hand if you just want to replace everything with anything at a certain line and and ed is acceptable/available.
#!/usr/bin/env bash
printf '%s\n' ,n | ed -s file.txt
read -rp 'Change 4 to anything: ' input
printf '%s\n' "4c" "$input" . ,n w | ed -s file.txt
A more flexible version of the previous script.
#!/usr/bin/env bash
total=$(printf '%s\n' '$=' | ed -s file.txt)
printf '%s\n' ,n | ed -s file.txt
read -rp 'Enter the line number you want to change: ' int
if [[ $int == *[!0-9]* ]]; then
printf >&2 '%s is not an int\n' "$int"
exit 1
elif (( int > total )); then
printf >&2 '%s is out of range!' "$int"
exit 1
fi
read -rp "Enter the replacement at line $int: " input
printf '%s\n' "${int}c" "$input" . ,n w | ed -s file.txt
Caveat The file.txt name and path is still hard coded to the script, just add an additional read for the the file.
I have below shell script in which I have two arrays number1 and number2. I have a variable range which has list of numbers.
Now I need to figure out what are all numbers which are in number1 array are also present in range variable. Similarly for number2 array as well. Below is my shell script and it is working fine.
number1=(1220 1374 415 1097 1219 557 401 1230 1363 1116 1109 1244 571 1347 1404)
number2=(411 1101 273 1217 547 1370 286 1224 1362 1091 567 561 1348 1247 1106 304 435 317)
range=90,197,521,540,552,554,562,569:570,573,576,579,583,594,597,601,608:609,611,628,637:638,640:641,644:648
range_f=" "$(eval echo $(echo $range | perl -pe 's/(\d+):(\d+)/{$1..$2}/g;s/,/ /g;'))" "
echo "$range_f"
for item in "${number1[#]}"; do
if [[ $range_f =~ " $item " ]] ; then
new_number1+=($item)
fi
done
echo "new list: ${new_number1[#]}"
for item in "${number2[#]}"; do
if [[ $range_f =~ " $item " ]] ; then
new_number2+=($item)
fi
done
echo "new list: ${new_number2[#]}"
Is there any better way to write above stuff? As of now I have two for loops iterating and then figuring out new_number1 and new_number2 arrays.
Note:
Numbers like 644:648 means, it starts with 644 and ends with 648. It is just short form.
You can use comm with process substitution instead of looping:
mapfile -t new_number1 < <(comm -12 <(printf '%s\n' "${number1[#]}" | sort) <(printf '%s\n' $range_f | sort))
mapfile -t new_number2 < <(comm -12 <(printf '%s\n' "${number2[#]}" | sort) <(printf '%s\n' $range_f | sort))
mapfile -t name reads from the nested process substitution into the named array
printf ... | sort pair provides the sorted input streams for comm
comm -12 emits the items common to the two streams
Aside from codeforester's answer, I can think of two other ways of doing this:
Load the values of $range as keys of an associative array. The
values will be 1. Loop through each member of ${number1[#]} and
${number2[#]}, testing them against the values in the associative
array.
Use codeforester's printf ... | sort trick, but pipe both the list
and the range through sort | uniq -c, then grep for the
duplicates.
I'm not sure if either one of these is an actual improvement on your code. ... I would create a 'find duplicates' shell function, but otherwise your code looks solid.
I've written a shell script to get the PIDs of specific process names (e.g. pgrep python, pgrep java) and then use top to get the current CPU and Memory usage of those PIDs.
I am using top with the '-p' option to give it a list of comma-separated PID values. When using it in this mode, you can only query 20 PIDs at once, so I've had to come up with a way of handling scenarios where I have more than 20 PIDs to query. I'm splitting up the list of PIDs passed to the function below and "despatching" multiple top commands to query the resources:
# $1 = List of PIDs to query
jobID=0
for pid in $1; do
if [ -z $pidsToQuery ]; then
pidsToQuery="$pid"
else
pidsToQuery="$pidsToQuery,$pid"
fi
pidsProcessed=$(($pidsProcessed+1))
if [ $(($pidsProcessed%20)) -eq 0 ]; then
debugLog "DESPATCHED QUERY ($jobID): top -bn 1 -p $pidsToQuery | grep \"^ \" | awk '{print \$9,\$10}' | grep -o '.*[0-9].*' | sed ':a;N;\$!ba;s/\n/ /g'"
resourceUsage[$jobID]=`top -bn 1 -p "$pidsToQuery" | grep "^ " | awk '{print $9,$10}' | grep -o '.*[0-9].*' | sed ':a;N;$!ba;s/\n/ /g'`
jobID=$(($jobID+1))
pidsToQuery=""
fi
done
resourceUsage[$jobID]=`top -bn 1 -p "$pidsToQuery" | grep "^ " | awk '{print $9,$10}' | grep -o '.*[0-9].*' | sed ':a;N;$!ba;s/\n/ /g'`
The top command will return the CPU and Memory usage for each PID in the format (CPU, MEM, CPU, MEM etc)...:
13 31.5 23 22.4 55 10.1
The problem is with the resourceUsage array. Say, I have 25 PIDs I want to process, the code above will place the results of the first 20 PIDs in to $resourceUsage[0] and the last 5 in to $resourceUsage[1]. I have tested this out and I can see that each array element has the list of values returned from top.
The next bit is where I'm having difficulty. Any time I've ever wanted to print out or use an entire array's set of values, I use ${resourceUsage[#]}. Whenever I use that command in the context of this script, I only get element 0's data. I've separated out this functionality in to a script below, to try and debug. I'm seeing the same issue here too (data output to debug.log in same dir as script):
#!/bin/bash
pidList="1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25"
function quickTest() {
for ((i=0; i<=1; i++)); do
resourceUsage[$i]=`echo "$i"`
done
echo "${resourceUsage[0]}"
echo "${resourceUsage[1]}"
echo "${resourceUsage[#]}"
}
function debugLog() {
debugLogging=1
if [ $debugLogging -eq 1 ]; then
currentTime=$(getCurrentTime 1)
echo "$currentTime - $1" >> debug.log
fi
}
function getCurrentTime() {
if [ $1 -eq 0 ]; then
echo `date +%s`
elif [ $1 -eq 1 ]; then
echo `date`
fi
}
jobID=0
for pid in $pidList; do
if [ -z $pidsToQuery ]; then
pidsToQuery="$pid"
else
pidsToQuery="$pidsToQuery,$pid"
fi
pidsProcessed=$(($pidsProcessed+1))
if [ $(($pidsProcessed%20)) -eq 0 ]; then
debugLog "DESPATCHED QUERY ($jobID): top -bn 1 -p $pidsToQuery | grep \"^ \" | awk '{print \$9,\$10}' | grep -o '.*[0-9].*' | sed ':a;N;\$!ba;s/\n/ /g'"
resourceUsage[$jobID]=`echo "10 10.5 11 11.5 12 12.5 13 13.5"`
debugLog "Resource Usage [$jobID]: ${resourceUsage[$jobID]}"
jobID=$(($jobID+1))
pidsToQuery=""
fi
done
#echo "Dispatched job: $pidsToQuery"
debugLog "DESPATCHED QUERY ($jobID): top -bn 1 -p $pidsToQuery | grep \"^ \" | awk '{print \$9,\$10}' | grep -o '.*[0-9].*' | sed ':a;N;\$!ba;s/\n/ /g'"
resourceUsage[$jobID]=`echo "14 14.5 15 15.5"`
debugLog "Resource Usage [$jobID]: ${resourceUsage[$jobID]}"
memUsageInt=0
memUsageDec=0
cpuUsage=0
i=1
debugLog "Row 0: ${resourceUsage[0]}"
debugLog "Row 1: ${resourceUsage[1]}"
debugLog "All resource usage results: ${resourceUsage[#]}"
for val in ${resourceUsage[#]}; do
resourceType=$(($i%2))
if [ $resourceType -eq 0 ]; then
debugLog "MEM RAW: $val"
memUsageInt=$(($memUsageInt+$(echo $val | cut -d '.' -f 1)))
memUsageDec=$(($memUsageDec+$(echo $val | cut -d '.' -f 2)))
debugLog " MEM INT: $memUsageInt"
debugLog " MEM DEC: $memUsageDec"
elif [ $resourceType -ne 0 ]; then
debugLog "CPU RAW: $val"
cpuUsage=$(($cpuUsage+$val))
debugLog "CPU TOT: $cpuUsage"
fi
i=$(($i+1))
done
debugLog "$MEM DEC FINAL: $memUsageDec (pre)"
memUsageDec=$(($memUsageDec/10))
debugLog "$MEM DEC FINAL: $memUsageDec (post)"
memUsage=$(($memUsageDec+$memUsageInt))
debugLog "MEM USAGE: $memUsage"
debugLog "CPU USAGE: $cpuUsage"
debugLog "MEM USAGE: $memUsage"
debugLog "PROCESSED VALS: $cpuUsage,$memUsage"
echo "$cpuUsage,$memUsage"
I'm really stuck here as I've printed out entire arrays before in Bash Shell with no problem. I've even repeated this in the shell console with a few lines and it works fine there:
listOfValues[0]="1 2 3 4"
listOfValues[1]="5 6 7 8"
echo "${listOfValues[#]}"
Am I missing something totally obvious? Any help would be greatly appreciated!
Thanks in advance! :)
Welcome to StackOverflow, and thanks for providing a test case! The bash tag wiki has additional suggestions for creating small, simplified test cases. Here's a minimal version that shows your problem:
log() {
echo "$1"
}
array=(foo bar)
log "Values: ${array[#]}"
Expected: Values: foo bar. Actual: Values: foo.
This happens because ${array[#]} is magic in quotes, and turns into multiple arguments. The same is true for $#, and for brevity, let's consider that:
Let's say $1 is foo and $2 is bar.
The single parameter "$#" (in quotes) is equivalent to the two arguments "foo" "bar".
"Values: $#" is equivalent to the two parameters "Values: foo" "bar"
Since your log statement ignores all arguments after the first one, none of them show up. echo does not ignore them, and instead prints all arguments space separated, which is why it appeared to work interactively.
This is as opposed to ${array[*]} and $*, which are exactly like $# except not magic in quotes, and does not turn into multiple arguments.
"$*" is equivalent to "foo bar"
"Values: $*" is equivalent to "Values: foo bar"
In other words: If you want to join the elements in an array into a single string, Use *. If you want to add all the elements in an array as separate strings, use #.
Here is a fixed version of the test case:
log() {
echo "$1"
}
array=(foo bar)
log "Values: ${array[*]}"
Which outputs Values: foo bar
I would use ps, not top, to get the desired information. Regardless, you probably want to put the data for each process in a separate element of the array, not one batch of 20 per element. You can do this using a while loop and a process substitution. I use a few array techniques to simplify the process ID handling.
pid_array=(1 2 3 4 5 6 7 8 9 ... )
while (( ${#pid_array[#]} > 0 )); do
printf -v pidsToQuery "%s," "${pid_array[#]:0:20}"
pid_array=( "${pid_array[#]:20}" )
while read cpu mem; do
resourceUsage+=( "$cpu $mem" )
done < <( top -bn -1 -p "${pidsToQuery%,}" ... )
done