Bash parse dynamic arrays and store specific values in an other - arrays

I am sending a dbus-send command which returns something like :
method return sender=:1.833 -> dest=:1.840 reply_serial=2
array of bytes [
00 01 02 03 04 05
]
int 1
boolean true
The "array of bytes" size is dynamic an can contains n values.
I store the result of the dbus-send command in an array by using :
array=($(dbus-send --session --print-repl ..readValue))
I want to be able to retrieve the values contained into the array of bytes and be able to display one or all of them if necessary like this :
data read => 00 01 02 03 04 05
or
first data read => 00
First data is always reachable by {array[10]} and I think is it possible to use a structure like :
IFS=" " read -a array
for element in "${array[#]:10}"
do
...
done
Any thoughts on how to accomplish this?

You really should use some library for dbus, like Net::DBus or something similar.
Anyway, for the above example you could write:
#fake dbus-send command
dbus-send() {
cat <<EOF
method return sender=:1.833 -> dest=:1.840 reply_serial=2
array of bytes [
00 01 02 03 04 05
]
int 1
boolean true
EOF
}
array=($(dbus-send --session --print-repl ..readValue))
data=($(echo "${array[#]}" | grep -oP 'array\s*of\s*bytes\s*\[\s*\K[^]]*(?=\])'))
echo "ALL data ==${data[#]}=="
echo "First item: ${data[0]}"
echo "All items as lines"
printf "%s\n" "${data[#]}"
data=($(echo "${array[#]}" | sed 's/.*array of bytes \[\([^]]*\)\].*/\1/'))
echo "ALL data ==${data[#]}=="
echo "First item: ${data[0]}"
echo "All items as lines"
printf "%s\n" "${data[#]}"
for the both example prints
ALL data ==00 01 02 03 04 05==
First item: 00
All items as lines
00
01
02
03
04
05

Related

How to create an array with leading zeros in Bash?

Not very difficult:
#!/bin/bash
hr=(00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23)
for i in ${hr[#]}; do
echo ${hr[i]}
done
But:
user#userver:$ ./stat.sh
00
01
02
03
04
05
06
07
./stat.sh: string 7: 08: value too great for base (error token is "08")
Bash thinks that leading zero means octal number system. What to do?
I think your fundamental problem is that you're using each element of the array to index the array itself.
If you want to just print out the array elements, you should be outputting ${i} rather than ${hr[i]}. The latter is useless in your current code since element zero is 00, element one is 01, and so on.
On the chance that this is a simplified example and you do want to reference a different array based on the content of this one, you have a couple of options:
Realise that the value of an integer and the presentation of it are two distinct things. In other words, use 0, 1, 2, ... but something like printf "%02d" $i if you need to output it (noting this is only ouputting the value in the first array, not the one you're looking up things in).
Exclusively use strings and a string-based associative array rather than an integer-based one, see typeset -A for detail.
Use brace expansion (ranges, repetition).
To create an array:
hours=({00..23})
Loop through it:
for i in "${hours[#]}"; do
echo "$i"
done
Or loop through a brace expansion:
for i in {00..23}; do
echo "$i"
done
Regarding your error, it's because in bash arithmetic, all numbers with leading zeroes are treated as octals, and 08 and 09 are invalid octal numbers. All indexed array subscripts are evaluated as arithmetic expressions. You can fix the problem by using the notation base#number to specify a number system. So for base 10: 10#09, or for i=09, 10#$i. The variable must be prefixed with $, 10#i does not work.
You should be printing your array like this anyway:
Loop through elements:
for i in "${hr[#]}"; do
echo "$i"
done
Loop through indexes:
for i in "${!hr[#]}"; do
echo "index is $i"
echo "element is ${hr[i]}"
done
If you need to do arithmetic on the hours, or any zero padded number, you will lose the zero padding. You can print it again with printf: printf %.2d "$num", where 2 is the minimum width.
When you iterate with:
for i in ${hr[#]}; do
It is iterating the values of the array witch are:
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23
But when within the loop it has:
echo ${hr[i]}
it is using i as the index of the hr array.
In Bash, an index within the brackets of an array like [i] is an arithmetic context. It means while the value of i=08 the leading 0 within the arithmetic context causes the number to be treated as an octal number, and 8 is an invalid octal number.
If you wanted to iterate your array indexes to process its values by index, then you'd start the loop as:
for i in "${!hr[#]}"; do
This one will perfectly work as it iterates the index into the variable i :
#!/usr/bin/env bash
declare -a hr=(00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23)
for i in "${!hr[#]}"; do
printf '%s\n' "${hr[i]}"
done
Now if all you want is iterate the values of the hr array, just do this way:
#!/usr/bin/env bash
declare -a hr=(00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23)
for e in "${hr[#]}"; do
printf '%s\n' "$e"
done
No need to index the array within the loop, since the elements are already expanded into e.
You can specify it's 10-based :
echo ${hr[10#$i]}

How to increment through the values in a list?

I have repeatedly tried to search for this question, but cannot form a search that produces results that are actually relevant to my question.
I am trying to build a script that parses an SVG file (XML text format file that produces graphic content) looking for specific cues and assigns RGB values. (Each RGB value will be slightly different.)
To do this, I picture multiple incrementing variables (for instance $i, $j & $k) that increment based on triggers found while parsing the text file.
However, the amounts that I need to increment are not "1". Also, the values needed are hexadecimal in form.
I could set up something where the vars are incremented by a given amount, such as +33, but I would also need to convert numbers to hex, figure out how to start over, etc.
A far more versatile, powerful and elegant approach occurs to me, but I don't know how to go about it.
How can I set up these incrementing variables to increment through the values I've set up in an array?
For example, say my RGB potential values are #rgbval = (00,33,66,99,cc,ff). How can I make $i go from one value in this list to the next?
Even better, how could I make $i++ (or something similar) mean "go to the next element's value in #rgbval"?
And assuming this is possible, how would I tell Perl to start over at element [0] after reaching the end of the array?
So you have a string that's the hex representation of a number
my $hex = 'cc';
To do arithmetic on it, you first need to convert that to a number.
my $num = hex($hex);
Now we can do arithmetic on it.
$num += 33;
If we want to convert it back to hex, we can use
$hex = sprintf("%02x", $num);
$i is usually used for indexes. If that's what you want, you can use the following:
for my $i (0..$#rgbval) {
# Do something with $i and/or $rgbval[$i]...
}
If instead you want $i to take on each value, you can use the following:
for my $i (#rgbval) {
# Do something with $i...
}
But it seems to be you want a counter that wraps around.
The straightforward solution would be use an if statement.
my $i = 0;
while (...) {
# Do something with $i and/or $rgbval[$i]...
++$i;
$i = 0 if $i == #rgbval;
}
But I'd use modulo arithmetic.
my $i = 0;
while (...) {
# Do something with $i and/or $rgbval[$i]...
$i = ( $i + 1 ) % #rgbval;
}
Alternatively, you could rotate the array.
while (...) {
# Do something with $rgbval[0]...
push #rgbval, shift(#rgbval);
}
Ikegami, what an excellent bunch of information your response held.
I found three of your proposals too appealing to ignore, and tried to understand them. Your first section was about processing the math described, converting into and out of hex.
I tried to wrestle these steps into a "test of concept" script, along with your suggestion of a modulo reset. (Okay, a "test of understanding of concept".)
For the test script I used an iterator rather than searching for a triggering event, that seemed simpler.
The goal was to have the values increment through the hex numbers listed in the example array, and to start over after the last value.
So I iterated up to ten times, to give the values a chance to start over. After I figured out that I needed to add 51 each time instead of 33 to get those example values, and also had to make the numerical value of my array 51 times larger since I was incrementing by 51, it worked pretty well:
my $num = hex("00");
my #rgbval = qw(a b c d e f);
for my $i (0..10) {
print ( "For i=$i, \$num is " , sprintf("%02x ", $num) , "\n");
$num = ( $num + 51 ) % ( 51 * #rgbval );
}
output:
~\Perlscripts>iterate2.pl
For i=0, $num is 00 For i=1, $num is 33 For i=2, $num is 66 For i=3,
$num is 99 For i=4, $num is cc For i=5, $num is ff For i=6, $num is 00
For i=7, $num is 33 For i=8, $num is 66 For i=9, $num is 99 For i=10,
$num is cc
As far as the non-mathy approach, incrementing through the strings of the array, I understood you to be saying I would need to increment the indices that reference the array values. I did manage to confuse my self with the different iterators and what they were doing, but after a few stumbles, I was able to make this approach work as well:
my #hue = qw(00 33 66 99 cc ff);
my $v = 0;
for my $i (0..10) {
print "\$i=$i, \$hue[$v]=" , $hue[$v] , "\n";
$v = ( $v + 1 ) % #hue;
}
output:
~\Perlscripts>iterate2.pl
$i=0, $hue[0]=00 $i=1, $hue[1]=33 $i=2, $hue[2]=66 $i=3,
$hue[3]=99 $i=4, $hue[4]=cc $i=5, $hue[5]=ff $i=6,
$hue[0]=00 $i=7, $hue[1]=33 $i=8, $hue[2]=66 $i=9,
$hue[3]=99 $i=10, $hue[4]=cc
Your last proposed solution, rotating the array with push and shift seemed perhaps the most novel approach and quite compelling, especially once I realized that if i have a variable that stores the shifted value, that will be the correct value to push next time, around and around.
In this approach I don't even have to worry about starting over after the last value; the changing array takes care of that automatically for me:
my #hue = qw(00 33 66 99 cc ff);
for my $i (0..10) {
my $curval = shift(#hue);
print "\$i=$i, \$curval is $curval \.\.\.And the array is currently: ( #hue )\n";
push(#hue,$curval);
}
output:
~\Perlscripts>iterate2.pl
$i=0, $curval is 00 ...And the array is currently: (
33 66 99 cc ff ) $i=1, $curval is 33 ...And the
array is currently: ( 66 99 cc ff 00 ) $i=2, $curval is 66 ...And the array is currently: ( 99 cc ff 00 33 ) $i=3, $curval is
99 ...And the array is currently: ( cc ff 00 33 66 )
$i=4, $curval is cc ...And the array is currently: (
ff 00 33 66 99 ) $i=5, $curval is ff ...And the
array is currently: ( 00 33 66 99 cc ) $i=6, $curval is 00 ...And the array is currently: ( 33 66 99 cc ff ) $i=7, $curval is
33 ...And the array is currently: ( 66 99 cc ff 00 )
$i=8, $curval is 66 ...And the array is currently: (
99 cc ff 00 33 ) $i=9, $curval is 99 ...And the
array is currently: ( cc ff 00 33 66 ) $i=10, $curval is cc ...And the array is currently: ( ff 00 33 66 99 )
Most educational and helpful! Thanks so much.

LIBMODBUS: Writing to a double register?

Is there a way I can write one value to a double register using LIBMODBUS? For example writing value 100,000 to be spread across one register. Currently using modbus_write_registers to write 10,000 I am sending the modbus message
rc = modbus_write_registers(ctx, 4, 2, tab_reg); (Where tab_reg[0] = 10,000 and tab_reg[1] = 0)
0A 10 00 04 00 02 04 27 10 00 00 DC 09
Ideally the message i believe I would like to send would not send the 00 00 for the zero value. Is this possible to utilise using Libmodbus?
NB - I have also attempted using modbus_write_register() and this produced a much longer message so I am inclined to believe write registerS is the way to go.

combine string in python pandas

I got a problem when analyzing dataset about combining string together.
The data frame looks like the one below:
IP Event
01 check
01 redo
01 view
02 check
02 check
03 review
04 delete
As you can see, the IP contains duplicates. My question is, how can I get the results of combining the Event group by each IP in order.For example, the result I'm looking for is:
IP result
01 check->redo->view
02 check->check
03 review
04 delete
try this:
In [27]: df.groupby('IP').agg('->'.join).reset_index()
Out[27]:
IP Event
0 01 check->redo->view
1 02 check->check
2 03 review
3 04 delete
or
In [26]: df.groupby('IP').agg('->'.join)
Out[26]:
Event
IP
01 check->redo->view
02 check->check
03 review
04 delete
Try this with lambda:
df.groupby("IP")['Event'].apply(lambda x: '->'.join(x)).reset_index()
# IP Event
# 0 1 check->redo->view
# 1 2 check->check
# 2 3 review
# 3 4 delete

shell programming: define array including zero-padded values

I just started using shell programming. I want to automatically change directories and then rename some files in there. Here's my problem: The name of the directories are numbered but directories < 10 are zero-padded (01 02...09). How can I define an array using some sort of sequencing without typing each directory name manually?
This is what I've tried so far:
array = (printf "%.2d " {1..8} {11..27} {29..32} {34..50}) ## should say 01 02 03 ..08 11..27 29..32 34..50
for i in "${array[#]}"
do
echo "dir_a/dir_b/sub$i/dir_c/"
done
However, it doesn't work and the result looks like: "subprintf", "sub%.2s", "sub1" etc.
Can you help me there?
In a next step I want to filter certain numbers in the array, e.g. 03, 09, 10, 28, 33 as these directories don't exist. Is there some easy solution to create such an array without concatenating 5 separate arrays?
Many thanks in advance,
Kati
Is there a need to use arrays? Otherwise, for bash 4, you can do
for i in {01..08} {11..27} {29..32} {34..50}; do
echo "dir_a/dir_b/sub${i}/dir_c/"
done
For an older version of bash you have to add the 0 yourself:
for i in 0{1..8} {11..27} {29..32} {34..50}; do
echo "dir_a/dir_b/sub${i}/dir_c/"
done
Of course, if you want to have an array, you can do
array=({01..08} {11..27} {29..32} {34..50})
or
array=(0{1..8} {11..27} {29..32} {34..50})
You could do this:
declare -a dirs=('01' '02' '03' '04' '05' '06' '07' '08')
echo ${dirs[#]}
01 02 03 04 05 06 07 08
# Make up next sequence
declare -a b=`seq 11 18`
echo ${b[#]}
11 12 13 14 15 16 17 18
# Add sequences together
dirs=("${dirs[#]}" ${b})
echo ${dirs[#]}
01 02 03 04 05 06 07 08 11 12 13 14 15 16 17 18
find [0-9][0-9] -type d | while read dirname
do
if [ $(echo "${dirname}" | sed -n '/01/p') ]
then
cd "${dirname}"
mv foo bar
cd ..
fi
done
Then you can just write another elif and sed check for every directory which contains files you want to rename. I know it's not what you asked for, but it is infinitely simpler. If you're allowed to, I'd also strongly recommend renaming that directory tree, as well.
#/bin/bash
raw=({01..08} {11..27} {29..32} {34..50})
filter=(03 09 10 28 33)
is_in() {
for e in "${#:2}"; do [[ "$e" == "$1" ]] && return 0; done
return 1
}
for i in ${raw[#]}; do
is_in $i ${filter[#]} || echo "dir_a/dir_b/sub$i/dir_c"
done
It'll take the numbers in the raw array and exclude every occurance of the ones in the filter array.

Resources