Working on a simple bash script that I can use to ultimately tell me if a rogue process is running that we don't want - this one will ultimately be running with a different parent pid. a monitor of sorts. Where I'm having an issue is getting all the specific pids that I want into an array that I can perform some actions on. Script first:
#!/bin/bash
rmanRUNNING=`ps -ef|grep /etc/process/process.conf|egrep -v grep|wc -l`
if [ $rmanRUNNING -gt 0 ]
then
rmanPPID=( $(ps -ef|grep processname|egrep -v grep|egrep -v /etc/process/process.conf|awk '{ printf $3 }') )
for i in "${rmanPPID[#]}"
do
:
echo $i
done
fi
So, goal is to check for existence of the main process, this is the one running with the config file in it, the first variable tells me this. Next, if it's running (based on the count greather than 0) the intention is to populate an array with all the parent pids, excluding what would be determined as the main process (we don't need to analyze this one). So, in the array definition we get the list of processes, grep process name, egrep -v the grep output, also egrep -v the "main" process and then awk the parent pids then iterate through and attempt to echo each one individually (more would be done in this section, but it's not working). Unfortunately, when I output $i all of the parent pids are simply concatenated together in one long string. If I try to output a specific array item I get an empty output.
Obviously the question here is, what's wrong with my array definition that is preventing it from being declared as an array, or some other odd thing.
This is on RHEL, 6.2 on the test environment, probably 7 in production by the time this is live.
Full disclosure, I'm a monitoring engineer, not an SA - definitely not a bash scripter by nature!
Thanks in advance.
EDIT: just for clarity, an echo to screen of the PIDs is NOT the end desired output, it's just a simple way to test that I'm getting back what I'm expecting. Based on comment below I believe pgrep type output is the preferred output. In the end I'll be tying these pids back one at a time against the original process to ensure that it is the parent, and if it is not I'll spit out an error.
It's not so much $i that will be one concatenated number, as well as that your array is just a single element of that concatenated number. This is because the output of awk is concatenated together, without any separator.
If you simply add a space within awk, you may get what you want:
rmanPPID=( $(ps -ef|grep processname | ... | awk '{ printf "%d ", $3 }') )
or even simpler, use print instead of printf:
rmanPPID=( $(ps -ef|grep processname | ... | awk '{ print $3 }') )
(Thanks to Jonathan Leffler, see comment below.)
Related
How to perform multiple functions on each object in an array and output the results to a file?
Heres my array, its the value of a command, that command lists all text files in the current directory:
#!/bin/sh
declare -a FILES
FILES=( $(find . -mindepth 1 -maxdepth 1 -type f -iname "*.txt") )
As you can see below, it works:
# Print the full array:
echo "${FILES[#]}"
./1.txt ./2.txt ./3.txt
# Print the number of objects in the array:
echo "${#FILES[#]}"
3
# Loop through each object in the array and print its name:
for file in "${FILES[#]}"
do
echo "$file"
done
./1.txt
./2.txt
./3.txt
For each object in the array I need to perform three functions:
# Use grep to return a whole line:
GREP="$(cat "$file" | grep 'String_A')"
# Use awk to return part of a line only (the second column, after the first colon)
AWK="$(cat "$file" | awk -F: '/String_B/ {print $2; exit}')"
# Use sed to return all lines from C to E (C + D + E)
SED="$(cat "$file" | sed -nE '/String_C/,/String_E/p')"
For this, a "for loop":
for file in "${FILES[#]}"
do
echo "$GREP : $AWK : $SED"
done
File_1_Grep_String_A : File_1_Awk_String_B : File_1_Sed_String_C
File_1_Sed_String_D
File_1_Sed_String_E
File_3_Grep_String_A : File_3_Awk_String_B : File_3_Sed_String_C
File_3_Sed_String_D
File_3_Sed_String_E
File_2_Grep_String_A : File_2_Awk_String_B : File_2_Sed_String_C
File_2_Sed_String_D
File_2_Sed_String_E
This is all the that I require, but I need this info written to a file called results,
the problem is, when I redirect this output to a file, only the first object in the array is parsed:
for file in "${FILES[#]}"
do
echo "$GREP : $AWK : $SED" > Some_File
done
cat Some_File
File_2_Grep_String_A : File_2_Awk_String_B : File_2_Sed_String_C
File_2_Sed_String_D
File_2_Sed_String_E
Summary:
The command works correctly when it outputs to stdout but not when redirected to a file.
I have found similar questions here on stackoverflow (performing functions on arrays) but none redirect output to a file.
I find this quite weird, I have never had a command output to stdout fine but unable to get that in text elsewhere.
Whilst I'm new here, my last couple questions wasnt recieved well by some members as I was too brief,
so I think this time I have fully explained and shown the output step-by-step.
Also, to save you the burden of making an identical test environment and copy/paste'ing all of this,
I put it up on github for you to clone, hopefully this helps.
Again, to ease things, in there you will find two scripts,
the main one in question, called script.sh (this is the one that needs fixing, its clean/uncommented)
but I also included one called tests.sh, this is basically all the steps I have taken,
it contains alot of comments that will further help you understand better than I can explain here.
To be honest, I would definately have a quick look at that first,
its also safe to execute (it contains all the working bits I have shown above, like printing the array),
this saves you typing out commands to test the array etc.
Thank you in advance! I hope I've done better this time?!?
git clone https://github.com/5c0tt-b0t/stackoverflow_test
Test script output:
########## TESTS: ##########
FULL ARRAY:
./1.txt ./3.txt ./2.txt
COUNT:
3
OBJECTS:
./1.txt
OBJECTS:
./3.txt
OBJECTS:
./2.txt
########## INFO THAT I NEED: ##########
File_1_Grep_String_A : File_1_Awk_String_B : File_1_Sed_String_C
File_1_Sed_String_D
File_1_Sed_String_E
File_3_Grep_String_A : File_3_Awk_String_B : File_3_Sed_String_C
File_3_Sed_String_D
File_3_Sed_String_E
File_2_Grep_String_A : File_2_Awk_String_B : File_2_Sed_String_C
File_2_Sed_String_D
File_2_Sed_String_E
You can redirect the output of the whole loop:
for file in "${FILES[#]}"
do
echo "$GREP : $AWK : $SED"
done > Some_File
Also, use #!/bin/bash when using bashisms like arrays, /bin/sh doesn't necessarily support arrays.
While I've handled this task in other languages easily, I'm at a loss for which commands to use when Shell Scripting (CentOS/BASH)
I have some regex that provides many matches in a file I've read to a variable, and would like to take the regex matches to an array to loop over and process each entry.
Regex I typically use https://regexr.com/ to form my capture groups, and throw that to JS/Python/Go to get an array and loop - but in Shell Scripting, not sure what I can use.
So far I've played with "sed" to find all matches and replace, but don't know if it's capable of returning an array to loop from matches.
Take regex, run on file, get array back. I would love some help with Shell Scripting for this task.
EDIT:
Based on comments, put this together (not working via shellcheck.net):
#!/bin/sh
examplefile="
asset('1a/1b/1c.ext')
asset('2a/2b/2c.ext')
asset('3a/3b/3c.ext')
"
examplearr=($(sed 'asset\((.*)\)' $examplefile))
for el in ${!examplearr[*]}
do
echo "${examplearr[$el]}"
done
This works in bash on a mac:
#!/bin/sh
examplefile="
asset('1a/1b/1c.ext')
asset('2a/2b/2c.ext')
asset('3a/3b/3c.ext')
"
examplearr=(`echo "$examplefile" | sed -e '/.*/s/asset(\(.*\))/\1/'`)
for el in ${examplearr[*]}; do
echo "$el"
done
output:
'1a/1b/1c.ext'
'2a/2b/2c.ext'
'3a/3b/3c.ext'
Note the wrapping of $examplefile in quotes, and the use of sed to replace the entire line with the match. If there will be other content in the file, either on the same lines as the "asset" string or in other lines with no assets at all you can refine it like this:
#!/bin/sh
examplefile="
fooasset('1a/1b/1c.ext')
asset('2a/2b/2c.ext')bar
foobar
fooasset('3a/3b/3c.ext')bar
"
examplearr=(`echo "$examplefile" | grep asset | sed -e '/.*/s/^.*asset(\(.*\)).*$/\1/'`)
for el in ${examplearr[*]}; do
echo "$el"
done
and achieve the same result.
There are several ways to do this. I'd do with GNU grep with perl-compatible regex (ah, delightful line noise):
mapfile -t examplearr < <(grep -oP '(?<=[(]).*?(?=[)])' <<<"$examplefile")
for i in "${!examplearr[#]}"; do printf "%d\t%s\n" $i "${examplearr[i]}"; done
0 '1a/1b/1c.ext'
1 '2a/2b/2c.ext'
2 '3a/3b/3c.ext'
This uses the bash mapfile command to read lines from stdin and assign them to an array.
The bits you're missing from the sed command:
$examplefile is text, not a filename, so you have to send to to sed's stdin
sed's a funny little language with 1-character commands: you've given it the "a" command, which is inappropriate in this case.
you only want to output the captured parts of the matches, not every line, so you need the -n option, and you need to print somewhere: the p flag in s///p means "print the [line] if a substitution was made".
sed -n 's/asset\(([^)]*)\)/\1/p' <<<"$examplefile"
# or
echo "$examplefile" | sed -n 's/asset\(([^)]*)\)/\1/p'
Note that this returns values like ('1a/1b/1c.ext') -- with the parentheses. If you don't want them, add the -r or -E option to sed: among other things, that flips the meaning of ( and \(
New to StackOverflow and new to bash scripting. I have a shell script that is attempting to do the following:
cd into a directory on a remote machine. Assume I have already established a successful SSH connection.
Save the email addresses from the command line input (these could range from 1 to X number of email addresses entered) into an array called 'emails'
Save the brand IDs (integers) from the command line input (these could range from 1 to X number of brand IDs entered) into an array called 'brands'
Use nested for loops to iterate over the 'emails' and 'brands' arrays and add each email address to each brand via add.py
I am running into trouble splitting up and saving data into each array, because I do not know where the command line indices of the emails will stop, and where the indices of the brands will begin. Is there any way I can accomplish this?
command line input I expect to look as follows:
me#some-remote-machine:~$ bash script.sh person1#gmail.com person2#gmail.com person3#gmail.com ... personX#gmail.com brand1 brand2 brand3 ... brandX
The contents of script.sh look like this:
#!/bin/bash
cd some/directory
emails= ???
brands= ???
for i in $emails
do
for a in $brands
do
python test.py add --email=$i --brand_id=$a --grant=manage
done
done
Thank you in advance, and please let me know if I can clarify or provide more information.
Use a sentinel argument that cannot possibly be a valid e-mail address. For example:
$ bash script.sh person1#gmail.com person2#gmail.com '***' brand1 brand2 brand3
Then in a loop, you can read arguments until you reach the non-email; everything after that is a brand.
#!/bin/bash
cd some/directory
while [[ $1 != '***' ]]; do
emails+=("$1")
shift
done
shift # Ignore the sentinal
brands=( "$#" ) # What's left
for i in "${emails[#]}"
do
for a in "${brands[#]}"
do
python test.py add --email="$i" --brand_id="$a" --grant=manage
done
done
If you can't modify the arguments that will be passed to script.sh, then perhaps you can distinguish between an address and a brand by the presence or absence of a #:
while [[ $1 = *#* ]]; do
emails+=("$1")
shift
done
brands=("$#")
I'm assuming that the number of addresses and brands are independent. Otherwise, you can simply look at the total number of arguments $#. Say there are N of each. Then
emails=( "${#:1:$#/2}" ) # First half
brands=( "${#:$#/2+1}" ) # Second half
I do not know why this code stopped working
I tested it a couple of times and it was running great
what I am trying to do hear is place first and second in 2 different arrays
and then comparing argument $2 ==> $comment to the array varA and if it is in the array i do not want to store it in the text file $file
comment=$2
dueD=$3
x=0
hasData()
{
declare -a varA varB
cat $file | while IFS=$'\t' read -r num first second;do
varA+=("$first")
varB+=("$second")
done
if [[ ${varA[#]} == ~$comment ]]; then
echo "already in the Todo list"
else
x=$(cat $file | wc -l)
x=$(($x+1))
echo -e "$x\t$comment\t$dueD" >> $file
fi
I think I am storing the values wrong in the array because when I try
echo ${varA[#]}
nothing gets printed
more over I think my if statement is not accurate enough since this is the 4th time I edit it and it works but after a while it no longer works
need assistance kindly
Your pipeline creates a sub-shell. Therefore your assignments to varA and varB happen in the sub-shell and are lost as soon as the sub-shell exits. See How can I read a file (data stream, variable) line-by-line (and/or field-by-field)? for how to do this without a sub-shell. – Etan Reisner
Look at the solutions there. See how they don't use a pipe? That's the solution: Don't use a pipe. Use one of the other input redirection options. – Etan Reisner
I'm very new in shell scripting, and I encountered a problem that is quite wired. The program is rather simple so I just post it here:
#!/bin/bash
list=""
list=`mtx -f /dev/sg2 status | while read status
do
result=$(echo ${status} | grep "Full")
if [ -z "$result" ]; then
continue
else
echo $(echo ${result} | cut -f3 -d' ' | tr -d [:alpha:] | tr -d [:punct:])
fi
done`
echo ${list}
for haha in ${list}
do
printf "current slot is:%s \n" ${haha}
done
What the program does is that it executes mtx -f /dev/sg2 status and goes to each line and see if there's a full disk. If it has "Full" in that line, I'll record the slot number in that line, and put in the list.
Notice that I put a back quote after list= at line 6, and it covers the whole "while" loop after that. The reason is unclear to me, but I got this usage by just googling it. It is said that the while loop will open up a separate shell or something like that, so when the while loop is done, whatever you concatenated in the loop will get lost, so in my initial implementation, list is still empty after the while loop.
My question is: even if the code above works fine, it looks pretty tricky to others, and what's worse, I can only make only ONE list after the loop is done. Is there a better way to fix this so that I can pull out more information from the loop? Like what if I need list2 to store other values? Thanks.
Your shell script does work. If you wanted to get two pieces of info per line insteal of one, you would have to change this line
echo $(echo ${result} | cut -f3 -d' ' | tr -d [:alpha:] | tr -d [:punct:])
to concatenate the desired values separated by a comma or any other "special" character. Then you could parse your list this way :
for haha in ${list}
do
printf "current slot is:%s, secondary info:%s \n" $(echo ${haha} | cut -f1 -d',') $(echo ${haha} | cut -f2 -d',')
done
See this explanation. As a pipe is involved, the while read... code isn't executed in your current shell, but in a subshell (A child process which can't update your current process' (environment/shell) variables).
Choose on of the listed workarounds to make the while read... loop execute in your current shell.