In Bash script trying to pass local variable to SSH and then execute the other commands - arrays

#!/bin/bash
count2=1
declare -a input
input=( "$#" )
echo " "
echo " Hostname passed by user is " ${input[0]}
HOST="${input[0]}"
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no user#$HOST /bin/bash << ENDSSH
echo " Connected "
echo $count2
echo $input
pwd
echo $count2: ${input[$count2]}
nic=${input[$count2]}
echo $nic
echo $(ethtool "${nic}" |& grep 'Link' | awk '{print $3}')
ENDSSH
So Actually want to pass variable 'count2' and 'input' to remote SSH and execute.
But unfortunately it is not getting passed. It is not echoing anything after SSH.
Need help in this.!!
I have sshpass installed in sever.
code output:
[user#l07 ~]$ ./check.sh <hostname> eno6
Hostname passed by user is <hostname>
Connected
After SSH it only echos "Connected". I'm not sure why $count2 and $input is not echoing.
I tired with backlash '\$count2' but that is also not working. All possible combination tried even with quote and unquote of ENDSSH. Pls help
Any help will be really appreciated!!

You basically want to supply to your remote bash a HERE-document to be executed. This is tricky, since you need to "compose" the full text of this document before you can supply it to ssh. I would therefore separate the task into two parts:
Creating the HERE-document
Running it on ssh
This makes it easy for debugging to output the document between steps 1 and 2 and to visually inspect its contents for correctness. Don't forget that once this code runs on the remote host, it can't access any of your variables anymore, unless you have "promoted" them to the remote side using the means provided by ssh.
Hence you could start like this:
# Create the parameters you want to use
nic=${input[$count2]}
# Create a variable holding the content of the remote script,
# which interpolates your parameters
read -r -d '' remote_script << ENDSSH
echo "Connected to host \$(hostname)"
echo "Running bash version: \$BASH_VERSION"
....
ethtool "$nic" |& grep Link | awk '{ print $3 }'
ENDSSH
# Print your script for verification
echo "$remote_script"
# Submit it to the host
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no "user#$HOST" /bin/bash <<<"$remote_script"

You have to add escapes(\) here:
...
echo \$nic
...
echo \$(ethtool "\${nic}" |& grep 'Link' | awk '{print \$3}')
...
But why echoing this? Try it without echo
...
ethtool "\${nic}" |& grep -i 'Link' | awk '{print \$3}'
...

#!/bin/bash
count2=1
declare -a input
input=( "$#" )
echo " Hostname passed by user is " "${input[0]}"
HOST="${input[0]}"
while [ $# -gt $count2 ]
do
sed -i 's/VALUE/'"${input[$count2]}"'/g' ./check.sh
sshpass -p '<pass>' scp ./check.sh user#"$HOST":/home/user/check.sh
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no user#"$HOST" "sh /home/user/check.sh && rm -rf /home/user/check.sh"
sed -i 's/'"${input[$count2]}"'/VALUE/g' ./check.sh
((count2++))
done
Found the another solution of this issue: It is working for me now !!!!
I wrote my entire logic which needs to be executed remotely in check.sh file and now replacing or storing the user input into this check.sh file and copying this file into remote server via scp and executing it over remotely and after successful execution removing this file from remote server and after ssh , again changing the user input to it's original value in local server using sed command.
Made this as dynamic script to work for multiple servers.

Related

Bash: how to print and run a cmd array which has the pipe operator, |, in it

This is a follow-up to my question here: How to write bash function to print and run command when the command has arguments with spaces or things to be expanded
Suppose I have this function to print and run a command stored in an array:
# Print and run the cmd stored in the passed-in array
print_and_run() {
echo "Running cmd: $*"
# run the command by calling all elements of the command array at once
"$#"
}
This works fine:
cmd_array=(ls -a /)
print_and_run "${cmd_array[#]}"
But this does NOT work:
cmd_array=(ls -a / | grep "home")
print_and_run "${cmd_array[#]}"
Error: syntax error near unexpected token `|':
eRCaGuy_hello_world/bash$ ./print_and_run.sh
./print_and_run.sh: line 55: syntax error near unexpected token `|'
./print_and_run.sh: line 55: `cmd_array=(ls -a / | grep "home")'
How can I get this concept to work with the pipe operator (|) in the command?
If you want to treat an array element containing only | as an instruction to generate a pipeline, you can do that. I don't recommend it -- it means you have security risk if you don't verify that variables into your string can't consist only of a single pipe character -- but it's possible.
Below, we create a random single-use "$pipe" sigil to make that attack harder. If you're unwilling to do that, change [[ $arg = "$pipe" ]] to [[ $arg = "|" ]].
# generate something random to make an attacker's job harder
pipe=$(uuidgen)
# use that randomly-generated sigil in place of | in our array
cmd_array=(
ls -a /
"$pipe" grep "home"
)
exec_array_pipe() {
local arg cmd_q
local -a cmd=( )
while (( $# )); do
arg=$1; shift
if [[ $arg = "$pipe" ]]; then
# log an eval-safe copy of what we're about to run
printf -v cmd_q '%q ' "${cmd[#]}"
echo "Starting pipeline component: $cmd_q" >&2
# Recurse into a new copy of ourselves as a child process
"${cmd[#]}" | exec_array_pipe "$#"
return
fi
cmd+=( "$arg" )
done
printf -v cmd_q '%q ' "${cmd[#]}"
echo "Starting pipeline component: $cmd_q" >&2
"${cmd[#]}"
}
exec_array_pipe "${cmd_array[#]}"
See this running in an online sandbox at https://ideone.com/IWOTfO
Do this instead. It works.
print_and_run() {
echo "Running cmd: $1"
eval "$1"
}
Example usage:
cmd='ls -a / | grep -C 9999 --color=always "home"'
print_and_run "$cmd"
Output:
Running cmd: ls -a / | grep -C 9999 --color=always "home"
(rest of output here, with the word "home" highlighted in red)
The general direction is that you don't. You do not store the whole command line to be printed later, and this is not the direction you should take.
The "bad" solution is to use eval.
The "good" solution is to store the literal '|' character inside the array (or some better representation of it) and parse the array, extract the pipe parts and execute them. This is presented by Charles in the other amazing answer. It is just rewriting the parser that already exists in the shell. It requires significant work, and expanding it will require significant work.
The end result is, is that you are reimplementing parts of shell inside shell. Basically writing a shell interpreter in shell. At this point, you can just consider taking Bash sources and implementing a new shopt -o print_the_command_before_executing option in the sources, which might just be simpler.
However, I believe the end goal is to give users a way to see what is being executed. I would propose to approach it like .gitlab-ci.yml does with script: statements. If you want to invent your own language with "debug" support, do just that instead of half-measures. Consider the following YAML file:
- ls -a / | grep "home"
- echo other commands
- for i in "stuff"; do
echo "$i";
done
- |
for i in "stuff"; do
echo "$i"
done
Then the following "runner":
import yaml
import shlex
import os
import sys
script = []
input = yaml.safe_load(open(sys.argv[1], "r"))
for line in input:
script += [
"echo + " + shlex.quote(line).replace("\n", "<newline>"), # some unicode like ␤ would look nice
line,
]
os.execvp("bash", ["bash", "-c", "\n".join(script)])
Executing the runner results in:
+ ls -a / | grep "home"
home
+ echo other commands
other commands
+ for i in "stuff"; do echo "$i"; done
stuff
+ for i in "stuff"; do<newline> echo "$i"<newline>done<newline>
stuff
This offers greater flexibility and is rather simple, supports any shell construct with ease. You can try gitlab-ci/cd on their repository and read the docs.
The YAML format is only an example of the input format. Using special comments like # --- cut --- between parts and extracting each part with the parser will allow running shellcheck over the script. Instead of generating a script with echo statements, you could run Bash interactively, print the part to be executed and then "feed" the part to be executed to interactive Bash. This will alow to preserve $?.
Either way - with a "good" solution, you end up with a custom parser.
Instead of passing an array, you can pass the whole function and use the output of declare -f with some custom parsing:
print_and_run() {
echo "+ $(
declare -f "$1" |
# Remove `f() {` and `}`. Remove indentation.
sed '1d;2d;$d;s/^ *//' |
# Replace newlines with <newline>.
sed -z 's/\n*$//;s/\n/<newline>/'
)"
"$#"
}
cmd() { ls -a / | grep "home"; }
print_and_run cmd
Results in:
+ ls --color -F -a / | grep "home"
home/
It will allow for supporting any shell construct and still allow you to check it with shellcheck and doesn't require that much work.

Using loop to convert multiple files into separate files

I used this command to convert multiple pcap log files to text using tcpdump :
$ cat /home/dalya/snort-2.9.9.0/snort_logs/snort.log.* | tcpdump -n -r - > /home/dalya/snort-2.9.9.0/snort_logs2/bigfile.txt
and it worked well.
Now I want to separate the output, each converted file in a separate output file using loop like this :
for f in /home/dalya/snort-2.9.9.0/snort_logs/snort.log.* ; do
tcpdump -n -r "$f" > /home/dalya/snort-2.9.9.0/snort_logs2/"$f.txt" ;
done
But it gave me :
bash: /home/dalya/snort-2.9.9.0/snort_logs2//home/dalya/snort-2.9.9.0/snort_logs/snort.log.1485894664.txt: No such file or directory
bash: /home/dalya/snort-2.9.9.0/snort_logs2//home/dalya/snort-2.9.9.0/snort_logs/snort.log.1485894770.txt: No such file or directory
bash: /home/dalya/snort-2.9.9.0/snort_logs2//home/dalya/snort-2.9.9.0/snort_logs/snort.log.1487346947.txt: No such file or directory
I think the problem in $f, Where did I go wrong?
If you run
for f in /home/dalya/snort-2.9.9.0/snort_logs/snort.log.* ; do
echo $f
done
You'll find that you're getting
/home/dalya/snort-2.9.9.0/snort_logs/snort.log.1485894664
/home/dalya/snort-2.9.9.0/snort_logs/snort.log.1485894770
/home/dalya/snort-2.9.9.0/snort_logs/snort.log.1487346947
You can use basename
To get only the filename, something like this:
for f in /home/dalya/snort-2.9.9.0/snort_logs/snort.log.* ; do
base="$(basename $f)"
echo $base
done
Once you're satisfied that this is working, remove the echo statement and use
tcpdump -n -r "$f" > /home/dalya/snort-2.9.9.0/snort_logs2/"$base.txt"
instead.
Edit: tcpdump -n -r "$base" > ... should have been tcpdump -n -r "$f" > ...; you only want to use $base in the context of creating the new filename, not in the context of reading the existing data.

Populate array to ssh in bash

Just some background, I have a file with 1000 servers in it new line delimted. I have to read them to an array the run about 5 commands over SSH. I have been using heredoc notation but that seems to fail. Currently I get an error saying the host isn't recognized.
IFS='\n' read -d '' -r -a my_arr < file
my_arr=()
for i in "${my_arr[#]}"; do
ssh "$1" bash -s << "EOF"
echo "making back up of some file"
cp /path/to/file /path/to/file.bak
exit
EOF
done
I get output that lists the first server but then all the ones in the array as well. I know that I am missing a redirect for STDIN that causes this.
Thanks for the help.
Do you need an array? What is wrong with:
while read -r host
do
ssh "$host" bash -s << "EOF"
echo "making back up of some file"
cp /path/to/file /path/to/file.bak
EOF
done < file
To be clear -- the problem here, and the only problem present in the code actually included in your question, is that you're using $1 inside your loop, whereas you specified $i as the variable that contains the entry being iterated over on each invocation of the loop.
That is to say: ssh "$1" needs to instead by ssh "$i".

Output of wget (to CouchDB) as array used in for-loop

I try to use the output of wget to build an array I can use for a for loop:
RESULT="`wget -qO- http://myCouchDBServer.com:5984/_all_dbs`"
echo $RESULT
What I get is something like:
["_replicator","_users","mydb1","mydb2","mydb3","mydb4","mydb5"]
At the moment I loop through an array i wrote manually:
databases=(mydb1 mydb2 mydb2)
for var in "${databases[#]}"
do
# echo "${var}"
# do something on $var
filename="${var}_$(date +%Y_%m_%d_%H_%M_%S).json"
echo $filename
bash couchdb-backup.sh -b -H myHostIP_Adress -d ${var} -f $filename -u myUsername -p myPassword
done
Now I would like to use the response from wget and build an array from this response which I can use for my for-loop.
I tried
RESULT2= echo "$RESULT" | sed 's/\(\[\|\]\)//g'
to get rid of the square brackets and tried to build an array
but there is no output and therefore also no loop running...
STR_ARRAY=(`echo $RESULT2 | tr "," "\n"`)
for x in "${STR_ARRAY[#]}"
do
echo "> [$x]"
done
Any idea how to get a step further?
PS:
(my couchdb is only reachable through intranet but I found an url for testing: http://example.iriscouch.com/_all_dbs )
Since couch is spitting out JSON, you'll want to process it with a JSON parser. For example, jq:
echo '["_replicator","_users","mydb1","mydb2","mydb3","mydb4","mydb5"]' | jq -r '.[]|.'
_replicator
_users
mydb1
mydb2
mydb3
mydb4
mydb5
So, you could do this:
mapfile -t all_dbs < <(
wget -qO- http://myCouchDBServer.com:5984/_all_dbs | jq -r '.[]|.'
)
for db in "${all_dbs[#]}"; do ...; done

Check database connectivity using Shell script

I am trying to write a shell script to check database connectivity. Within my script I am using the command
sqlplus uid/pwd#database-schemaname
to connect to my Oracle database.
Now I want to save the output generated by this command (before it drops to SQL prompt) in a temp file and then grep / find the string "Connected to" from that file to see if the connectivity is fine or not.
Can anyone please help me to catch the output and get out of that prompt and test whether connectivity is fine?
Use a script like this:
#!/bin/sh
echo "exit" | sqlplus -L uid/pwd#dbname | grep Connected > /dev/null
if [ $? -eq 0 ]
then
echo "OK"
else
echo "NOT OK"
fi
echo "exit" assures that your program exits immediately (this gets piped to sqlplus).
-L assures that sqlplus won't ask for password if credentials are not ok (which would make it get stuck as well).
(> /dev/null just hides output from grep, which we don't need because the results are accessed via $? in this case)
You can avoid the SQL prompt by doing:
sqlplus uid/pwd#database-schemaname < /dev/null
SqlPlus exits immediately.
Now just grep the output of the above as:
if sqlplus uid/pwd#database-schemaname < /dev/null | grep 'Connected to'; then
# have connectivity to Oracle
else
# No connectivity
fi
#! /bin/sh
if echo "exit;" | sqlplus UID/PWD#database-schemaname 2>&1 | grep -q "Connected to"
then echo connected OK
else echo connection FAIL
fi
Not knowing whether the "Connected to" message is put to standard output or standard error, this checks both. "qrep -q" instead of "grep... >/dev/null" assumes Linux.
#!/bin/bash
output=`sqlplus -s "user/pass#POLIGON.TEST " <<EOF
set heading off feedback off verify off
select distinct machine from v\\$session;
exit
EOF
`
echo $output
if [[ $output =~ ERROR ]]; then
echo "ERROR"
else
echo "OK"
fi
Here's a good option which does not expose the password on the command line
#!/bin/bash
CONNECT_STRING=<USERNAME>/<PASS>#<SID>
sqlplus -s -L /NOLOG <<EOF
whenever sqlerror exit 1
whenever oserror exit 1
CONNECT $CONNECT_STRING
exit
EOF
SQLPLUS_RC=$?
echo "RC=$SQLPLUS_RC"
[ $SQLPLUS_RC -eq 0 ] && echo "Connected successfully"
[ $SQLPLUS_RC -ne 0 ] && echo "Failed to connect"
exit SQLPLUS_RC
none of the proposed solutions works for me, as my script is executed in machines running several countries, with different locales, I can't simply check for one String simply because this string in the other machine is translated to a different language. As a solution I'm using SQLcl
https://www.oracle.com/database/technologies/appdev/sqlcl.html
which is compatible with all sql*plus scripts and allow you to test the database connectivity like this:
echo "disconnect" | sql -L $DB_CONNECTION_STRING > /dev/null || fail "cannot check connectivity with the database, check your settings"
#!/bin/sh
echo "exit" | sqlplus -S -L uid/pwd#dbname
if [ $? -eq 0 ]
then
echo "OK"
else
echo "NOT OK"
fi
For connection validation -S would be sufficient.
The "silent" mode doesn't prevent terminal output. All it does is:
-S Sets silent mode which suppresses the display of
the SQL*Plus banner, prompts, and echoing of
commands.
If you want to suppress all terminal output, then you'll need to do something like:
sqlplus ... > /dev/null 2>&1
This was my one-liner for docker container to wait until DB is ready:
until sqlplus -s sys/Oracle18#oracledbxe/XE as sysdba <<< "SELECT 13376411 FROM DUAL; exit;" | grep "13376411"; do echo "Could not connect to oracle... sleep for a while"; sleep 3; done
And the same in multiple lines:
until sqlplus -s sys/Oracle18#oracledbxe/XE as sysdba <<< "SELECT 13376411 FROM DUAL; exit;" | grep "13376411";
do
echo "Could not connect to oracle... sleep for a while";
sleep 3;
done
So it basically does select with magic number and checks that correct number was actually returned.

Resources