I try to use the output of wget to build an array I can use for a for loop:
RESULT="`wget -qO- http://myCouchDBServer.com:5984/_all_dbs`"
echo $RESULT
What I get is something like:
["_replicator","_users","mydb1","mydb2","mydb3","mydb4","mydb5"]
At the moment I loop through an array i wrote manually:
databases=(mydb1 mydb2 mydb2)
for var in "${databases[#]}"
do
# echo "${var}"
# do something on $var
filename="${var}_$(date +%Y_%m_%d_%H_%M_%S).json"
echo $filename
bash couchdb-backup.sh -b -H myHostIP_Adress -d ${var} -f $filename -u myUsername -p myPassword
done
Now I would like to use the response from wget and build an array from this response which I can use for my for-loop.
I tried
RESULT2= echo "$RESULT" | sed 's/\(\[\|\]\)//g'
to get rid of the square brackets and tried to build an array
but there is no output and therefore also no loop running...
STR_ARRAY=(`echo $RESULT2 | tr "," "\n"`)
for x in "${STR_ARRAY[#]}"
do
echo "> [$x]"
done
Any idea how to get a step further?
PS:
(my couchdb is only reachable through intranet but I found an url for testing: http://example.iriscouch.com/_all_dbs )
Since couch is spitting out JSON, you'll want to process it with a JSON parser. For example, jq:
echo '["_replicator","_users","mydb1","mydb2","mydb3","mydb4","mydb5"]' | jq -r '.[]|.'
_replicator
_users
mydb1
mydb2
mydb3
mydb4
mydb5
So, you could do this:
mapfile -t all_dbs < <(
wget -qO- http://myCouchDBServer.com:5984/_all_dbs | jq -r '.[]|.'
)
for db in "${all_dbs[#]}"; do ...; done
Related
My first bash script.
myvar=$(aws ec2 describe-regions|jq '.Regions[] | {RegionName: .RegionName }')
echo "${myvar}"| jq -c '.| .RegionName'
Using the 2 commands above. I successfully got the the following result in terminal
"af-south-1"
"eu-north-1"
"ap-south-1"
"eu-west-3"
"eu-west-2"
"eu-west-1"
"ap-northeast-3"
"ap-northeast-2"
"me-south-1"
"ap-northeast-1"
"sa-east-1"
"ca-central-1"
"ap-east-1"
"ap-southeast-1"
"ap-southeast-2"
"eu-central-1"
"us-east-1"
"us-east-2"
"us-west-1"
"us-west-2"
How to save the answer into an array, and iterate it?
I thought about
awsRegionList =$(${myvar}| jq -c '.| .RegionName')
echo awsRegionList
But it finishes with an error.
First of all you can simplify the jq query to process the EC2 output in one go.
#!/usr/bin/env bash
# Map the output of jq parsed EC2 response into the $regionList array
mapfile -d '' regionList < <(
# Gets EC2 response in JSON piped
aws ec2 describe-regions |
# Processes JSON into a null-delimited stream
jq --join-output '.Regions[] | .RegionName + "\u0000"'
)
# Iterates the list
for regionName in "${regionList[#]}"; do
# Do stuffs with $regionName
printf 'Doing stuff with: %s\n' "$regionName"
done
You could create a single jq command to save the output into an array. Afterwards you can iterate over it with a for loop:
#!/bin/bash
awsRegionList=($(aws ec2 describe-regions | jq -r -c '.Regions[] | .RegionName'))
for region in ${awsRegionList[#]}; do
echo $region
done;
Note, using -r will remove the quotes as well, so the output will be something like:
eu-north-1
ap-south-1
eu-west-3
eu-west-2
eu-west-1
ap-northeast-3
ap-northeast-2
ap-northeast-1
sa-east-1
ca-central-1
ap-southeast-1
ap-southeast-2
eu-central-1
us-east-1
us-east-2
us-west-1
us-west-2
Data in log file is something like:
"ipHostNumber: 127.0.0.1
ipHostNumber: 127.0.0.2
ipHostNumber: 127.0.0.3"
that's my code snippet:
readarray -t iparray < "$destlog"
unset iparray[0]
for ip in "${iparray[#]}"
do
IFS=$'\n' read -r -a iparraychanged <<< "${iparray[#]}"
done
And what I wanted to receive is to transfer IP's to another array and then read every line from that array and ping it.
UPDATE: I've acknowledged something, it's probably that I want one array to another but without some string, in this case cut "ipHostNumber: " and have remained only IPs.
Thanks in advance, if there's anything missing please let me know.
Why do you need arrays at all? Just read the input and do the work.
while IFS=' ' read -r _ ip; do
ping -c1 "$ip"
done < "$destlog"
See https://mywiki.wooledge.org/BashFAQ/001 .
Another way is to use xargs and filtering:
awk '{print $2}' "$destlog" | xargs -P0 -n1 ping -c1
I prefer KamilCuk's xargs solution, but if you just wanted the array because you plan to reuse it -
$: readarray -t ip < <( sed '1d; s/^.* //' file )
Now it's loaded.
$: declare -p ip
declare -a ip=([0]="127.0.0.2" [1]="127.0.0.3")
$: for addr in "${ip[#]}"; do ping -c 1 "$addr"; done
#!/bin/bash
count2=1
declare -a input
input=( "$#" )
echo " "
echo " Hostname passed by user is " ${input[0]}
HOST="${input[0]}"
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no user#$HOST /bin/bash << ENDSSH
echo " Connected "
echo $count2
echo $input
pwd
echo $count2: ${input[$count2]}
nic=${input[$count2]}
echo $nic
echo $(ethtool "${nic}" |& grep 'Link' | awk '{print $3}')
ENDSSH
So Actually want to pass variable 'count2' and 'input' to remote SSH and execute.
But unfortunately it is not getting passed. It is not echoing anything after SSH.
Need help in this.!!
I have sshpass installed in sever.
code output:
[user#l07 ~]$ ./check.sh <hostname> eno6
Hostname passed by user is <hostname>
Connected
After SSH it only echos "Connected". I'm not sure why $count2 and $input is not echoing.
I tired with backlash '\$count2' but that is also not working. All possible combination tried even with quote and unquote of ENDSSH. Pls help
Any help will be really appreciated!!
You basically want to supply to your remote bash a HERE-document to be executed. This is tricky, since you need to "compose" the full text of this document before you can supply it to ssh. I would therefore separate the task into two parts:
Creating the HERE-document
Running it on ssh
This makes it easy for debugging to output the document between steps 1 and 2 and to visually inspect its contents for correctness. Don't forget that once this code runs on the remote host, it can't access any of your variables anymore, unless you have "promoted" them to the remote side using the means provided by ssh.
Hence you could start like this:
# Create the parameters you want to use
nic=${input[$count2]}
# Create a variable holding the content of the remote script,
# which interpolates your parameters
read -r -d '' remote_script << ENDSSH
echo "Connected to host \$(hostname)"
echo "Running bash version: \$BASH_VERSION"
....
ethtool "$nic" |& grep Link | awk '{ print $3 }'
ENDSSH
# Print your script for verification
echo "$remote_script"
# Submit it to the host
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no "user#$HOST" /bin/bash <<<"$remote_script"
You have to add escapes(\) here:
...
echo \$nic
...
echo \$(ethtool "\${nic}" |& grep 'Link' | awk '{print \$3}')
...
But why echoing this? Try it without echo
...
ethtool "\${nic}" |& grep -i 'Link' | awk '{print \$3}'
...
#!/bin/bash
count2=1
declare -a input
input=( "$#" )
echo " Hostname passed by user is " "${input[0]}"
HOST="${input[0]}"
while [ $# -gt $count2 ]
do
sed -i 's/VALUE/'"${input[$count2]}"'/g' ./check.sh
sshpass -p '<pass>' scp ./check.sh user#"$HOST":/home/user/check.sh
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no user#"$HOST" "sh /home/user/check.sh && rm -rf /home/user/check.sh"
sed -i 's/'"${input[$count2]}"'/VALUE/g' ./check.sh
((count2++))
done
Found the another solution of this issue: It is working for me now !!!!
I wrote my entire logic which needs to be executed remotely in check.sh file and now replacing or storing the user input into this check.sh file and copying this file into remote server via scp and executing it over remotely and after successful execution removing this file from remote server and after ssh , again changing the user input to it's original value in local server using sed command.
Made this as dynamic script to work for multiple servers.
I tried to fetch all repos from a git user and put them into a array within a shell script. Somehow the array doesn´t recognize the new line as a separator and acts like there´s only one multi line element within the array.
Here´s my sample code:
someUser=Joe
declare -a repos=$(curl -s "https://api.github.com/users/$someUser/repos?page=$PAGE&per_page=100" | grep -e 'git_url*' | cut -d \" -f 4 | cut -d"/" -f5 | cut -d"." -f1)
for repo in $repos; do
echo $repo
// some more stuff
done
The output from the curl and cut looks like that:
RepoA
RepoB
RepoC
[...]
How do I get a new line element treated as a new element within the array? I use the array several times so I need one fixed container with all the repositories.
This is the correct way to iterate over the elements of a Bash array:
for repo in "${repos[#]}"; do
Also, to create an array with the output of a command, you need to wrap the $(...) subshell within (...), like this:
declare -a repos=($(curl -s ...))
Thanks to #janos i fixed the script. The additionsl bracket within the delcare line was the problem. Here is the full code. Maybe somebody would like to copy it.
#!/bin/bash
gitUrl="https://github.com"
gitUser="foobar"
cloneCmd="git clone"
fetchCmd="git fetch"
magenta="\033[35m"
green="\033[32m"
def="\033[0m"
declare -a repos=($(curl -s "https://api.github.com/users/$gitUser/repos?page=$PAGE&per_page=100" | grep -e 'git_url*' | cut -d \" -f 4 | cut -d"/" -f5 | cut -d"." -f1))
# Init clone
echo -e "$magenta Cloning new Repositories $def"
for repo in "${repos[#]}"
do
if [ -d $repo ]; then
echo -e "$green \tRepo $repo already exists $def"
continue
fi
$cloneCmd $gitUrl/$gitUser/$repo
done
echo -e "$green Colning finished $def"
# Update Repos
echo -e "$magenta Updating Repositories $def"
for repo in "${repos[#]}"
do
cd $repo
$fetchCmd
cd ..
done
echo -e "$green Update finished $def"
This script was working just fine on an AIX box, but now on RH linux box the arrays just doesn't seem to work. Version on the new RH box is 4.1.2
I declare my array
declare -a gridNames=()
I get information about a grid
gridstats=`snmpwalk -v 2c -c splunk $host gridStatsTable -m $APPLIANCEMIB -OUQs -Ln`
As well as getting the stats from the above, I reuse it to find all the gridNames and then will use the array of gridNames to get stats about their maps.
while read -r process; do
gridNames=(${gridNames[#]} `grep gridName | awk -F "\"" '{print $(NF-1)}'`)
done <<< "$gridstats"
The awk part is tested and correctly returns a list of gridnames (just one in this case) but when I echo the array gridNames its empty.
I have also tried using
gridNames+=(`grep gridName | awk -F "\"" '{print $(NF-1)}'`)
but that doesnt work either
You need to use += operator for appending elements to an array with process substitution:
while read -r process; do
gridNames+=( $(grep 'gridName' | awk -F '"' '{print $(NF-1)}' <<< "$process") )
done < <(snmpwalk -v 2c -c splunk $host gridStatsTable -m $APPLIANCEMIB -OUQs -Ln)