bash to parse XML into multidimensional array - arrays

I have a XML file which I want to parse into bash variables/arrays.
I have limited linux commands (busybox) available since I am working on a NAS box.
My XML file looks like this:
<?xml version="1.0" encoding="UTF-8"?> .
<WEBCAMS>
<CAM>
<DESCRIPTION>description for cam 1</DESCRIPTION>
<URL>http://myURLtoWebcam1/cam1/pic.jpg</URL>
<FILENAME>filename1</FILENAME>
</CAM>
<CAM>
<DESCRIPTION>description for cam 2</DESCRIPTION>
<URL>http://myURLtoWebcam2/cam2/pic.jpg</URL>
<FILENAME>filename2</FILENAME>
</CAM>
</WEBCAMS>
my bash script so far:
#!/bin/sh
rdom () { local IFS=\> ; read -d \< E C ;}
while rdom; do
if [[ $E = DESCRIPTION ]]; then
counter=$((counter+1))
declare cam$counter="$C"
fi
done < webcams.xml
I want to get the XML content like the following:
echo "Cam1 description: ${cam1[0]}"; ## should show: description for cam 1
echo "Cam1 URL: ${cam1[1]}"; ## should show: http://myURLtoWebcam1/cam1/pic.jpg
echo "Cam1 filename: ${cam1[2]}"; should show: filename1
echo "Cam2 description: ${cam2[0]}"; ## should show: description for cam 2
echo "Cam2 URL: ${cam2[1]}"; ## should show: http://myURLtoWebcam1/cam2/pic.jpg
echo "Cam2 filename: ${cam2[2]}"; ## should show: filename2
So far I am only able to read the "DESCRIPTION" fields into bash variables.
Any idea how to get the other fields "URL" and "FILENAME" into my arrays/variables? The so far found solutions did not fit or could not be modified to my needs due to the limited Linux commands on my NAS.

If XSLTPROC is available you can use it - bonus its a real XML parser.
> xsltproc transform.xsl webcams.xml
Cam1 description: description for cam 1
Cam1 URL: http://myURLtoWebcam1/cam1/pic.jpg
Cam1 filename: filename1
Cam2 description: description for cam 2
Cam2 URL: http://myURLtoWebcam2/cam2/pic.jpg
Cam2 filename: filename2
Where transform.xsl is
<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="text" omit-xml-declaration="yes" indent="no"/>
<xsl:strip-space elements="*"/>
<xsl:template match="CAM">
<xsl:variable name="i" select="position()" />
Cam<xsl:value-of select="$i"/> description: <xsl:value-of select="DESCRIPTION"/>
Cam<xsl:value-of select="$i"/> URL: <xsl:value-of select="URL"/>
Cam<xsl:value-of select="$i"/> filename: <xsl:value-of select="FILENAME"/>
</xsl:template>
<xsl:template match="/WEBCAMS"><xsl:apply-templates select="*"/>
<xsl:text>
</xsl:text>
</xsl:template>
</xsl:stylesheet>

This seems to work ... but still the 1,5D dimensional array or the way to escape variables makes me headache - see below problems within a FOR loop:
#!/bin/sh
rdom () { local IFS=\> ; read -d \< E C ;}
while rdom; do
if [[ $E = DESCRIPTION ]]; then
counter0=$((counter0+1))
declare -a cam$((counter0))[0]="$C"
fi
if [[ $E = URL ]]; then
counter1=$((counter1+1))
declare -a cam$((counter1))[1]="$C"
fi
if [[ $E = FILENAME ]]; then
counter2=$((counter2+1))
declare -a cam$((counter2))[2]="$C"
fi
done < webcams.xml
echo "Cam1 description: ${cam1[0]}";
echo "Cam1 URL: ${cam1[1]}";
echo "Cam1 filename: ${cam1[2]}";
echo "Cam2 description: ${cam2[0]}";
echo "Cam2 URL: ${cam2[1]}";
echo "Cam2 filename: ${cam2[2]}";
But it is still impossible to get the values within a FOR loop like:
for (( c=1; c<=$counter0; c++ ))
do
var=cam$c;
echo "Cam$c description: ${!var[0]}";
echo "Cam$c URL: ${!var[1]}";
echo "Cam$c filename: ${!var[2]}";
done

Related

bad substitution when creating nested array variables in bash

I don't know what I am doing and could use some assistance with my script.
$ ./mysql.sh LOCALIP 'SELECT LOCALIP FROM Host'
mysql.sh
#!/bin/bash
source $PWD/data/login
mapfile -t "$1" < <(mysql -N ""$DB"" -h""$HOST"" -u""$USER"" -p""$PASS"" -se "$2")
echo ${$1[0]}
echo ${$1[1]}
echo ${$1[2]}
echo ${$1[3]}
fi
Output
[シ]owner#gwpi ~/scriptdir $./mysql.sh LOCALIP 'SELECT LOCALIP FROM Host'
./mysql.sh: line 10: ${$1[0]}: bad substitution
Simply replace the varible $1 with var here.It works.
$ mapfile -t var < <(mysql -N ""$DB"" -h""$HOST"" -u""$USER"" -p""$PASS"" -se "$2")
$ echo" ${var[#]}"
Original script has two problems.
You cannot change $1 arg in this way.Right style refers to this
should be better using more meaningful variables than $1

Bash array not passed out of loop

Am having this little bash script to build a custom ROM (see below)
Now the issue I encounter is the fact that I can't seem to be able to read the content of the array after the loop is done
While calling it before the new loop pass, all is good, however when the loop is done, I can not seem to be able to read it (seems to be empty)
I need to say that I am no bash expert and I usually do my code from out pal Google
What am I doing wrong here?
#!/bin/bash
#Build crDroid and optionally upload to FTP for all devices in devices.txt
# << Start configuration >>
#crDroid_path is the location of build environment root path aka {ANDROID_BUILD_TOP}
crDroid_path=~/crDroid
#set how much RAM is used by JACK if building with
RAM=12
#CCache size
ccachesize=30G
#set if you want to save changelog file to script_path (from where the script runs) at end of build (useful to add changelog info to forums and so on... easy to find)
copy_changelog=true
#FTP config
upload_build=true
FTP_hostname="ftp://domain.tld"
FTP_username=username
FTP_password=password
# << End configuration >>
# Specify colors utilized in the terminal
red=$(tput setaf 1) # red
grn=$(tput setaf 2) # green
ylw=$(tput setaf 3) # yellow
blu=$(tput setaf 4) # blue
cya=$(tput rev)$(tput bold)$(tput setaf 6) # bold cyan reversed
ylr=$(tput rev)$(tput bold)$(tput setaf 3) # bold yellow reversed
grr=$(tput rev)$(tput bold)$(tput setaf 2) # bold green reversed
txtrst=$(tput sgr0) # Reset
echo "==========================================="
echo "${cya}Initiate build script - v2.9 by Gabriel Lup (gwolfu#xda)"${txtrst}
echo "==========================================="
#detect path where the script is running
script_path="`dirname \"$0\"`" # relative
script_path="`( cd \"$script_path\" && pwd )`" # absolutized and normalized
if [ -z "$script_path" ] ; then
# error; for some reason, the path is not accessible
echo "${red}Can not read run path"
echo "Build can not continue"${txtrst}
exit 1 # fail
fi
#check if devices.txt exists
devices=$script_path/devices.txt
if [ ! -f $devices ]; then
echo "${red}devices.txt missing"
echo "Build can not continue"${txtrst}
exit 1 # fail
fi
#cleanup old changelog.txt
changelog=$script_path/changelog.txt
if [ -e $changelog ]; then
rm -f $changelog
fi
#check if already synced
crdroid_synced=$crDroid_path/vendor/lineage/config/crdroid.mk
if [ ! -f $crdroid_synced ]; then
echo "${ylw}Detected missing first sync... atempting first time sync..."${txtrst}
cd $crDroid_path
repo sync -f --force-sync --no-clone-bundle
fi
if [ ! -f $crdroid_synced ]; then
read -p "${red}Something went wrong :( - Maybe misconfigured script!?"
exit
fi
cd $crDroid_path
echo "${blu}Run sync?${txtrst}"
select yn in "Yes" "No"; do
case $yn in
Yes ) repo sync -f --force-sync --no-clone-bundle; break;;
No ) break;;
esac
done
echo "${blu}Make clean build?${txtrst}"
select yn in "Yes" "No"; do
case $yn in
Yes ) . build/envsetup.sh && make clean; break;;
No ) break;;
esac
done
#check if repopick.txt exists and execute commands from it
repopick_file=$script_path/repopick.txt
if [ -f $repopick_file ]; then
. build/envsetup.sh
cat $script_path/repopick.txt | while read line
do
$line
done
fi
INFO=()
#Set CCache size
$crDroid_path/prebuilts/misc/linux-x86/ccache/ccache -M $ccachesize
#detect android version based on crdroid.mk
rl2="`sed -n '2p' $crDroid_path/vendor/lineage/config/crdroid.mk`"
set -- "$rl2"
IFS=" "; declare -a android_major=($*)
rl3="`sed -n '3p' $crDroid_path/vendor/lineage/config/crdroid.mk`"
set -- "$rl3"
IFS=" "; declare -a android_minor=($*)
android=${android_major[2]}"."${android_minor[2]}
#detect crDroid version based on crdroid.mk
rl6="`sed -n '6p' $crDroid_path/vendor/lineage/config/crdroid.mk`"
set -- "$rl6"
IFS=" "; declare -a crDroid_version=($*)
crDroid=${crDroid_version[2]}
echo "==========================================="
echo "${ylr}Setting build environment"${txtrst}
echo "-------------------------------------------"
echo "Script path set to: "${grn}$script_path${txtrst}
echo "crDroid path set to: "${grn}$crDroid_path${txtrst}
echo "Jack RAM usage set to: "${grn}$RAM"GB RAM"${txtrst}
echo "Copy changelog to script path at end of build?: "${grn}$copy_changelog${txtrst}
echo "Upload complete build to FTP?: "${grn}$upload_build${txtrst}
echo "Trying to compile crDroid "${grn}$crDroid${txtrst}" based on Android "${grn}$android${txtrst}
echo "==========================================="
echo ""
echo "${ylw}Initiate build for all devices...${txtrst}" #described in devices.txt
cat $script_path/devices.txt | while read line
do
IFS=', ' read -r -a device_uploadpath <<< "$line"
#set device name
device=${device_uploadpath[0]}
if [[ $device == *"#"* ]]; then
device="${device//#}"
echo "${red}Found comment (#): Skipping build for ${ylw}$device ${red}${txtrst}"
INFO+=('1 '$device' (Reason: instruction to skip found in devices.txt)')
else
echo "${grn}Now building "${ylw}$device${txtrst}
#set BuildID - aka name of the zip file from OUT folder at the end of the build
BuildID="crDroidAndroid-"$android"-"$(date -d "$D" '+%Y')$(date -d "$D" '+%m')$(date -d "$D" '+%d')"-"$device"-v"$crDroid"-BETA.zip"
#Jack settings
echo "Adding "$RAM" RAM to JACK"
export JACK_SERVER_VM_ARGUMENTS="-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx"$RAM"g"
./prebuilts/sdk/tools/jack-admin kill-server
./prebuilts/sdk/tools/jack-admin start-server
#initiate build script and start actual build
. build/envsetup.sh
brunch $device
#define upload file path
uploadfile=$crDroid_path/out/target/product/$device/$BuildID
echo "Checking build result status..."
if [ -e "$uploadfile" ]; then
echo "${grn}Seems compilation is ready!${txtrst}"
if [ "$upload_build" = true ] ; then
echo "${grn}Uploading new "$device" build to FTP server${txtrst}"
echo "${ylw}Creating folder over FTP if not exist${txtrst}"
curl $FTP_hostname/${device_uploadpath[1]}/ --user $FTP_username:$FTP_password --ftp-create-dirs
echo "${ylw}Uploading...${txtrst}"
curl -T $uploadfile $FTP_hostname/${device_uploadpath[1]}/ --user $FTP_username:$FTP_password
res=$?
if test "$res" != "0"; then
echo "${red}Upload of build for device $device failed with code $res${txtrst}"
INFO+=('1 '$device' (Reason: CURL error no.'$res' ,however compilation appears to be made)')
else
INFO+=('2 '$device)
fi
fi
if [ "$copy_changelog" = true ] ; then
echo "Copy changelog file to "$script_path
cp $crDroid_path/out/target/product/$device/system/etc/Changelog.txt $script_path/changelog.txt
fi
else
echo "${red}Device "$device "did not produce a proper build${txtrst}"
INFO+=('0 '$device' (Reason: unknown - better run a new build manually)')
fi
fi
done
echo ${grr}"Script finished with following results"${txtrst}
for i in "${INFO[#]}"
do
#echo $i
if [[ $i == *"0"* ]]; then
codename=${i//0}
echo "${red}Compilation error for device" $codename${txtrst}
elif [[ $i == *"1"* ]]; then
codename=${i//1}
echo "${ylw}Warning for device" $codename${txtrst}
else
codename=${i//2}
echo "${grn}Device" $codename "compiled and uploaded successfully :)${txtrst}"
fi
done
This is the part that I am lost
echo ${grr}"Script finished with following results"${txtrst}
for i in "${INFO[#]}"
do
#echo $i
if [[ $i == *"0"* ]]; then
codename=${i//0}
echo "${red}Compilation error for device" $codename${txtrst}
elif [[ $i == *"1"* ]]; then
codename=${i//1}
echo "${ylw}Warning for device" $codename${txtrst}
else
codename=${i//2}
echo "${grn}Device" $codename "compiled and uploaded successfully :)${txtrst}"
fi
done

BASH - find regex in array, print found array items

I have tested the regex here:
http://regexr.com/3bchs
I cant get the array to only print the regex search terms.
files=(`ls $BACKUPDIR`)
daterange='(2015\-06\-)[0-9]+\s?'
for i in "${files[#]}"
do
if [[ "$files[$i]" =~ $daterange ]];
then
echo $i
fi
done
Input: 2015-06-06 2015-06-13 2015-06-20 2015-06-27 2015-07-04 2015-07-11
Output:
2015-06-06
2015-06-13
2015-06-20
2015-06-27
2015-07-04
2015-07-11
By running bash -vx <script> I found out that the files it was compering was wrong. I needed to change $files[$i] to $i.
$files[$i] = 2015-06-06[2015-06-06]
I have further improved my answer thanks to Etan Reisner comment. By not parsing output from ls.
Reference: https://stackoverflow.com/a/11584156/3371795
#!/bin/bash
# Enable special handling to prevent expansion to a
# literal '/example/backups/*' when no matches are found.
shopt -s nullglob
# Variables
YEAR=`date +%Y`
MONTH=`date +%m`
DIR=(/home/user/backups/backup.weekly/*)
# REGEX - Get curent month
DATE_RANGE='('$YEAR'\-'$MONTH'\-)[0-9]+\s?'
# Loop through dir
for i in "${DIR[#]}";
do
# Compare dir name with date range
if [[ "$i" =~ $DATE_RANGE ]];
then
# I have no idea what this is, works fine without it.
[[ -d "$i" ]]
# Echo found dirs
echo "$i"
fi
done

Appending to elements in a Bash associative array

My name is Mike and I am a total noob at Bash programming. With that being said, please don't chew me to a pulp. Here's my scenario. I'm importing a csv and processing it line by line. It looks up information in AD and pipes that result to a variable $ADresult1.
I then take that output and send it into the array.
declare -A arrmail
arrmail[$ADresult1]="$v1, $v3"
appendvar+="; ${arrmail[$ADresult1]}"
if [ ! -z "${arrmail[$ADresult1]}" ]; then
echo ${arrmail[#]}
else
echo ${arrmail[#]}="$appendvar"
fi
Now in my CSV file, the var $ADresult1 has more than one entry that is the same, however each element should be different.
For example: The first line might contain arrmail[p100]=Y2K, Y2J
The second line might arrmail[p100]=GGG, GG1
My question would be how to combine the elements from the first line and second line into one line of output and/or if this is possible.
For instance arrmail[p100]=Y2K, Y2J; GGG, GG1
If needs be, I can post the whole entire script for a bigger picture.
Here's the script.
#!/bin/bash
###--------------------------------------------------------------------------
# Author : Volha Zhdanava (p146819)
# Date : 11-FEB-2013
# Purpose: To automate the process of sending out notification emails to developers when
# Assigned DEV Machine is on the Expired Devices List.
#----------------------------------------------------------------------------
# ----MODIFIED----
#----------------------------------------------------------------------------------------------------
# Author Revsn Date Revsn ID Revsn Purpose
# Mike Bell 02-28-14 Rev.03 Add ability to email j or p number
# Mike Bell 01-02-15 Rev.04 Add verbiage about 90 day licensing
###----------------------------------------------------------------------------------------------------
Purple='\e[1;35m'
ColorOFF='\e[0m'
TO_TEAM="dev-tools#edwardjones.com"
#-----------------------------------------
#Ask for input CSV file
#-----------------------------------------
FILEDIR="/export/share/is/dev_env/expdev"
FILENAME="expired_devices.csv"
#if the directory is not empty, find the csv file
if [[ $(ls -A $FILEDIR) ]]
then
#check if the csv file was stored with default name <expired_devices.csv>
if [ -f $FILEDIR/$FILENAME ]
then
#absolute path to the downloaded file
CSVFILE=`echo $FILEDIR/$FILENAME`
#if the csv file was stored under a user-defined name,
#re-define the absolute path to that file
else
#count how many files are saved in the expdev directory
NAME=( `ls -A $FILEDIR` )
CNT=${#NAME[#]}
echo -e "There is/are ${Purple}${CNT}${ColorOFF} file(s) in the ${Purple}$FILEDIR${ColorOFF} directory:"
echo ${NAME[#]}
#if there is only one file saved within the expdev directory
if [ ${CNT} -eq 1 ]
then
FILENAME=${NAME[0]}
else
#it will prompt to enter the name of the file
REPLY="n"
while [ "$REPLY" != "y" ]
do
echo -e "Please enter the full name of the downloaded csv file: \c"
read FILENAME
echo -e "You entered ${Purple}$FILENAME${ColorOFF}. Is this correct? y/n: \c\n"
read ANSWER
REPLY=`echo $ANSWER | tr [:upper:] [:lower:]`
done
fi
#verify the specified file exists in <expdev> directory
echo `ls -Al $FILEDIR/$FILENAME`
echo -e "\nThe script will run against the listed above ${Purple}$FILENAME${ColorOFF} file.\n${Purple}NOTE${ColorOFF}: Do NOT proceed if it says ${Purple}No such file or directory${ColorOFF}.\nWould you like to proceed? y/n -->:\c "
read PROCEED
PROCEED=`echo $PROCEED | tr [:upper:] [:lower:]`
if [ "$PROCEED" == "y" ]
#user agrees
then
CSVFILE=`echo $FILEDIR/$FILENAME`
else
echo -e "Make sure you saved your CSV file in ${Purple}$FILEDIR${ColorOFF} directory and run the script again."
exit 1
fi
fi
#----------------------------------------------
#Remove header from CSVFILE
#and create a temp csvfile.txt
#for "WHILE READ" loop
#----------------------------------------------
Nrows=`cat ${CSVFILE}| wc -l`
N=`expr $Nrows - 1`
tail -$N ${CSVFILE} > csvfile.txt
#----------------------------------------------
#Determine the Extension Date
#----------------------------------------------
TODAYsDATE=`date +"%a"`
case $TODAYsDATE in
Mon ) ExtDate=`date --date='4 day' +"%A, %d-%b-%Y"`
;;
Tue ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
Wed ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
Thu ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
Fri ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
* ) echo "Cannot determine the Extention Date."
REPLY="n"
while [ "$REPLY" != "y" ]
do
echo -e "Please enter Extended Date: \c"
read ExtDate
echo -e "You entered ${Purple}$ExtDate${ColorOFF}. Is this correct? y/n: \c "
read ANSWER
REPLY=`echo $ANSWER | tr [:upper:] [:lower:]`
done
esac
echo -e "The Extended Date for the Application Owners will be ${Purple}$ExtDate${ColorOFF}.\nPlease make a note for End Date in the APEX Workstation Tracking Tool."
#create a temp confirmation file that will be sent to dev-tools
echo -e "\nThe Extended Date is $ExtDate.\nSent emails to: " > confirmation.txt
#----------------------------------------------
#Read the csvfile.txt
#----------------------------------------------
while IFS=',' read v1 v2 v3 v4 other
do
#----------------------------------------------
#Determine email address
#----------------------------------------------
v2=` echo $v2 | tr '"/,\&' ' ' `
NAME=( ${v2} )
cnt=${#NAME[#]}
#-----------------------------------------------
# Filter j or p<number> userids from table V2
#--------------------------------------------------
if [[ "$NAME" =~ [P|p][[:digit:]]{6}$|[J|j][[:digit:]]{5}$ ]]
then
ADresult1=`getent passwd "$NAME" | awk -F":" '{print $1}'`
echo ${ADresult1}
#------------------------------------------------------------------------------------------------------
# Processing Array
#-----------------------------------------------------------------------------------------------------
declare -A arrmail
arrmail[$ADresult1]="$v1, $v3"
appendvar+="; ${arrmail[$ADresult1]}"
if [ ! -z "${arrmail[$ADresult1]}" ]
then
echo ${arrmail[#]}
else
echo ${arrmail[#]}="$appendvar"
fi
#--------------------------------------------------------------------------------------
# If query matches j or p#, then email user
#------------------------------------------------------------------------------------------
#
#
# v1=` echo $v1 | tr '"' ' ' `
# SUBJECT=" ${v1}, ${v3} - Your Development Machine Status Update"
#
# TO="${ADresult1}#edwardjones.com"
#
# v4=` echo $v4 | tr '"' ' ' `
#
# devmailmessage="\nGreetings ${NAME[0]}, \n\nI am in the process of cleaning up the machines in our lab.\n\nAssigned device: ${v1} \n\nEffort: ${v3}\n\nThe above machine have been assigned to you since ${v4}.\n\nDo you still need this machine or can I reclaim it?\nIf this machine is still required, please let us know how much longer you will need it.\nPlease keep in mind that Dev. workstations or laptops can only be assigned for 90 days due to licensing restrictions.\n\nIf this machine is no longer needed, please let us know also.\n\nThis machine will be rebuilt on or after $ExtDate.\n\nPlease reply to dev-tools#edwardjones.com by $ExtDate.\n\n\nThank you,\n\nDevelopment Environment Support team\ndev-tools#edwardjones.com "
#
#echo -e ${devmailmessage}|mail -s "$SUBJECT" $TO -- -f $TO_TEAM
#--------------------------------------------------------------------------------------------
# If query does not match j or p number, then lookup username via first and last name
#----------------------------------------------------------------------------------------------
elif [ ${cnt} -eq 2 ]
then
ADresult=`getent passwd | grep -i "${NAME[0]}" | grep -i "${NAME[1]}"`
echo ${ADresult}
#count search results from Active Directory; only email if there is one unique result
ADsearch1=`echo ${ADresult} | grep -i "${NAME[0]}" | wc -l`
ADsearch2=`echo ${ADresult} | grep -i "${NAME[1]}" | wc -l`
if [ ${ADsearch1} -eq 1 ] && [ ${ADsearch2} -eq 1 ]
then
pNumber=`echo ${ADresult} | awk -F":" '{print $1}'`
#------------------------------------------------------------------------------------------------------
# Processing Array
#-----------------------------------------------------------------------------------------------------
#declare -A arrmail
#arrmail[$pNumber]="$v1, $v3"
#appendvar+="; ${arrmail[$pNumber]}"
#if [ ! -z "${arrmail[$pNumber]}" ]
# then
# {arrmail[$pNumber]}="${arrmail[$ADresult1]}"
#else
# {arrmail[$ADresult1]}="$appendvar"
#fi
#------------------------------------------------------------------------------------------------------------
# v1=` echo $v1 | tr '"' ' ' `
# SUBJECT=" ${v1}, ${v3} - Your Development Machine Status Update"
# TO="${pNumber}#edwardjones.com"
# v4=` echo $v4 | tr '"' ' ' `
# devmailmessage="\nGreetings ${NAME[0]}, \n\nI am in the process of cleaning up the machines in our lab.\n\nAssigned device: ${v1} \n\nEffort: ${v3}\n\nThe above machine have been assigned to you since ${v4}.\n\nDo you still need this machine or can I reclaim it?\nIf this machine is still required, please let us know how much longer you will need it.\nPlease keep in mind that Dev. workstations or laptops can only be assigned for 90 days due to licensing restrictions.\n\nIf this machine is no longer needed, please let us know also.\n\nThis machine will be rebuilt on or after $ExtDate.\n\nPlease reply to dev-tools#edwardjones.com by $ExtDate.\n\n\nThank you,\n\nDevelopment Environment Support team\ndev-tools#edwardjones.com "
# echo -e ${devmailmessage}| mail -s "$SUBJECT" $TO -- -f $TO_TEAM
# else
# TO=`echo "ACTION REQUIRED: Email address cannot be determined for the following user: ${v2}, with assigned device: ${v1}. Please email this user yourself."`
fi
# else
# TO=`echo "ACTION REQUIRED: Email address cannot be determined for the following user: ${v2}, with assigned device: ${v1}. Please email this user yourself."`
fi
#echo -e "\n$TO ${v2}, ${v1}" >> confirmation.txt
echo "NEXT ROW"
done < csvfile.txt
#----------------------------------------------
#sending out confirmation email to dev-tools
#----------------------------------------------
#subject="WW - Notification Emails to Developers - Status"
#cat confirmation.txt | mail -s "$subject" $TO_TEAM -- -f $TO_TEAM
#echo -e "Result: Processing is COMPLETED. Please check the ${Purple}$subject${ColorOFF} email. Note: ${Purple}$FILENAME${ColorOFF} file was deleted from ${Purple}$FILEDIR${ColorOFF} directory to avoid duplicates in the future."
rm ${CSVFILE}
rm csvfile.txt
rm confirmation.txt
else
#----------------------------------------------------
#if the directory is empty
#------------------------------------------------------
echo -e "Please make sure the CSV file was downloaded to ${Purple}$FILEDIR${ColorOFF} directory and run the script again."
fi
###
#--------THE END---------------------------------
###
Here's some of the output of the script.
IFS=,
read v1 v2 v3 v4 other
++ echo p098650
++ tr '"/,\&' ' '
v2=p098650
NAME=(${v2})
cnt=1
[[ p098650 =~ [P|p][[:digit:]]{6}$|[J|j][[:digit:]]{5}$ ]]
++ getent passwd p098650
++ awk -F: '{print $1}'
ADresult1=p098650
echo p098650
p098650
declare -A arrmail
arrmail[$ADresult1]='CNU327B1Y3, EDGARLITE_0340_COR_00'
appendvar+='; CNU327B1Y3, EDGARLITE_0340_COR_00'
'[' '!' -z 'CNU327B1Y3, EDGARLITE_0340_COR_00' ']'
echo CNU327B1Y3, EDGARLITE_0340_COR_00
CNU327B1Y3, EDGARLITE_0340_COR_00
echo 'NEXT ROW'
NEXT ROW
IFS=,
read v1 v2 v3 v4 other
++ echo p098650
++ tr '"/,\&' ' '
v2=p098650
NAME=(${v2})
cnt=1
[[ p098650 =~ [P|p][[:digit:]]{6}$|[J|j][[:digit:]]{5}$ ]]
++ getent passwd p098650
++ awk -F: '{print $1}'
ADresult1=p098650
echo p098650
p098650
declare -A arrmail
arrmail[$ADresult1]='CNU327B1GP, BUZZLITE_0340_COR_00'
appendvar+='; CNU327B1GP, BUZZLITE_0340_COR_00'
'[' '!' -z 'CNU327B1GP, BUZZLITE_0340_COR_00' ']'
echo CNU327B1GP, BUZZLITE_0340_COR_00
CNU327B1GP, BUZZLITE_0340_COR_00
echo 'NEXT ROW'
NEXT ROW
What's I'm expecting to see is, arrmail[$ADresult1]='CNU327B1Y3, EDGARLITE_0340_COR_00';'CNU327B1GP, BUZZLITE_0340_COR_00'
I couldn't include a screenshot of the CSV, so I'm just going to cut and paste some of it.
Sernum $V1 Username $V2 Effort $V3 Startdate $V4
CNU327B1Y3 p098650 EDGARLITE_0340_COR_00 4-Mar-14
CNU3199N31 Bell SDLCONTENTPORTER_2013_COR_00 10-Mar-14
CNU327B1GP p098650 BUZZLITE_0340_COR_00 4-Mar-14
CNU2479GLB Mike XENDESKAGENTX64_0640_COR_00 28-Feb-14
CNU327B1PB Mike Bell BONDONEUOBRA_2001_COR_00 6-Mar-14
2UA24705YY Bell Mike SASENTGUIDEX64_0610_COR_00 25-Nov-13
CNU2479K7Z Bell, Mike Software Testing 12-Dec-13
2UA24705ZZ Bell Mike TESTGUIDEX64_0610_COR_00 25-Nov-13
I know this is a lot. I've been working on this for a long time and I'm trying to gain a greater understanding of how to use and manipulate arrays. Also if you have an recommendations on books covering the subject, I'll be happy to review those also. Thank you in advance for all the help.

How to format a bash array as a JSON array

I have a bash array
X=("hello world" "goodnight moon")
That I want to turn into a json array
["hello world", "goodnight moon"]
Is there a good way for me to turn this into a json array of strings without looping over the keys in a subshell?
(for x in "${X[#]}"; do; echo $x | sed 's|.*|"&"|'; done) | jq -s '.'
This clearly doesn't work
echo "${X[#]}" | jq -s -R '.'
You can do this:
X=("hello world" "goodnight moon")
printf '%s\n' "${X[#]}" | jq -R . | jq -s .
output
[
"hello world",
"goodnight moon"
]
Since jq 1.6 you can do this:
jq --compact-output --null-input '$ARGS.positional' --args -- "${X[#]}"
giving:
["hello world","goodnight moon"]
This has the benefit that no escaping is required at all. It handles strings containing newlines, tabs, double quotes, backslashes and other control characters. (Well, it doesn't handle NUL characters but you can't have them in a bash array in the first place.)
This ...
X=("hello world" "goodnight moon" 'say "boo"' 'foo\bar')
json_array() {
echo -n '['
while [ $# -gt 0 ]; do
x=${1//\\/\\\\}
echo -n \"${x//\"/\\\"}\"
[ $# -gt 1 ] && echo -n ', '
shift
done
echo ']'
}
json_array "${X[#]}"
... yields:
["hello world", "goodnight moon", "say \"boo\"", "foo\\bar"]
If you are planning to do a lot of this (as your reluctance to use a subshell suggests) then something such as this that does not rely on any subprocess is likely to your advantage.
You can use:
X=("hello world" "goodnight moon")
sed 's/^/[/; s/,$/]/' <(printf '"%s",' "${X[#]}") | jq -s '.'
[
[
"hello world",
"goodnight moon"
]
]
If the values do not contain ASCII control characters, which have to be escaped in strings in valid JSON, you can also use sed:
$ X=("hello world" "goodnight moon")
$ printf %s\\n "${X[#]}"|sed 's/["\]/\\&/g;s/.*/"&"/;1s/^/[/;$s/$/]/;$!s/$/,/'
["hello world",
"goodnight moon"]
If the values contain ASCII control characters, you can do something like this:
X=($'a\ta' $'a\n\\\"')
for((i=0;i<${#X[#]};i++));do
[ $i = 0 ]&&printf \[
printf \"
e=${X[i]}
e=${e//\\/\\\\}
e=${e//\"/\\\"}
for((j=0;j<${#e};j++));do
c=${e:j:1}
if [[ $c = [[:cntrl:]] ]];then
printf '\\u%04x' "'$c"
else
printf %s "$c"
fi
done
printf \"
if((i<=${#X[#]}-2));then
printf ,
else
printf \]
fi
done
As improve of answer
https://stackoverflow.com/a/26809278/16566807
Script produce few formats which could be usefull when include. Script meets BASH spec, checked with shellcheck.
#!/bin/bash
#
#
X=("hello world" "goodnight moon" 'say "boo"' 'foo\bar')
#
# set parameter to define purpose: return_format
# php5 -> for 5.x
# -> https://stackoverflow.com/questions/7073672/how-to-load-return-array-from-a-php-file/7073686
# php -> for 7.x and greater
# json -> for $array=#file_get_contents($f); json_decode($array, true);
# /none/ -> for JS to JSON.Parse(myJSON);
# function call with array as parameter: return_array "${array[#]}"
return_array() {
rf="${return_format}"
if [[ $rf = "php5" ]]; then
q=("<?php return array(" ");")
elif [[ $rf = "php" ]];then
q=("<?php return [" "];")
elif [[ $rf = "json" ]];then
q=("{" "}")
else
q=("[" "]")
fi
echo -n "${q[0]}"
while [[ $# -gt 0 ]]; do
x=${1//\\/\\\\}
echo -n "\"${x//\"/\\\"}\""
[[ $# -gt 1 ]] && echo -n ', '
shift
done
echo "${q[1]}"
}
echo "PHP 5.x"
return_format="php5"
return_array "${X[#]}"
echo "PHP 7.x"
return_format="php"
return_array "${X[#]}"
echo "JSON for PHP"
return_format="json"
return_array "${X[#]}"
echo "JSON for JS"
return_format=
return_array "${X[#]}"
will produce output:
PHP 5.x
<?php return array("hello world", "goodnight moon", "say \"boo\"", "foo\\bar");
PHP 7.x
<?php return ["hello world", "goodnight moon", "say \"boo\"", "foo\\bar"];
JSON for PHP
{"hello world", "goodnight moon", "say \"boo\"", "foo\\bar"}
JSON for JS
["hello world", "goodnight moon", "say \"boo\"", "foo\\bar"]

Resources