shell script array won't populate from for loop - arrays

Can anyone tell me why this array creation: cccr[$string_1]=$string_2 #doesn't work?
#!/bin/bash
firstline='[Event "Marchand Open"][Site "Rochester NY"][Date "2005.03.19"][Round "1"][White "Smith, Igor"][Black "Jones, Matt"][Result "1-0"][ECO "C01"][WhiteElo "2409"][BlackElo "1911"]'
unset cccr
declare -A cccr
(IFS='['; for word in $firstline; do
string_1=$(echo $word | cut -f1 -d'"' | tr -d ' ')
string_2=$( echo $word | cut -f2 -d'"' )
if [ ! -z $string_1 ]; then # If $string_1 is not empty
cccr[$string_1]=$string_2 # why doesn't this line work?
fi
done)
echo ${cccr[Event]} # echos null string

It happens because the value of string_1 is empty at the first iteration.
Example :
#!/bin/bash
firstline='[Event "Marchand Open"][Site "Rochester NY"][Date "2005.03.19"][Round "1"][White "Smith, Igor"][Black "Jones, Matt"][Result "1-0"][ECO "C01"][WhiteElo "2409"][BlackElo "1911"]'
unset cccr
declare -A cccr
(IFS='['; for word in $firstline; do
string_1=$( echo $word | cut -f1 -d'"' )
string_2=$( echo $word | cut -f2 -d'"' )
echo "$string_1 - $string_2"
#cccr[$string_1]=$string_2
done)
Output :
- # Problem !
Event - Marchand Open
Site - Rochester NY
...
You have to modify your script to prevent the value of being empty.
A very simple workaround is to check the value of string_1 before using it.
Example :
# ...
string_1=$( echo $word | cut -f1 -d'"' )
string_2=$( echo $word | cut -f2 -d'"' )
if [ ! -z $string_1 ]; then # If $string_1 is not empty
echo "$string_1 - $string_2"
cccr[$string_1]=$string_2
fi
# ...
From the man page of [
-z STRING
the length of STRING is zero
Output :
Event - Marchand Open
Site - Rochester NY
# ... No problem
EDIT
BTW, if look at the value of string_1, you will see that the value is Event' ' and not Event (there's a whitespace at the end of Event)
So cccr[Event] does not exist, but cccr[Event ] exists.
To fix that, you can delete the whitespaces in string_1 :
string_1=$(echo $word | cut -f1 -d'"' | tr -d ' ') # tr -d ' ' deletes all the whitespaces
EDIT 2
I forgot to tell you that it's normal if it does not work. Indeed, the loop is executed in a subshell environment. So the array is filled in the subshell, but not in the current shell.
From the man page of bash :
(list) list is executed in a subshell environment (see COMMAND EXECUTION ENVIRONMENT below). Variable
assignments and builtin commands that affect the shell's environment do not remain in effect
after the command completes. The return status is the exit status of list.
So there are 2 solutions :
1. Don't run the loop in a subshell (remove the parentheses).
# ...
OLDIFS=$IFS
IFS='['
for word in $firstline; do
string_1=$(echo $word | cut -f1 -d'"' | tr -d ' ')
string_2=$(echo $word | cut -f2 -d'"')
if [ ! -z $string_1 ]; then
cccr[$string_1]=$string_2
fi
done
IFS=$OLDIFS
echo "Event = ${cccr[Event]}"
echo "Site = ${cccr[Site]}"
Output :
Event = Marchand Open
Site = Rochester NY
2. Use your array in the subshell.
# ...
(IFS='['
for word in $firstline; do
string_1=$(echo $word | cut -f1 -d'"' | tr -d ' ')
string_2=$(echo $word | cut -f2 -d'"')
if [ ! -z $string_1 ]; then # If $string_1 is not empty
cccr[$string_1]=$string_2
fi
done
echo "Event = ${cccr[Event]}"
echo "Site = ${cccr[Site]}"
)
Output :
Event = Marchand Open
Site = Rochester NY

Related

Weird array output in shell script

The content extracted from the file is separated and stored in an array, and print the content using loop. Only printing the last element is weird. I'll show you my code.
How can I resolve this problem?
[config.json]
{
"id": "hello",
"passwd": "1234",
"languageList": ["ko", "en"]
}
[test.sh]
# BEFORE_CONFIG and AFTER_CONFIG have same code
BEFORE_CONFIG=~/workspace/env/config.json
AFTER_CONFIG=~/workspace/config/config.json
BEF_LANG=$(grep "\[" ${BEFORE_CONFIG} | tr -d '\[' | tr -d '\]' | tr -d '"' | tr -d ' ' | cut -d ":" -f2)
AF_LANG=$(grep "\[" ${AFTER_CONFIG} | tr -d '\[' | tr -d '\]' | tr -d '"' | tr -d ' ' | cut -d ":" -f2)
echo "before lang :: ${BEF_LANG}"
echo "after lang :: ${AF_LANG}"
IFS=',' read -r -a AF_LANG_LIST <<< "$AF_LANG"
echo "after lang list print >> ${AF_LANG_LIST[#]}"
echo "list length >> ${#AF_LANG_LIST[#]}"
for element in ${AF_LANG_LIST[#]}
do
echo "${element}"
echo "This language !!! ${element} !!! print !!!!"
done
[result]
$ source tesh.sh
before lang :: ko,en
after lang :: ko,en
after lang list >> ko en
list length >> 2
ko
This language !!! ko !!! print !!!!
en
!!! print !!!!!! en # expect result → This language !!! en !!! print !!!!
You can use jq to parse the json correctly, and extract languageList do:
cat ~/workspace/env/config.json|jq .languageList[] -r|xargs
which will output:
ko en
which you can later use in your script
Trying to parse JSON with tr & cut is prone to so many errors.
Here is the example for your script:
#!/bin/bash
# BEFORE_CONFIG and AFTER_CONFIG have same code
BEFORE_CONFIG=~/workspace/env/config.json
AFTER_CONFIG=~/workspace/config/config.json
BEF_LANG=$(grep "\[" ${BEFORE_CONFIG} | tr -d '\[' | tr -d '\]' | tr -d '"' | tr -d ' ' | cut -d ":" -f2)
#AF_LANG=$(grep "\[" ${AFTER_CONFIG} | tr -d '\[' | tr -d '\]' | tr -d '"' | tr -d ' ' | cut -d ":" -f2)
AF_LANG=$(cat ${BEFORE_CONFIG}|jq .languageList[] -r|xargs)
echo "before lang :: ${BEF_LANG}"
echo "after lang :: ${AF_LANG}"
# you do not need this
#IFS=',' read -r -a AF_LANG_LIST <<< "$AF_LANG"
#echo "after lang list print >> ${AF_LANG_LIST[#]}"
#echo "list length >> ${#AF_LANG_LIST[#]}"
for element in ${AF_LANG[#]}
do
echo "${element}"
echo "This language !!! ${element} !!! print !!!!"
done

diff two arrays each containing files paths into a third array (for removal)

In the function below you will see notes on several attempts to solve this problem; each attempt has a note indicating what went wrong. Between my attempts there is a line from another question here which purports to solve some element of the matter. Again, I've added a note indicating what that is supposed to solve. My brain is mush at this point. What is the stupid simple thing I've overlooking?
function func_removeDestinationOrphans() {
readarray -d '' A_Destination_orphans < <( find "${directory_PMPRoot_destination}" -type f -print0 )
for (( i = 0 ; i < ${#A_Destination_orphans[#]} ; i++ )) ; do
printf '%s\n' "→ ${A_Destination_orphans[${i}]}" # path to each track
done
printf '%b\n' ""
# https://stackoverflow.com/questions/2312762/compare-difference-of-two-arrays-in-bash
# echo ${Array1[#]} ${Array2[#]} | tr ' ' '\n' | sort | uniq -u ## original
# Array3=(`echo ${Array1[#]} ${Array2[#]} | tr ' ' '\n' | sort | uniq -u `) ## store in array
# A_Destination_orphans_diff=(`echo "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" | tr ' ' '\n' | sort | uniq -u `) # drops file path after space
# printf "%s\0" "${Array1[#]}" "${Array2[#]}" | sort -z | uniq -zu ## newlines and white spaces
# A_Destination_orphans_diff=($( printf "%s\0" "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" | sort -z | uniq -zu )) # throws warning and breaks at space but not newline
# printf '%s\n' "${Array1[#]}" "${Array2[#]}" | sort | uniq -u ## manage spaces
# A_Destination_orphans_diff=($( printf '%s\n' "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" | sort | uniq -u )) # breaks at space and newline
# A_Destination_orphans_diff="($( printf '%s\n' "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" | sort | uniq -u ))" # creates string surrounded by ()
# A_Destination_orphans_diff=("$( printf '%s\n' "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" | sort | uniq -u )") # creates string
# A_Destination_orphans_diff=($( printf '%s\n' ${A_Destination_dubUnders[#]} ${A_Destination_orphans[#]} | sort | uniq -u )) # drops file path after space
for (( i = 0 ; i < ${#A_Destination_orphans_diff[#]} ; i++ )) ; do
printf '%s\n' "→ ${A_Destination_orphans_diff[${i}]}" # path to each track
done
printf '%b\n' ""
for (( i = 0 ; i < ${#A_Destination_orphans_diff[#]} ; i++ )) ; do
echo # rm "${A_Destination_orphans_diff[i]}"
done
func_EnterToContinue
}
This throws warning and breaks at space but not newline because you build the array with direct assignment of syntax construct. When an entry contains spaces, it also splits break to a new entry.
A_Destination_orphans_diff=($( printf "%s\0" "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" | sort -z | uniq -zu ))
To avoid the issue of the method above, you can mapfile/readarray a null delimited entries stream.
mapfile -t -d '' A_Destination_orphans_diff < <(
printf "%s\0" "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" |
sort -z |
uniq -zu
)
In case your shell version is too old to support mapfile you can perform the same task with IFS=$'\37' read -r -d '' -a array.
$'\37' is shell's C-Style string syntax with octal code 37, which is ASCII 31 US for Unit Separator:
IFS=$'\37' read -r -d '' -a A_Destination_orphans_diff < <(
printf "%s\0" "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" |
sort -z |
uniq -zu |
xargs -0 printf '%s\37'
)
To remove all files not present in A_Destination_dubUnders array you could:
func_removeDestinationOrphans() {
find "${directory_PMPRoot_destination}" -type f -print0 |
sort -z |
join -z -v1 -t '' - <(printf "%s\0" "${A_Destination_dubUnders[#]}" | sort -z) |
xargs -0 echo rm
}
Use join or comm to find elements not present in one list and present in another list. I am usually wrong about -v1, so try with -v2 if it echoes the elements from wrong list (I do not understand if you want to remove files present in A_Destination_dubUnders list or not present, you did not specify that).
Note that function name() is a mix of ksh and posix function definition. Just name() {. See bash hackers wiki obsolete
Here is the working version with modifications thanks to suggested input from the first two respondents (thanks!).
function func_removeDestinationOrphans() {
printf '%s\n' " → Purge playlist orphans: " ""
printf '%b\n' "First we will remove any files not present in your proposed playlist. "
func_EnterToContinue
bash_version="$( bash --version | head -n1 | cut -d " " -f4 | cut -d "(" -f1 )"
if printf '%s\n' "4.4.0" "${bash_version}" | sort -V -C ; then
readarray -d '' A_Destination_orphans < <( find "${directory_PMPRoot_destination}" -type f -print0 ) # readarray or mapfile -d fails before bash 4.4.0
readarray -t -d '' A_Destination_orphans_diff < <(
printf "%s\0" "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" |
sort -z |
uniq -zu
)
else
while IFS= read -r -d $'\0'; do
A_Destination_orphans+=( "$REPLY" )
done < <( find "${directory_PMPRoot_destination}" -type f -print0 )
IFS=$'\37' read -r -d '' -a A_Destination_orphans_diff < <(
printf "%s\0" "${A_Destination_dubUnders[#]}" "${A_Destination_dubUnders[#]}" "${A_Destination_orphans[#]}" |
sort -z |
uniq -zu |
xargs -0 printf '%s\37'
)
fi
if [[ ! "${A_Destination_orphans_diff[*]}" = '' ]] ; then
for (( i = 0 ; i < ${#A_Destination_orphans_diff[#]} ; i++ )) ; do
rm "${A_Destination_orphans_diff[i]}"
done
fi
}
If you would like to see the entire Personal Music Player sync script, you can find that via my GitHub.

Read delimited multiline string file into multiple arrays in Bash

I began with a file like so:
Table_name1 - Table_desc1
Table_name2 - Table_desc2
...
...
I have a script that parses this file and splits them into two arrays:
declare -a TABLE_IDS=()
declare -a TABLE_DESCS=()
while IFS= read -r line || [[ -n "${line}" ]]; do
TABLE_IDS[i]=${line%' '-' '*}
TABLE_DESCS[i++]=${line#*' '-' '}
done < "${TABLE_LIST}"
for i in "${!TABLE_IDS[#]}"; do
echo "Creating Table ID: "${TABLE_IDS[i]}", with Table Description: "${TABLE_DESCS[i]}""
done
This works really well, with no problems whatsoever.
I wanted to extend this and make the file:
Table_name1 - Table_desc1 - Table_schema1
Table_name2 - Table_desc2 - Table_schema2
...
...
For this, I tried:
declare -a TABLE_IDS=()
declare -a TABLE_DESCS=()
while IFS= read -r line || [[ -n "${line}" ]]; do
TABLE_IDS[i]="$(echo $line | cut -f1 -d - | tr -d ' ')"
TABLE_DESCS[i++]="$(echo $line | cut -f2 -d - | tr -d ' ')"
TABLE_SCHEMAS[i++]="$(echo $line | cut -f3 -d - | tr -d ' ')"
done < "${TABLE_LIST}"
for i in "${!TABLE_IDS[#]}"; do
echo "Creating Table ID: "${TABLE_IDS[i]}", with Table Description: "${TABLE_DESCS[i]}" and schema: "${TABLE_SCHEMAS[i]}""
done
And while this will faithfully list all the Table IDs and the Table descriptions, the schemas are omitted. I tried:
while IFS= read -r line || [[ -n "${line}" ]]; do
TABLE_IDS[i]="$(echo $line | cut -f1 -d - | tr -d ' ')"
TABLE_DESCS[i]="$(echo $line | cut -f2 -d - | tr -d ' ')"
TABLE_SCHEMAS[i]="$(echo $line | cut -f3 -d - | tr -d ' ')"
done < "${TABLE_LIST}"
And it returns just the last line's Table name, description AND schema. I suspect this is an indexing/looping problem, but am unable to figure out what exactly is going wrong. Please help! Thanks!
perhaps set the delimiter to the actual delimiter - and do the processing in the read loop instead of deferring and using arrays.
$ while IFS=- read -r t d s;
do
echo "Creating Table ID: ${t// }, with Table Description: ${d// } and schema: ${s// }";
done < file

Splitting files into multiple files based on some pattern and take some information

I'm working with a lot of files with this structure:
BEGIN
TITLE=id=PRD000012;PRIDE_Exp_Complete_Ac_1645.xml;spectrum=1393
PEPMASS=946.3980102539062
CHARGE=3.0+
USER03=
SEQ=DDDIAAL
TAXONOMY=9606
272.228 126847.000
273.252 33795.000
END
BEGIN IONS
TITLE=id=PRD000012;PRIDE_Exp_Complete_Ac_1645.xml;spectrum=1383
PEPMASS=911.3920288085938
CHARGE=2.0+
USER03=
SEQ=QGKFEAAETLEEAAMR
TAXONOMY=9606
1394.637 71404.000
1411.668 122728.000
END
BEGIN IONS
TITLE=id=PRD000012;PRIDE_Exp_Complete_Ac_1645.xml;spectrum=2965
PEPMASS=946.3900146484375
CHARGE=3.0+
TAXONOMY=9606
1564.717 92354.000
1677.738 33865.000
END
This structure is repeated thousands of times but with different data inside. As you can see, between some begin-end, sometimes SEQ and USER03 are not there. This is because the protein is not identified ... And here comes my problem.
I would like to know how many proteins are identified and how many are unidentified. To do this I was trying this:
for i in $(ls *.txt ); do
echo $i
awk '/^BEGIN/{n++;w=1} n&&w{print > "./cache/out" n ".txt"} /^END/{w=0}' $i
done
I found this here (Split a file into multiple files based on a pattern and name the new files by the search pattern in Unix?)
And then use the outputs and classify them:
for i in $(ls cache/*.txt ); do
echo $i
if grep -q 'SEQ' $i; then
mv $i ./archive_identified
else
mv $i ./archive_unidentified
fi
done
After this, I'd like to take some data (Example: spectrum, USER03, SEQ, TAXONOMY) from classified files.
for I in $( ls archive_identified/*.txt ); do
echo $i
grep 'SEQ' $i | cut -d "=" -f2- | tr ',' '\n' >> ./sequences_ide.txt
grep 'TAXONOMY' $i | cut -d "=" -f2- | tr ',' '\n' >> ./taxonomy_ide.txt
grep 'USER' $i | cut -d "=" -f2- >> ./modifications_ide.txt
grep 'TITLE' $i | sed 's/^.*\(spectrum.*\)/\1/g' | cut -d "=" -f2- >> ./spectrum.txt
done
for i in $( ls archive_unidentified/*.txt ); do
echo $i
grep 'SEQ' $i | cut -d "=" -f2- | tr ',' '\n' >> ./sequences_unide.txt
grep 'TAXONOMY' $i | cut -d "=" -f2- | tr ',' '\n' >> ./taxonomy_unide.txt
grep 'USER' $i | cut -d "=" -f2- >> ./modifications_unide.txt
grep 'TITLE' $i | sed 's/^.*\(spectrum.*\)/\1/g' | cut -d "=" -f2- >> ./spectrum_unide.txt
done
The problem is that the first part of the script takes too much time due to the large size of the data (12-15gb.). Is there any way to do this easier?
Thank you in advance.
You can do all in one awk script. awk can iterate through all rows (records) so you don't need an external loop. For example, for the data file you provided
$ awk -v RS= '/\nSEQ/ {seq++; print > "file_path_with_seq" NR ".txt"; next}
{noseq++; print > "file_path_without_seq" NR ".txt"}
END { print "with seq:", seq;
print "without seq:", noseq}' file
will print
with seq: 2
without seq: 1
and produces the files
$ head file_path_with*
==> file_path_with_seq1.txt <==
BEGIN
TITLE=id=PRD000012;PRIDE_Exp_Complete_Ac_1645.xml;spectrum=1393
PEPMASS=946.3980102539062
CHARGE=3.0+
USER03=
SEQ=DDDIAAL
TAXONOMY=9606
272.228 126847.000
273.252 33795.000
END
==> file_path_with_seq2.txt <==
BEGIN IONS
TITLE=id=PRD000012;PRIDE_Exp_Complete_Ac_1645.xml;spectrum=1383
PEPMASS=911.3920288085938
CHARGE=2.0+
USER03=
SEQ=QGKFEAAETLEEAAMR
TAXONOMY=9606
1394.637 71404.000
1411.668 122728.000
END
==> file_path_without_seq3.txt <==
BEGIN IONS
TITLE=id=PRD000012;PRIDE_Exp_Complete_Ac_1645.xml;spectrum=2965
PEPMASS=946.3900146484375
CHARGE=3.0+
TAXONOMY=9606
1564.717 92354.000
1677.738 33865.000
END

Appending to elements in a Bash associative array

My name is Mike and I am a total noob at Bash programming. With that being said, please don't chew me to a pulp. Here's my scenario. I'm importing a csv and processing it line by line. It looks up information in AD and pipes that result to a variable $ADresult1.
I then take that output and send it into the array.
declare -A arrmail
arrmail[$ADresult1]="$v1, $v3"
appendvar+="; ${arrmail[$ADresult1]}"
if [ ! -z "${arrmail[$ADresult1]}" ]; then
echo ${arrmail[#]}
else
echo ${arrmail[#]}="$appendvar"
fi
Now in my CSV file, the var $ADresult1 has more than one entry that is the same, however each element should be different.
For example: The first line might contain arrmail[p100]=Y2K, Y2J
The second line might arrmail[p100]=GGG, GG1
My question would be how to combine the elements from the first line and second line into one line of output and/or if this is possible.
For instance arrmail[p100]=Y2K, Y2J; GGG, GG1
If needs be, I can post the whole entire script for a bigger picture.
Here's the script.
#!/bin/bash
###--------------------------------------------------------------------------
# Author : Volha Zhdanava (p146819)
# Date : 11-FEB-2013
# Purpose: To automate the process of sending out notification emails to developers when
# Assigned DEV Machine is on the Expired Devices List.
#----------------------------------------------------------------------------
# ----MODIFIED----
#----------------------------------------------------------------------------------------------------
# Author Revsn Date Revsn ID Revsn Purpose
# Mike Bell 02-28-14 Rev.03 Add ability to email j or p number
# Mike Bell 01-02-15 Rev.04 Add verbiage about 90 day licensing
###----------------------------------------------------------------------------------------------------
Purple='\e[1;35m'
ColorOFF='\e[0m'
TO_TEAM="dev-tools#edwardjones.com"
#-----------------------------------------
#Ask for input CSV file
#-----------------------------------------
FILEDIR="/export/share/is/dev_env/expdev"
FILENAME="expired_devices.csv"
#if the directory is not empty, find the csv file
if [[ $(ls -A $FILEDIR) ]]
then
#check if the csv file was stored with default name <expired_devices.csv>
if [ -f $FILEDIR/$FILENAME ]
then
#absolute path to the downloaded file
CSVFILE=`echo $FILEDIR/$FILENAME`
#if the csv file was stored under a user-defined name,
#re-define the absolute path to that file
else
#count how many files are saved in the expdev directory
NAME=( `ls -A $FILEDIR` )
CNT=${#NAME[#]}
echo -e "There is/are ${Purple}${CNT}${ColorOFF} file(s) in the ${Purple}$FILEDIR${ColorOFF} directory:"
echo ${NAME[#]}
#if there is only one file saved within the expdev directory
if [ ${CNT} -eq 1 ]
then
FILENAME=${NAME[0]}
else
#it will prompt to enter the name of the file
REPLY="n"
while [ "$REPLY" != "y" ]
do
echo -e "Please enter the full name of the downloaded csv file: \c"
read FILENAME
echo -e "You entered ${Purple}$FILENAME${ColorOFF}. Is this correct? y/n: \c\n"
read ANSWER
REPLY=`echo $ANSWER | tr [:upper:] [:lower:]`
done
fi
#verify the specified file exists in <expdev> directory
echo `ls -Al $FILEDIR/$FILENAME`
echo -e "\nThe script will run against the listed above ${Purple}$FILENAME${ColorOFF} file.\n${Purple}NOTE${ColorOFF}: Do NOT proceed if it says ${Purple}No such file or directory${ColorOFF}.\nWould you like to proceed? y/n -->:\c "
read PROCEED
PROCEED=`echo $PROCEED | tr [:upper:] [:lower:]`
if [ "$PROCEED" == "y" ]
#user agrees
then
CSVFILE=`echo $FILEDIR/$FILENAME`
else
echo -e "Make sure you saved your CSV file in ${Purple}$FILEDIR${ColorOFF} directory and run the script again."
exit 1
fi
fi
#----------------------------------------------
#Remove header from CSVFILE
#and create a temp csvfile.txt
#for "WHILE READ" loop
#----------------------------------------------
Nrows=`cat ${CSVFILE}| wc -l`
N=`expr $Nrows - 1`
tail -$N ${CSVFILE} > csvfile.txt
#----------------------------------------------
#Determine the Extension Date
#----------------------------------------------
TODAYsDATE=`date +"%a"`
case $TODAYsDATE in
Mon ) ExtDate=`date --date='4 day' +"%A, %d-%b-%Y"`
;;
Tue ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
Wed ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
Thu ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
Fri ) ExtDate=`date --date='6 day' +"%A, %d-%b-%Y"`
;;
* ) echo "Cannot determine the Extention Date."
REPLY="n"
while [ "$REPLY" != "y" ]
do
echo -e "Please enter Extended Date: \c"
read ExtDate
echo -e "You entered ${Purple}$ExtDate${ColorOFF}. Is this correct? y/n: \c "
read ANSWER
REPLY=`echo $ANSWER | tr [:upper:] [:lower:]`
done
esac
echo -e "The Extended Date for the Application Owners will be ${Purple}$ExtDate${ColorOFF}.\nPlease make a note for End Date in the APEX Workstation Tracking Tool."
#create a temp confirmation file that will be sent to dev-tools
echo -e "\nThe Extended Date is $ExtDate.\nSent emails to: " > confirmation.txt
#----------------------------------------------
#Read the csvfile.txt
#----------------------------------------------
while IFS=',' read v1 v2 v3 v4 other
do
#----------------------------------------------
#Determine email address
#----------------------------------------------
v2=` echo $v2 | tr '"/,\&' ' ' `
NAME=( ${v2} )
cnt=${#NAME[#]}
#-----------------------------------------------
# Filter j or p<number> userids from table V2
#--------------------------------------------------
if [[ "$NAME" =~ [P|p][[:digit:]]{6}$|[J|j][[:digit:]]{5}$ ]]
then
ADresult1=`getent passwd "$NAME" | awk -F":" '{print $1}'`
echo ${ADresult1}
#------------------------------------------------------------------------------------------------------
# Processing Array
#-----------------------------------------------------------------------------------------------------
declare -A arrmail
arrmail[$ADresult1]="$v1, $v3"
appendvar+="; ${arrmail[$ADresult1]}"
if [ ! -z "${arrmail[$ADresult1]}" ]
then
echo ${arrmail[#]}
else
echo ${arrmail[#]}="$appendvar"
fi
#--------------------------------------------------------------------------------------
# If query matches j or p#, then email user
#------------------------------------------------------------------------------------------
#
#
# v1=` echo $v1 | tr '"' ' ' `
# SUBJECT=" ${v1}, ${v3} - Your Development Machine Status Update"
#
# TO="${ADresult1}#edwardjones.com"
#
# v4=` echo $v4 | tr '"' ' ' `
#
# devmailmessage="\nGreetings ${NAME[0]}, \n\nI am in the process of cleaning up the machines in our lab.\n\nAssigned device: ${v1} \n\nEffort: ${v3}\n\nThe above machine have been assigned to you since ${v4}.\n\nDo you still need this machine or can I reclaim it?\nIf this machine is still required, please let us know how much longer you will need it.\nPlease keep in mind that Dev. workstations or laptops can only be assigned for 90 days due to licensing restrictions.\n\nIf this machine is no longer needed, please let us know also.\n\nThis machine will be rebuilt on or after $ExtDate.\n\nPlease reply to dev-tools#edwardjones.com by $ExtDate.\n\n\nThank you,\n\nDevelopment Environment Support team\ndev-tools#edwardjones.com "
#
#echo -e ${devmailmessage}|mail -s "$SUBJECT" $TO -- -f $TO_TEAM
#--------------------------------------------------------------------------------------------
# If query does not match j or p number, then lookup username via first and last name
#----------------------------------------------------------------------------------------------
elif [ ${cnt} -eq 2 ]
then
ADresult=`getent passwd | grep -i "${NAME[0]}" | grep -i "${NAME[1]}"`
echo ${ADresult}
#count search results from Active Directory; only email if there is one unique result
ADsearch1=`echo ${ADresult} | grep -i "${NAME[0]}" | wc -l`
ADsearch2=`echo ${ADresult} | grep -i "${NAME[1]}" | wc -l`
if [ ${ADsearch1} -eq 1 ] && [ ${ADsearch2} -eq 1 ]
then
pNumber=`echo ${ADresult} | awk -F":" '{print $1}'`
#------------------------------------------------------------------------------------------------------
# Processing Array
#-----------------------------------------------------------------------------------------------------
#declare -A arrmail
#arrmail[$pNumber]="$v1, $v3"
#appendvar+="; ${arrmail[$pNumber]}"
#if [ ! -z "${arrmail[$pNumber]}" ]
# then
# {arrmail[$pNumber]}="${arrmail[$ADresult1]}"
#else
# {arrmail[$ADresult1]}="$appendvar"
#fi
#------------------------------------------------------------------------------------------------------------
# v1=` echo $v1 | tr '"' ' ' `
# SUBJECT=" ${v1}, ${v3} - Your Development Machine Status Update"
# TO="${pNumber}#edwardjones.com"
# v4=` echo $v4 | tr '"' ' ' `
# devmailmessage="\nGreetings ${NAME[0]}, \n\nI am in the process of cleaning up the machines in our lab.\n\nAssigned device: ${v1} \n\nEffort: ${v3}\n\nThe above machine have been assigned to you since ${v4}.\n\nDo you still need this machine or can I reclaim it?\nIf this machine is still required, please let us know how much longer you will need it.\nPlease keep in mind that Dev. workstations or laptops can only be assigned for 90 days due to licensing restrictions.\n\nIf this machine is no longer needed, please let us know also.\n\nThis machine will be rebuilt on or after $ExtDate.\n\nPlease reply to dev-tools#edwardjones.com by $ExtDate.\n\n\nThank you,\n\nDevelopment Environment Support team\ndev-tools#edwardjones.com "
# echo -e ${devmailmessage}| mail -s "$SUBJECT" $TO -- -f $TO_TEAM
# else
# TO=`echo "ACTION REQUIRED: Email address cannot be determined for the following user: ${v2}, with assigned device: ${v1}. Please email this user yourself."`
fi
# else
# TO=`echo "ACTION REQUIRED: Email address cannot be determined for the following user: ${v2}, with assigned device: ${v1}. Please email this user yourself."`
fi
#echo -e "\n$TO ${v2}, ${v1}" >> confirmation.txt
echo "NEXT ROW"
done < csvfile.txt
#----------------------------------------------
#sending out confirmation email to dev-tools
#----------------------------------------------
#subject="WW - Notification Emails to Developers - Status"
#cat confirmation.txt | mail -s "$subject" $TO_TEAM -- -f $TO_TEAM
#echo -e "Result: Processing is COMPLETED. Please check the ${Purple}$subject${ColorOFF} email. Note: ${Purple}$FILENAME${ColorOFF} file was deleted from ${Purple}$FILEDIR${ColorOFF} directory to avoid duplicates in the future."
rm ${CSVFILE}
rm csvfile.txt
rm confirmation.txt
else
#----------------------------------------------------
#if the directory is empty
#------------------------------------------------------
echo -e "Please make sure the CSV file was downloaded to ${Purple}$FILEDIR${ColorOFF} directory and run the script again."
fi
###
#--------THE END---------------------------------
###
Here's some of the output of the script.
IFS=,
read v1 v2 v3 v4 other
++ echo p098650
++ tr '"/,\&' ' '
v2=p098650
NAME=(${v2})
cnt=1
[[ p098650 =~ [P|p][[:digit:]]{6}$|[J|j][[:digit:]]{5}$ ]]
++ getent passwd p098650
++ awk -F: '{print $1}'
ADresult1=p098650
echo p098650
p098650
declare -A arrmail
arrmail[$ADresult1]='CNU327B1Y3, EDGARLITE_0340_COR_00'
appendvar+='; CNU327B1Y3, EDGARLITE_0340_COR_00'
'[' '!' -z 'CNU327B1Y3, EDGARLITE_0340_COR_00' ']'
echo CNU327B1Y3, EDGARLITE_0340_COR_00
CNU327B1Y3, EDGARLITE_0340_COR_00
echo 'NEXT ROW'
NEXT ROW
IFS=,
read v1 v2 v3 v4 other
++ echo p098650
++ tr '"/,\&' ' '
v2=p098650
NAME=(${v2})
cnt=1
[[ p098650 =~ [P|p][[:digit:]]{6}$|[J|j][[:digit:]]{5}$ ]]
++ getent passwd p098650
++ awk -F: '{print $1}'
ADresult1=p098650
echo p098650
p098650
declare -A arrmail
arrmail[$ADresult1]='CNU327B1GP, BUZZLITE_0340_COR_00'
appendvar+='; CNU327B1GP, BUZZLITE_0340_COR_00'
'[' '!' -z 'CNU327B1GP, BUZZLITE_0340_COR_00' ']'
echo CNU327B1GP, BUZZLITE_0340_COR_00
CNU327B1GP, BUZZLITE_0340_COR_00
echo 'NEXT ROW'
NEXT ROW
What's I'm expecting to see is, arrmail[$ADresult1]='CNU327B1Y3, EDGARLITE_0340_COR_00';'CNU327B1GP, BUZZLITE_0340_COR_00'
I couldn't include a screenshot of the CSV, so I'm just going to cut and paste some of it.
Sernum $V1 Username $V2 Effort $V3 Startdate $V4
CNU327B1Y3 p098650 EDGARLITE_0340_COR_00 4-Mar-14
CNU3199N31 Bell SDLCONTENTPORTER_2013_COR_00 10-Mar-14
CNU327B1GP p098650 BUZZLITE_0340_COR_00 4-Mar-14
CNU2479GLB Mike XENDESKAGENTX64_0640_COR_00 28-Feb-14
CNU327B1PB Mike Bell BONDONEUOBRA_2001_COR_00 6-Mar-14
2UA24705YY Bell Mike SASENTGUIDEX64_0610_COR_00 25-Nov-13
CNU2479K7Z Bell, Mike Software Testing 12-Dec-13
2UA24705ZZ Bell Mike TESTGUIDEX64_0610_COR_00 25-Nov-13
I know this is a lot. I've been working on this for a long time and I'm trying to gain a greater understanding of how to use and manipulate arrays. Also if you have an recommendations on books covering the subject, I'll be happy to review those also. Thank you in advance for all the help.

Resources