Refining Bash loop - arrays

Question

Hello Stackoverflow community,
Please forgive me for my naïveté, but I have a basic loop script running a more complex script that is looking to a text file for the inputs that I have as an array. I have it working, I guess, but I know this can be ran better and more automated.
Here is the text file my script looks to;
2014;14204;
2015;15042;
2015;15062;
...
end;
Here is the bash script I run as a loop;
{ while IFS=';' read YEAR1 PROJ1 YEAR2 PROJ2 YEAR3 PROJ3 fake
do
{ echo "$YEAR1" | egrep '^#|^ *$' ; } > /dev/null && continue
$local/script.sh \
--forinput1 $YEAR1/$PROJ1 \
--forinput2 $YEAR2/$PROJ2 \
--forinput3 $YEAR3/$PROJ3 \
done
} < textFile.txt
I have done some research myself and have found somethings I thought would work but haven't been able to properly implement into this. If you could some me some pointers I would appreciate it.
Edited:
I do apologize, so the script does recognize the text file as such:
YEAR1;PROJ1;
YEAR2;PROJ2;
YEAR3;PROJ3;
Using the ";" as its separator. It does run in a loop until the last variable it done. However, for it function I need to add to the text file extra lines
YEAR4;PROJ4;
YEAR5;PROJ5;
end;
then add the script
{ while IFS=';' read YEAR1 PROJ1 YEAR2 PROJ2 YEAR3 PROJ3 YEAR4 PROJ4 YEAR5 PROJ5 fake
do
{ echo "$YEAR1" | egrep '^#|^ *$' ; } > /dev/null && continue
$local/script.sh \
--forinput1 $YEAR1/$PROJ1 \
--forinput2 $YEAR2/$PROJ2 \
--forinput3 $YEAR3/$PROJ3 \
--forinput4 $YEAR4/$PROJ4 \
--forinput5 $YEAR5/$PROJ5 \
done
} < textFile.txt
What I am hoping to accomplish is adding the variables in the array but not having to add the extra syntax into the script
This is broken but I guess what I am looking along the lines of
{ while IFS=';' read -a YEAR PROJ < textFile.txt
for ((i = 0; i < "${#YEAR[#]};${#PROJ[#]}"; i++)); do
{ echo "$YEAR[$i]" | egrep '^#|^ *$' ; } > /dev/null && continue
$local/script.sh \
--forinput[$i] ${YEAR[$i]}/${PROJ[$i]} \
done
}

Answer 1

Your code suggests that each inputline has 6 fields, what might lead to code like
sed -nr 's#([^;]*);([^;]*);([^;]*);([^;]*);([^;]*);([^;]*);#$local/script.sh --forinput \1/\2 --forinput \3/\4 --forinput \5/\6#p' textFile.txt
# or shorter
f='([^;]*;)'
sed -nr 's#'$f$f$f$f$f$f'#$local/script.sh --forinput \1/\2 --forinput \3/\4 --forinput \5/\6#p' textFile.txt
When you have to combine 3 lines for the input, you should not try to be smart.
# Don't do this
cat textFile.txt | paste -d'X' - - - | tr -d 'X'
# Trying to make
sed -nr 's#'$f$f$f$f$f$f'#$local/script.sh --forinput \1/\2 --forinput \3/\4 --forinput \5/\6#p' <(cat textFile.txt | paste -d'X' - - - | tr -d 'X')
After proudly presenting the code, you see you will have to make the code even worse, when the second line is a comment (starts with '#'). You need to replace cat textFile.txt with grep -Ev '^#|^ *$' testFile.txt.
When you need to relate different lines, take a look at awk.
The solution without checks is
awk -F';' '{line++}
{param=param " --forinput " $1 "/" $2}
line==3 {print "$local/script.sh" param ; param=""; line=0}
' textFile.txt
You can add all kind of checks.

Answer 2

I'm going to assume your input file contains groups of 3 lines each with 2 fields, not lines of 6 fields.
$ cat file
y1a;p1a;
y2a;p2a;
y3a;p3a;
y1b;p1b;
y2b;p2b;
y3b;p3b;
Then, you can put multiple read commands as the while "condition":
while
IFS=';' read year1 proj1 x
IFS=';' read year2 proj2 x
IFS=';' read year3 proj3 x
do
echo script \
--forinput1 "$year1/$proj1" \
--forinput2 "$year2/$proj2" \
--forinput3 "$year3/$proj3"
done < file
script --forinput1 y1a/p1a --forinput2 y2a/p2a --forinput3 y3a/p3a
script --forinput1 y1b/p1b --forinput2 y2b/p2b --forinput3 y3b/p3b
However, this does not handle comments and blank lines. This version executes the script after the 3rd non-(comment/blank) line
$ cat file
# set 1
y1a;p1a;
y2a;p2a;
y3a;p3a;
# set 2
y1b;p1b;
y2b;p2b;
y3b;p3b;
and
n=0
args=()
while IFS=';' read year project x; do
{ [[ $year == *([[:blank:]])"#"* ]] || [[ $year == *([[:blank:]]) ]]; } && continue
((n++))
args+=( "--forinput$n" "$year/$project" )
if (( n == 3 )); then
echo script "${args[#]}"
args=()
n=0
fi
done < file
script --forinput1 y1a/p1a --forinput2 y2a/p2a --forinput3 y3a/p3a
script --forinput1 y1b/p1b --forinput2 y2b/p2b --forinput3 y3b/p3b
Another approach to handle arbitrary records per group:
$ cat file
# set 1
y1a;p1a;
y2a;p2a;
y3a;p3a;
y4a;p4a;
y5a;p5a;
end;
# set 2
y1b;p1b;
y2b;p2b;
y3b;p3b;
end;
$ grep -Pv '^\s*(#|$)' file | awk -F"\n" -v RS="\nend;\n" -v OFS=, '{
cmd = "script.sh"
for (i=1; i<=NF; i++) {
n = split($i, a, /;/)
cmd = sprintf( "%s --forinput%d \"%s/%s\"", cmd, i, a[1], a[2])
}
print cmd
}'
script.sh --forinput1 "y1a/p1a" --forinput2 "y2a/p2a" --forinput3 "y3a/p3a" --forinput4 "y4a/p4a" --forinput5 "y5a/p5a"
script.sh --forinput1 "y1b/p1b" --forinput2 "y2b/p2b" --forinput3 "y3b/p3b"
That uses grep to filter out the comments and blank lines. Then awk comes it to format the commands:
RS is the record separator, using the end; line
-F"\n" sets the field separator to a newline
then we iterate over the number of fields (NF) to construct the command you want to run, and print it out.
To actually execute it, pipe the awk output to | sh