Bash Delete doublons within array - arrays

Question

I have a file containing a several number of fields. I am trying to delete doublons (ex: two same attributes with a different date.) within a same field. For example from this :
Andro manual gene 1 100 . + . ID=truc;Name=truc;modified=13-09-1993;added=13-09-1993;modified=13-09-1997
Andro manual mRNA 1 100 . + . ID=truc-mRNA;Name=truc-mRNA;modified=13-09-1993;added=13-09-1993;modified=13-09-1997
We can see modified=13-09-1993 and modified=13-09-1997 are doublons. So I want to obtain this :
Andro manual gene 1 100 . + . ID=truc;Name=truc;added=13-09-1993;modified=13-09-1997
Andro manual mRNA 1 100 . + . ID=truc-mRNA;Name=truc-mRNA;added=13-09-1993;modified=13-09-1997
I want to keep the latest occurence of particular attribute and deleting the oldest one. They will only have at maximum twice the same attribute in a same row.
I've tried this code (which is now working):
INPUT=$1
ID=$2
ALL_FEATURES=()
CONTIG_FEATURES=$(grep $ID $INPUT)
while read LINE; do
FEATURES=$(echo -e "$LINE" | cut -f 9)
#For each line, store all attributes from every line in an array
IFS=';' read -r -a ARRAY <<< "$FEATURES"
#Once the array is created, loop in array to look for doublons
for INDEX in "${!ARRAY[#]}"
do
ELEMENT=${ARRAY[INDEX]}
#If we are not at the end of the array, compare actual element and next element
ACTUAL=$ELEMENT
for INDEX2 in "${!ARRAY[#]}"
do
NEXT="${ARRAY[INDEX2]}"
ATTRIBUTE1=$(echo -e "$ACTUAL" | cut -d'=' -f1)
ATTRIBUTE2=$(echo -e "$NEXT" | cut -d'=' -f1)
echo "Comparing element number $INDEX ($ATTRIBUTE1) with element number $INDEX2 ($ATTRIBUTE2) ..."
if [[ $ATTRIBUTE1 = $ATTRIBUTE2 ]] && [[ $INDEX -ne $INDEX2 ]]
then
echo "Deleting features..."
#Delete actual element, because next element will be more recent
NEW=()
for VAL in "${ARRAY[#]}"
do
[[ $VAL != "${ARRAY[INDEX]}" ]] && NEW+=($VAL)
done
ARRAY=("${NEW[#]}")
unset NEW
fi
done
done
#Rewriting array into string separated by ;
FEATURES2=$( IFS=$';'; echo "${ARRAY[*]}" )
sed -i "s/$FEATURES/$FEATURES2/g" $INPUT
done < <(echo -e "$CONTIG_FEATURES")
I need advices because I think my array approache may not be a clever one, but I want a bash solution in any case. If anyone has some bash adives/shortcuts, any suggestions will be appreciated to improve my bash understanding.
I'm sorry if I forgot any details, thanks for your help.
Roxane

Answer 1

In awk:
$ awk '
{
n=split($NF,a,";") # split the last field by ;
for(i=n;i>=1;i--) { # iterate them backwards to keep the last "doublon"
split(a[i],b,"=") # split key=value at =
if(b[1] in c==0) { # if key not in c hash
d=a[i] (d==""?"":";") d # append key=value to d with ;
c[b[1]] # hash key into c
}
}
$NF=d # set d to last field
delete c # clear c for next record
d="" # deetoo
}
1 # output
' file
Andro manual gene 1 100 . + . ID=truc;Name=truc;added=13-09-1993;modified=13-09-1997
Andro manual mRNA 1 100 . + . ID=truc-mRNA;Name=truc-mRNA;added=13-09-1993;modified=13-09-1997

Answer 2

Following awk could also help you in same.
awk -F';' '{
for(i=NF;i>0;i--){
split($i, array,"=");
if(++a[array[1]]>1){
$i="\b"
}
};
delete a
}
1
' OFS=";" Input_file
Output will be as follows.
Andro manual gene 1 100 . + . ID=truc;Name=truc;added=13-09-1993;modified=13-09-1997
Andro manual mRNA 1 100 . + . ID=truc-mRNA;Name=truc-mRNA;added=13-09-1993;modified=13-09-1997