How to speed up bash random name generation? - database

Question

i have problem with my code performance. It is running very slow. I need to generate million+ random persons for my postgres db and insert them into db. Person has parameters name,birthdate,gender,age. I created lists for first names and last names from which i am randomly selecting name. Can someone help me?
Here is my code:
#docker params
name="`docker ps | rev | cut -d " " -f1 | rev | grep -v NAMES`"
dbs_name="DBS_projekt"
#load names from files
firstName=(`cat generatorSource/firstNames.txt`)
firstNameCount="`wc -l generatorSource/firstNames.txt | tr -s ' ' | cut -d ' ' -f2`"
secondName=(`cat generatorSource/lastNames.txt`)
secondNameCount="`wc -l generatorSource/lastNames.txt| tr -s ' ' | cut -d ' ' -f2`"
#gender array
gender=("Male" "Female" "Other")
#actual date
now=$(date | rev | cut -d " " -f1 | rev)
array=()
for ((x = 1; x <= 1000;x++))
do
array+="INSERT INTO persons(name,birthdate,gender,age) VALUES"
for (( n=1; n<=1000; n++ ))
do
secondrand=$(( ( RANDOM % $secondNameCount ) ))
firstrand=$(( ( RANDOM % $firstNameCount ) ))
genderand=$(( ( RANDOM % 3 ) ))
year=$(( ( RANDOM % 118 ) + 1900 ))
month=$(((RANDOM % 12) + 1))
day=$(((RANDOM % 28) + 1))
age=$(expr $now - $year)
if [ $n -eq 1000 ]; then
array+="('${firstName[$firstrand]}
${secondName[$secondrand]}','$year-$month-$day',
'${gender[$genderand]}','$age');"
else
array+="('${firstName[$firstrand]}
${secondName[$secondrand]}','$year-$month-$day',
'${gender[$genderand]}','$age'),"
fi
done
done
#run psql in docker and run insert commands
docker exec -i $name psql -U postgres << EOF
\c $dbs_name
$array
EOF

Answer 1

Note that you declare "array" as an array, but you use it as a string.
array=()
...
array+="INSERT INTO persons(name,birthdate,gender,age) VALUES"
This is what's happening:
$ array=()
$ declare -p array
declare -a array='()'
$ array+="first"
$ array+="second"
$ declare -p array
declare -a array='([0]="firstsecond")'
To insert an element into an array, you must use parentheses:
$ array=()
$ array+=("first")
$ array+=("second")
$ declare -p array
declare -a array='([0]="first" [1]="second")'
I suspect this may be one source of slowness: you're constructing one gigantic string. Add the parentheses as shown, and then change the docker call to
IFS=$'\n'
docker exec -i $name psql -U postgres << EOF
\c $dbs_name
${array[*]}
EOF