Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
I am looking for a table of language names and codes like the ISO 639-1 set: http://en.wikipedia.org/wiki/List_of_ISO_639-1_codes
Thanks
You'll want ISO 639-3 if you want the up-to-date list.
Enhancing Obalix answer, I have created a bash script that will take a UTF-8 CSV file and insert it into a database.
Note that the file provided by Obalix is in UTF-16 NOT UTF-8. The below script checks for its encoding and advises user on how to convert it.
Of course you will need to modify the insert statement according to your schema.
#!/bin/bash
USAGE="Usage: $0 <csv-file>"
if [ $# -lt 1 ]
then
echo $USAGE
exit 1
fi
csv=$1
if [ ! -f $csv ]; then
echo "$csv: No such file"
exit 1
fi
file $csv | grep -q UTF-8
if [ $? -ne 0 ]
then
echo $csv: must be in UTF-8 format, use the following command to fix this:
echo "cat $csv | iconv -f UTF-16 -t UTF-8 | tr -d \"\r\" > utf8-$csv"
exit 1
fi
mysql=<PATH/TO/mysql/BINARY>
db=<DATABASE_NAME>
user=<USERNAME>
pass=<PASSWORD>
sql=insert-all-langs.sql
echo "-- Inserting all languages generated on `date`" > $sql
printf "Processing CSV file..."
# prepend _ to all lines so that no line starts by whitespace
sed 's/.*/_&/' $csv | while read l; do
iso6391=`echo "$l" | cut -f4`
name=`echo -e "$l" | cut -f3 | tr -d "\"" | sed 's/'\''/\\\\'\''/g'`
echo $iso6391:$name
# insert ignore supresses errors for duplicate locales (row still not inserted)
echo "insert ignore into languages (name, locale, rtl, created_at, updated_at) values ('$name', '$iso6391', 0, now(), now());" >> $sql
done
echo Done
printf "Executing SQL..."
cat $sql | $mysql -u$user -p$pass $db
echo Done
Related
I'm trying to improve my scripting skills.
I know there's online converters that are doing it but i'd like to make a script that compare in a certain folder (/opt/Citrix/ICAClient/keystore/cacerts/), if the .crt are already in the .pem format and if not convert them.
So i know this clearly isn't the best way to do so... But i've started like this:
`
#!/bin/bash
#remove both files if they already exists
rm crt.txt
rm pem.txt
#certificate selection in .crt format
Certificate_crt=$(sudo ls /opt/Citrix/ICAClient/keystore/cacerts/ | grep .crt | sed 's/\(.*\)\..*/\1/')
#certificate selection in .pem format
Certificate_pem=$(sudo ls /opt/Citrix/ICAClient/keystore/cacerts/ | grep .pem | sed 's/\(.*\)\..*/\1/')
#sending results in text files
echo "$Certificate_crt" >> crt.txt
echo "$Certificate_pem" >> pem.txt
#recovery of certificate names in .crt not having a .pem equivalent
Certificate_crt_WO_pem=$(cat crt.txt | grep -v "$Certificate_pem" | tr " " "\n")
#echo "$Certificate_crt_WO_pem"
#initialisation
i=0
#Print the split string
for i in "${Certificate_crt_WO_pem[#]}"
do
name_certificate=$(echo $i | tr " " "\n")
echo "${name_certificate[#]}"
echo "$i"
i=i+1
done
`
The thing is that when my "for" is launched, it stores all the result of "Certificate_crt_WO_pem" in the array $name_certificate[0] and then stop it self.
What i want is to store, line by line, the result of "cat crt.txt | grep -v "$Certificate_pem" | tr " " "\n"" into the array name_certificate.
This array will be use to launch something like this " openssl -in $name_certificate[$i].crt -out $name_certificate[$i].pem PEM" (in a for loop in the will to convert every namefile.crt in every namefile.pem).
If someone can help me i'll be more than gratefull... (And yes i've already tried to search on the net, had followed some online courses but none of them was saying the same thing about the bash's arrays, so i'm a bit lost...)
What i want is to store, line by line, the result of
Then do that.
var=$(something)
# ^^ - normal variable assignment
var=( $(something) )
# ^^ ^ - array assignment
# the unquoted result of expansions is split on IFS
# default on tabs, newlines and spaces
so I guess you want:
Certificate_crt_WO_pem=($(grep -v "$Certificate_pem" crt.txt))
Doing cat file | grep is a useless use of cat. Use grep .. file or < file grep ....
Remember do not parse ls output. Don't ls | something. Prefer globulation or find instead.
Read how to read a stream line by line in bashfaq. Read how to use arrays in bashfaq.
Note that grep parses a regex, so grep .pem matches any character followed by pem. I guess you wanted grep '\.pem'. I do not think grep -v "$Certificate_pem" does what you think it does - I guess you meant to use comm or join to filter elements from one newline separated separated that are present in the other list.
This array will be use to launch something like this " openssl -in $name_certificate[$i].crt -out $name_certificate[$i].pem PEM"
The best would be not do it, rather parse the data as they come, not store them in variables.
# get all .crt fiels
find /opt/Citrix/ICAClient/keystore/cacerts/ -maxdepth 1 -mindepth 1 -type f -name '*.crt' |
# remove extension
sed 's/\.crt$//' |
# filter out files where .pem does exists already
while IFS= read -r file; do
if [[ -r "$file".pem ]]; then continue; fi
printf "%s\n" "$file"
done |
# execute openssl for the rest
xargs -d'\n' -i{} openssl -in {}.crt -out {}.pem PEM
but if you want, rather use mapfile to store a string with a newline separated list into an array (be sure to understand how to store variables in a pipeline).
mapfile -t Certificate_crt_WO_pem < <(something)
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
How to print $arr appended value in new line? Below is the code.
arr=() |
hive -e 'show tables in database'|
while read line
do
echo "The name of the line is $line"
arr+="TABLE NAME : $line"
done
echo $arr
There are several issues in the code.
Piping the assignment to an array makes no sense. The assignment has no output, so there's nothing to pipe.
+= without parentheses does a string concatenation, so only ${arr[0]} gets changed. Use
arr+=("TABLE NAME : $line")
Commands in a pipeline are running in subshells, which means the assignment only happens in a subshell, the array in the main shell is not updated. Use "process substitution" instead:
while ...
done < <(hive ...)
Also, I'd rather store just the table names in the array, as you can reuse the values later, instead of storing the whole messages. Fixing all these, we can get something like
#!/bin/bash
tables=()
while IFS= read -r line ; do
echo "The name of the line is $line"
tables+=("$line")
done < <(hive -e 'show tables in database')
printf 'TABLE NAME: %s\n' "${tables[#]}"
This is a good place for the mapfile command: reads the output of a command and stores each line as an array element. Useful with the process substitution here
mapfile -t arr < <(hive ...)
for elem in "${arr[#]}"; do
echo "TABLE: $elem"
done
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
Currently for a header file named test_header.h I use -
#ifndef TEST_HEADER_H
#define TEST_HEADER_H
/* code */
#endif /* TEST_HEADER_H */
What I want is a header guard which doesn't use the file name directly. Something like (i.e. wishful crude hypothetical solution) -
#if __FILE__ not in INCLUDE_LIST
#APPEND(INCLUDE_LIST, __FILE__)
/* code */
#endif
As others have noted, what you use as a header guard isn't intrinsically important; it just needs to be unique across the set of headers that might ever be co-included.
You can create a UUID or GUID and use that as the header guard (or a hash of some sort — MD5, SHA1, SHA2, SHA3, …). The only trick is dealing with the possibility of a leading digit; that's easily worked around (I used H_ as a prefix).
Mostly though, I use a name based on the file name, and don't usually rename headers often enough that it is a problem.
Here's a script called hdrguard that I use for generating the header guard lines for a given header file:
#!/bin/sh
#
# #(#)$Id: hdrguard.sh,v 1.8 2016/05/09 18:41:57 jleffler Exp $
#
# Generate #ifndef sequence to guard header against multiple inclusion
arg0=$(basename $0 .sh)
usestr="Usage: $arg0 [-bdfhimV] header.h [...]"
usage()
{
echo "$usestr" 1>&2
exit 1
}
help()
{
echo "$usestr"
echo
echo " -b Use base name of file for guard"
echo " -d Use _DOT_H after name (instead of _H)"
echo " -f Use specified path name of file for guard (default)"
echo " -h Print this help message and exit"
echo " -i Omit _INCLUDED after name"
echo " -m Generate MD5 hash value as header guard"
echo " -V Print version information and exit"
exit 0
}
opt_incl=yes
opt_base=no
opt_dot=no
opt_md5=no
while getopts bdfhimV opt
do
case "$opt" in
(b) opt_base=yes;;
(d) opt_dot=yes;;
(f) opt_base=no;;
(h) help;;
(i) opt_incl=no;;
(m) opt_md5=yes;;
(V) echo "$arg0: HDRGUARD Version "'$Revision: 1.8 $ ($Date: 2016/05/09 18:41:57 $)' | rcsmunger; exit 0;;
(*) usage;;
esac
done
shift $(($OPTIND - 1))
[ $# -eq 0 ] && usage
for i in "$#"
do
if [ $opt_base = yes ]
then i=$(basename $i)
fi
if [ $opt_dot = yes ]
then i=$(echo "$i" | sed 's/\.h$/_dot_h/')
fi
i=$(echo $i | tr 'a-z' 'A-Z' | tr -s '/+.-' '____' | sed 's/^_//')
if [ $opt_incl = yes ]
then
case "$i" in
(*_INCLUDED)
: OK;;
(*)
i="${i}_INCLUDED";;
esac
fi
if [ $opt_md5 = yes ]
then
tmp=$(mktemp ./hdrgrd.XXXXXXXX)
trap "rm -f $tmp; exit 1" 0 1 2 3 13 15
echo "$i.$(isodate compact)" > "$tmp"
i=$(md5 "$tmp" | sed 'y/abcdef/ABCDEF/; s/\([^ ]*\) .*/H_\1/')
rm -f "$tmp"
trap 0 1 2 3 13 15
fi
echo
echo "#ifndef $i"
echo "#define $i"
echo
echo "#endif /* $i */"
echo
done
It doesn't have code for SHA1, SHA2 or SHA3 — it optionally uses MD5 (in the form of a command md5). It would not be very hard to add support for alternative hashing algorithms. It doesn't require the file to exist.
Example uses:
$ hdrguard header.h
#ifndef HEADER_H_INCLUDED
#define HEADER_H_INCLUDED
#endif /* HEADER_H_INCLUDED */
$ hdrguard -m header.h
#ifndef H_6DC5070597F88701EB6D2CCAACC73383
#define H_6DC5070597F88701EB6D2CCAACC73383
#endif /* H_6DC5070597F88701EB6D2CCAACC73383 */
$
I frequently use it from within vim, typing a command such as !!hdrguard % while the cursor is on an empty line to generate a header guard suitable for the header I'm editing. That's why it generates the blank lines top and bottom, too.
The command uses scripts isodate and rcsmunger. With the argument compact, the isodate command is equivalent to:
date +'%Y%m%d.%H%M%S'
The complete command supports a number of alternative formats and is more succinct than having to type the date command out everywhere. You're entirely at liberty to forego the use of a separate script and to just embed the expansion shown into hdrguard. Indeed, you could use just date and it would be OK; it is just seed material for the hashing operation to make the data being hashed unique.
$ isodate compact
20161228.185232
$
The rcsmunger command just converts RCS ID strings into a format I prefer for reporting version information:
#!/usr/bin/env perl -p
#
# #(#)$Id: rcsmunger.pl,v 1.9 2015/11/02 23:54:32 jleffler Exp $
#
# Remove the keywords around the values of RCS keywords
use strict;
use warnings;
# Beware of RCS hacking at RCS keywords!
# Convert date field to ISO 8601 (ISO 9075) notation
s%\$(Date:) (\d\d\d\d)/(\d\d)/(\d\d) (\d\d:\d\d:\d\d) \$%\$$1 $2-$3-$4 $5 \$%go;
# Remove keywords
s/\$([A-Z][a-z]+|RCSfile): ([^\$]+) \$/$2/go;
For example:
$ hdrguard -V
hdrguard: HDRGUARD Version 1.8 (2016-05-09 18:41:57)
$
You can regard the printing of version information as old-school version control; it has to be done differently if you use a DVCS such as git, which is one reason I've not done a wholesale migration to git for my personal software collection.
You can name the header guards whatever you like.
If your header is related to a certain topic, name it after that i.e. LIST_OPERATIONS
Omit the guards and write a script that inserts them based on the current file name. Run this script as part of your build process before compiling. (Remember to either create a modified copy or remove the includes after the build, otherwise you end up with a lot of include guards in your header)
Depending on the compilers that should be used for your project, they might support the #pragma once approach.
#pragma once
It's very portable, being well-supported by all major compilers, and by 13 out of 14 compilers (according to Wikipedia).
Also check out #pragma once vs include guards?.
I am new to database management and we are using psql. All I need to do is to migrate csv (around 200 tables) to our database. Manually creating tables for every csv file is bit tiresome so please help me out, Is there any way to generate table from csv file?
Answered at DBA Stackexchange by the OP. I'm copying the answer here because this was the first link returned by my search engine.
OP made a script like:
DATADIR='data' # this directory name
PREFIX='jobd'
DBNAME='divacsv'
function createSchema {
COLUMNS=`head -n 1 $1 |
awk -F, '{for(i=1; i<=NF; i++){out=out $i" text, ";} print out;}' |
sed 's/ text, $/MYEXTRA text/' |
sed 's/"//g'`
CMD_CREATE="psql $DBNAME -c \"CREATE TABLE $2 ($COLUMNS);\""
echo $CMD_CREATE
sh -c "$CMD_CREATE"
CMD_COPY="psql divacsv -c \"COPY $2 FROM '"`pwd`"/$1' DELIMITER ',' CSV;\""
echo $CMD_COPY
sh -c "$CMD_COPY"
}
for file in $DATADIR/*.csv; do
table=$PREFIX"_"`echo $file | sed 's/.*\///' | sed 's/.csv//' `
createSchema "$file" $table
done
Comments advise that HEADER might be needed to avoid loading first line with header texts, which is true.
I've tested this code but couldn't make it work under CentOS.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
Using bash script, I want to count the numbers between zeros only. Is it possible to use awk? Sorry, i am new to this.
5000000000009228247152000000000000000000003
5000000000006338293700000000000000000000001
grep -Po '0+\K[^0]+(?=0)'
gives you:
9228247152
63382937
EDIT
If things could be easily done in a single process (awk here is the case), I won't start 2nd process.
awk one-liner with count and text:
awk -F'0+' 'NF>2{for(i=2;i<NF;i++)printf "text:%s count:%d\n",$i,length($i)}'
gives:
text:9228247152 count:10
text:63382937 count:8
awk one-liner only with count:
awk -F'0+' 'NF>2{for(i=2;i<NF;i++)print length($i)}'
gives:
10
8
With sed:
echo 5000000000009228247152000000000000000000003 | \
sed -n -r -e 's/0+([^0]+)0+/\1/' | wc -m
$ grep -oP '0+\K[1-9]+(?=0+)' file
9228247152
63382937
To count the number of digits,
$ grep -oP '0+\K[1-9]+(?=0+)' file | awk -v FS="" '{print NF}'
10
8