my problem is the following: I have this bash script:
#!/bin/bash
IFSBAK=$IFS
if [ "$TXTEXT" = "" ];
then
CMD="find . -iname \"*.txt\" -or -iname \"*.text\""
else
CMDTEMP="find . "
IFS=":"
for i in $TXTEXT
do
CMDTEMP="${CMDTEMP} -iname \"*.${i}\" -or"
done
IFS=$IFSBAK
CMD=${CMDTEMP%-or}
fi
FILES=$(eval $CMD)
OUTPUT=$1
for f in $FILES
do
VAR=$(grep -ae [a-zA-Z0-9] "$f" | tr -cs "[:alnum:]" "\n")
IFS=$' \n\t-?=!*][.\",();\'\`\ยด:'
for v in $VAR
do
echo $v >> "${OUTPUT}"
done
IFS=$' \n\t'
done
and I need to insert this code inside a C program. I've tried to re-write the all script on a single line testing it directly with the shell and it works, but I'm having problems with quotes and escaping trying to use it as a parameter of the system() call.
Can you suggest me a way out?
Thank you for your help
If you really have no choice but to deliver a single binary and you cannot ship the shell script file with the binary consider the following:
Include the contents of the script in a single literal
During execution of the program, print the contents of the literal to a temporary file. You probably need some strategy to come up with a unique filename.
Call the temporary script through system() call
Delete the temporary file
However, consider this as a last resort.
Put it into a shell script and invoke the shell script from your C code. Much easier to maintain IMHO.
#define SHELLSCRIPT "
...
write your shell script code here
...
"
int main()
{
system(SHELLSCRIPT);
return 0;
//put the bash to C program
}
Related
I have two compiled .c files and I am trying to take the exit status of the first and use it as an arg to the next one.
#!/bash/bin
./decipher $1
key = $?
./cipher $key $1 $2
This is what I am typing on the command line
$ ./decryption_tool.bs ceaser1.txt output.txt
.ceaser1.txt is an encrypted message and decipher will return a key and cipher should take that key and unencrypt it. Both files work outside of the script as well, but when i use the script I get this error.
./decryption_tool.bs: line 4: key: command not found
./decryption_tool.bs: line 5: 5937 Segmentation fault (core dumped) ./cipher $key $1 $2
Thanks for any help.
Few things wrong:
You cannot have spaces when you set variables in shell script - it should be key=$? instead of key = $?
Invalid shebang: not #!/bash/bin, but #!/bin/bash or even better #!/usr/bin/env bash
It would be good to surround variables in double quotes: ./cipher "$key" "$1" "$2"
Why file extension is .bs? For shell script it's .sh and Bash doesn't have its own separate version
I have one ini configuration file, I need to create a shell script using this configuration. What is the easiest method to access all variable, Can be used effectively from the shell script.
Can I use an array or something? Now planing to find the count of [] brackets then through awk get all variables one by one. Please suggest if any easiest way to effectively
cat app.ini
Below the output of my sample configuration file. Can be N no of Blocks.
[APP1]
name=Application1
StatusScript=/home/status_APP1.sh
startScript=/home/start_APP1.sh
stopScript=/home/stop_APP1.sh
restartScript=/home/restart.APP1.sh
logdir=/log/APP1/
[APP2]
name=Application2
StatusScript=/home/status_APP2.sh
startScript=/home/start_APP2.sh
stopScript=/home/stop_APP2.sh
restartScript=/home/restart.APP2.sh
logdir=/log/APP2/
.
.
.
.
.
[APPN]
name=ApplicationN
StatusScript=/home/status_APPN.sh
startScript=/home/start_APPN.sh
stopScript=/home/stop_APPN.sh
restartScript=/home/restart.APPN.sh
logdir=/log/APPN
/
Consider using a library like bash-ini-parser https://github.com/albfan/bash-ini-parser. It covers a lot of nuances like indentation, whitespaces, comments etc.
The example for your case may look like this:
#!/bin/bash
. bash-ini-parser
cfg_parser app.ini
cfg_section_APP1
echo $name
cfg_section_APP2
echo $logdir
cfg_section_APPN
echo $logdir
Below line help us to locate particular values in each section.
sed -nr "/^\[APP1\]/ { :l /^name[ ]*=/ { s/.*=[ ]*//; p; q;}; n; b l;}" app.ini
I have a list of strings that I have in a variable and would like to remove those strings from a list of filenames. I'm pulling that string from a file that I can add to and modify over time. Some of the strings in the variable may include part of the item needed to be removed while the other may be another line in the list. Thats why I need to loop thru the entire variable list.
I'm familiar using a while loop to loop thru a list but not sure how I can loop thru each line to remove all strings from that filename.
Here's an example:
getstringstoremove=$(cat /text/from/some/file.txt)
echo "$getstringstoremove"
# Or the above can be an array
getstringstoremove=$(cat /text/from/some/file.txt)
declare -a arr=($getstringstoremove)
the above 2 should return the following lines
-SOMe.fil
(Ena)M-3_1
.So[Me].filEna)M-3_2
SOMe.fil(Ena)M-3_3
Here's the loop I was running to grab all filenames from a directory and remove anything other than the filenames
ls -l "/files/in/a/folder/" | awk -v N=9 '{sep=""; for (i=N; i<=NF; i++) {printf("%s%s",sep,$i); sep=OFS}; printf("\n")}' | while read line; do
echo "$line"
returns the following result after each loop
# 1st loop
ilikecoffee1-SOMe.fil(Ena)M-3_1.jpg
# iterate thru $getstringstoremove to remove all strings from the above file.
# 2nd loop
ilikecoffee2.So[Me].filEna)M-3_2.jpg
# iterate thru $getstringstoremove again
# 3rd loop
ilikecoffee3SOMe.fil(Ena)M-3_3.jpg
# iterate thru $getstringstoremove and again
done
the final desired output would be the following
ilikecoffee1.jpg
ilikecoffee2.jpg
ilikecoffee3.jpg
I'm running this in bash on Mac.
I hope this makes sense as I'm stuck and can use some help.
If someone has a better way of doing this by all means it doesn't have to be the way I have it listed above.
You can get the new filenames with this awk one-liner:
$ awk 'NR==FNR{a[$0];next} {for(i in a){n=index($0,i);if(n){$0=substr($0,0,n-1)substr($0,n+length(i))}}} 1' rem.txt files.lst
This assumes your exclusion strings are in rem.txt and there's a files list in files.lst.
Spaced out for easier commenting:
NR==FNR { # suck the first file into the indices of an array,
a[$0]
next
}
{
for (i in a) { # for each file we step through the array,
n=index($0,i) # search for an occurrence of this string,
if (n) { # and if found,
$0=substr($0,0,n-1)substr($0,n+length(i))
# rewrite the line with the string missing,
}
}
}
1 # and finally, print the line.
If you stow the above script in a file, say foo.awk, you could run it as:
$ awk -f foo.awk rem.txt files.lst
to see the resultant files.
Note that this just shows you how to build new filenames. If what you want is to do this for each file in a directory, it's best to avoid running your renames directly from awk, and use shell constructs designed for handling files, like a for loop:
for f in path/to/*.jpg; do
mv -v "$f" "$(awk -f foo.awk rem.txt - <<<"$f")"
done
This should be pretty obvious except perhaps for the awk options, which are:
-f foo.awk, use the awk script from this filename,
rem.txt, your list of removal strings,
-, a hyphen indicating that standard input should be used IN ADDITION to rem.txt, and
<<<"$f", a "here-string" to provide that input to awk.
Note that this awk script will work with both gawk and the non-GNU awk that is included in macos.
I think I have understood what you mean, and I would do it with Perl which comes built-in to the standard macOS - so nothing to install.
I assume you have a file called remove.txt with your list of stuff to remove, and that you want to run the script on all files in your current directory. If so, the script would be:
#!/usr/local/bin/perl -w
use strict;
# Load the strings to remove into array "strings"
my #strings = `cat remove.txt`;
for(my $i=0;$i<$#strings;$i++){
# Strip carriage returns and quote metacharacters - e.g. *()[]
chomp($strings[$i]);
$strings[$i] = quotemeta($strings[$i]);
}
# Iterate over all filenames
my #files = glob('*');
foreach my $file (#files){
my $new = $file;
# Iterate over replacements
foreach my $string (#strings){
$new =~ s/$string//;
}
# Check if name would change
if($new ne $file){
if( -f $new){
printf("Cowardly refusing to rename %s as %s since it involves overwriting\n",$file,$new);
} else {
printf("Rename %s as %s\n",$file,$new);
# rename $file,$new;
}
}
}
Then save that in your HOME directory as renamer. Make it executable - only necessary once - with this command in Terminal:
chmod +x $HOME/renamer
Then you can go in any directory where you madly named files are and run the script like this:
cd path/to/mad/files
$HOME/renamer
As with all things you download off the Internet, make a backup first and just run on a small, copied, subset of your files till you get the idea of how it works.
If you use homebrew as your package manager, you could install rename using:
brew install rename
You could then take all the Perl from my other answer and condense it down to a couple of lines and embed it in a rename command which would give you the added benefit of being able to do dry-runs etc. The code below does exactly the same as my other answer but is somewhat harder to read for non_perl folk.
Your command would simply be:
rename --dry-run '
my #strings = map { s/\r|\n//g; $_=quotemeta($_) } `cat remove.txt`;
foreach my $string (#strings){ s/$string//; } ' *
Sample Output
'ilikecoffee(Ena)M-3_1' would be renamed to 'ilikecoffee'
'ilikecoffee-SOMe.fil' would be renamed to 'ilikecoffee'
'ilikecoffee.So[Me].filEna)M-3_2' would be renamed to 'ilikecoffee'
To try and understand it, remember:
the rename part applies the following Perl to each file because of the asterisk at the end
the #strings part reads all the strings from the file remove.txt and removes any carriage returns and linefeeds from them and quotes any metacharacters
the foreach applies each of the deletions to the current filename which rename stores in $_ for you
Note that this method trades simplicity for performance somewhat. If you have millions of files to do, the other method will be quicker because here I read the remove.txt file for each and every file whose name is checked, but if you only have a few hundred/thousand files, I doubt you'll notice it.
This should be much the same, just shorter:
rename --dry-run '
my #strings = `cat remove.txt`; chomp #strings;
foreach my $string (#strings){ s/\Q$string\E//; } ' *
I'm stucked on a bash script.
I'm having a config.ini files like this :
#Username
username=user
#Userpassword
userpassword=password
And i'm looking in a bash script to extract this information and put it in a associative array. My script looks like :
declare -A array
OIFS=$IFS
IFS='='
grep -vE '^(\s*$|#)' file | while read -r var1 var2
do
array+=([$var1]=$var2)
done
echo ${array[#]}
But the array seems to be empty because the commande echo ${array[#]} gives no output.
Any idea why me script don't work ? Thanks for your help and sorry for my bad english.
Common error - "grep | while" causes the while loop to be executed in a separate shell and the variables inside the loop are not global to your shell. Use a here string instead:
while read -r var1 var2
do
array+=([$var1]=$var2)
done <<< $(grep -vE '^(\s*$|#)' file)
Assuming the file can be trusted (ie the content is regulated and known), the simplest method would be to source the ini file and then directly use the variable names within the script:
. config.ini
You can either use the period (.) as above or the source builtin command
I have some bash script where I get values of variable, that I would like use in awk.
Does it possible include whole awk (like it possible with bash script files) file in bash e.g.:
#!/bin/sh
var1=$1
source myawk.sh
and myawk.sh:
print $1;
Bash and awk are different languages, each with their own interpreter of the same name. The tiny sample you show is stripped down too far to make much sense:
You've marked both files as shell scripts; one using the shebang #!/bin/sh and the other using the extension .sh. Obviously the shell can read shell script, and the command to do so is called . in Bourne shell (or source in csh and bash).
The shell script assigns a variable, but you're not using it anywhere. Did you mean passing it on to the awk script?
Both the awk and shell script use $1, which has different meanings for them (in bash, it's from the command line or a set command; in awk, it's from a parsed input line).
The two tools are often used in tandem, as the shell is better at combining separate programs and awk is better at reformatting tabular or structured text. It was so common that a whole language evolved to combine the tasks; Perl's roots are as a combination of shell, awk and sed.
If you just wanted to pass a variable from the shell script into an awk script, use -v. The man page is your friend.
first of all, if you're writing bash don't use #!/bin/sh that will put you in compatibility mode which is only necessarly if you're writing for portability (and then you have to adhere to the POSIX normative).
now regarding your question you just have to run awk from inside your bash script, like this:
#!/bin/bash
var1=$1
awk -f myawk.sh
also you should use .awk as extension I guess.
Or, many ppl do sth like this:
#!/bin/env bash
#Bash things start
...
var1=$1
#Bash things stop
#Awk things start,
#may use pipes or variable to interact with bash
awk -v V1=var1 '
#AWK program, can even include awk scripts here.
'
#Bash things
I suggest this page here by Bruce Barnett:
http://www.grymoire.com/Unix/Awk.html#uh-3
You can also use double quote to make use of shell's extract feature but it is confusing.
Personally I just try to avoid those fancy gnu additions of bash or awk and make my scripts ksh+(n)awk compatible.
As an hardcore AWK user, I soon realized that doing the following was really a huge help :
Defining and exporting an AWK_REPO variable in my bashrc
#Content of bashrc
export AWK_REPO=~/bin/AWK
Storing there every AWK script I write using the .awk extension.
You can then call it from anywhere like this :
awk -f $AWK_REPO/myScript.awk $file
or even, using Shebangs and adding AWK_REPO to PATH (with export PATH=${AWK_REPO}:${PATH})
myScript.awk $file