I refer to the following articles for installing SnowAlert.
https://datumstudio.jp/blog/1215_snowflake_17/
I pasted the cat command as shown below and pressed the Enter key, but it doesn't run.
"First, copy and paste the cat command output during the installation into your terminal to create the snowalert- .envs file."
Do you know the cause?
So I assume you are meaning this section:
cat <<END_OF_FILE > snowalert-<account>.envs
SNOWFLAKE_ACCOUNT=<account>
SA_USER=snowalert
SA_ROLE=snowalert
SA_DATABASE=snowalert
SA_WAREHOUSE=snowalert
REGION=<region>.<provider>
PRIVATE_KEY=XXXXXXXXXX
PRIVATE_KEY_PASSWORD=<key_passphase>
END_OF_FILE
Which is not Windows valid commands. I don't have docker or linux on this PC, but using gitBash (MINGW64) I can run that with substitutions just fine:
Simeon#Simeon2 MINGW64 ~
$
if I paste this block
cat <<END_OF_FILE > snowalert-account_name.envs
SNOWFLAKE_ACCOUNT=account_name
SA_USER=snowalert
SA_ROLE=snowalert
SA_DATABASE=snowalert
SA_WAREHOUSE=snowalert
REGION=region.provider
PRIVATE_KEY=XXXXXXXXXX
PRIVATE_KEY_PASSWORD=key_passphase
END_OF_FILE
it is accepted:
Simeon#Simeon2 MINGW64 ~
$ cat <<END_OF_FILE > snowalert-account_name.envs
> SNOWFLAKE_ACCOUNT=account_name
> SA_USER=snowalert
> SA_ROLE=snowalert
> SA_DATABASE=snowalert
> SA_WAREHOUSE=snowalert
> REGION=region.provider
> PRIVATE_KEY=XXXXXXXXXX
> PRIVATE_KEY_PASSWORD=key_passphase
> END_OF_FILE
Simeon#Simeon2 MINGW64 ~
and if I cat that file:
$ cat snowalert-account_name.envs
I get back
SNOWFLAKE_ACCOUNT=account_name
SA_USER=snowalert
SA_ROLE=snowalert
SA_DATABASE=snowalert
SA_WAREHOUSE=snowalert
REGION=region.provider
PRIVATE_KEY=XXXXXXXXXX
PRIVATE_KEY_PASSWORD=key_passphase
so <account>, <region>, <provider>, and <key_passphase> all have to be replaced with correct values that have no <,> values in them, into a bash shell.
OR you can just open the file with any file editor and put the same valid context in.
This question already has answers here:
Reading output of a command into an array in Bash
(4 answers)
Creating an array from a text file in Bash
(7 answers)
Read lines from a file into a Bash array [duplicate]
(6 answers)
Closed last year.
This is the script which I am trying to run
bRules=(`aws s3api get-bucket-lifecycle-configuration --bucket x | jq '.Rules[]'.ID`)
echo "*** Total number of LifecycleRules **** ${#bRules[#]}"
for rule in "${bRules[#]}"
do
echo "Bucket: x, Rule: $rule"
done
Actual output
*** Total number of LifecycleRules **** 6
Bucket: x, Rule: This
Bucket: x, Rule: is
Bucket: x, Rule: Rule-1
Bucket: x, Rule: This
Bucket: x, Rule: is
Bucket: x, Rule: Rule-2
Expected output:
*** Total number of LifecycleRules **** 2
Bucket: x, Rule: This is Rule-1
Bucket: x, Rule: This is Rule-2
What needs to be change in my code snippet to get the desired output? I'm kinda of lost to fix this issue.
Don't rely on command substitution and word splitting. Use readarray (or mapfile if you prefer) and process substitution instead.
Your loop can also be simplified to a single printf.
readarray -t bRules < <(aws s3api get-bucket-lifecycle-configuration --bucket x | jq '.Rules[]'.ID)
echo "*** Total number of LifecycleRules **** ${#bRules[#]}"
printf 'Bucket: x, Rule: %s\n' "${bRules[#]}"
Read the Bash Manual.
If doing:
ECHOCMD:=/bin/echo -e
SHELL := /bin/bash
GITIGNORE_SOURCE_PATH := ../.gitignore
GITIGNORE_DESTINE_PATH := ./setup/.gitignore
start_syncing: "${GITIGNORE_DESTINE_PATH}"
printf '\n'
"${GITIGNORE_DESTINE_PATH}":
cp -vr "${GITIGNORE_SOURCE_PATH}" ./setup/
Make keeps running the rule "${GITIGNORE_DESTINE_PATH}" every time when I call make, but it should only call it when the source file ../.gitignore is modified.
Update 3
This is the make file I was suggested to use on comment:
ECHOCMD:=/bin/echo -e
SHELL := /bin/bash
GITIGNORE_SOURCE_PATH := ../.gitignore
GITIGNORE_DESTINE_PATH := ./setup/.gitignore
start_syncing: "${GITIGNORE_DESTINE_PATH}"
printf '\n'
${GITIGNORE_DESTINE_PATH}: ${GITIGNORE_SOURCE_PATH}
cp -vr "${GITIGNORE_SOURCE_PATH}" ./setup/
Running it, it says:
$ ls -l ../.gitignore
-rwx---r-x+ 1 User None 488 Apr 27 23:23 ../.gitignore
$ ls -l ./setup/.gitignore
-rwx---r-x+ 1 User None 488 Apr 28 07:41 ./setup/.gitignore
$ make
make: *** No rule to make target '"./setup/.gitignore"', needed by 'start_syncing'. Stop.
The solution is to remove double quotes from the target rules. In case they have a space on the name, there is nothing make can do about it other than you manually escape the values of the space with \
ECHOCMD:=/bin/echo -e
SHELL := /bin/bash
GITIGNORE_SOURCE_PATH := ../.gitignore
GITIGNORE_DESTINE_PATH := ./setup/.gitignore
start_syncing: ${GITIGNORE_DESTINE_PATH}
printf '\n'
${GITIGNORE_DESTINE_PATH}: ${GITIGNORE_SOURCE_PATH}
cp -vr "${GITIGNORE_SOURCE_PATH}" ./setup/
I want to get bibliographical information from CTAN using ctanbib.
But instead of writing and executing
ctanbib datatool >> MyPackage-ctan.bib
ctanbib biblatex >> MyPackage-ctan.bib
ctanbib archaeologie >> MyPackage-ctan.bib
(and many more)
I thought of making a make file which would do the job.
But it will not get the information which must be due to the shell command.
NAME = MyPackage
PKG := datatool biblatex archaeologie
CTAN := $(shell ctanbib $$pkg >> $(NAME)-ctan.bib)
getCTAN:
for pkg in $(PKG);\
do \
$(CTAN); \
done
You use the $(shell...) make macro while you do not want shell evaluation. Your CTAN variable is set to the result of the evaluation by the shell of:
ctanbib $pkg >> MyPackage-ctan.bib
When I write "the result" I mean what this command echoes on the standard output. As there is probably no pkg shell variable set for the shell that performs this evaluation, this is probably the same as what the evaluation by the shell of:
ctanbib >> MyPackage-ctan.bib
sends to the standard output. And because you redirect the standard output to a file, this is the empty string. So, your CTAN variable is assigned the empty string.
Try:
CTAN := ctanbib $$pkg >> $(NAME)-ctan.bib
instead. The right hand side of the assignment will be expanded by make when assigning make variable CTAN. As a result the CTAN make variable will be assigned:
ctanbib $pkg >> MyPackage-ctan.bib
which is what you want in your recipe after make expands $(CTAN). Example with the ctanlib command wrapped in echo:
$ cat Makefile
NAME = MyPackage
PKG := datatool biblatex archaeologie
CTAN := echo "ctanbib $$pkg >> $(NAME)-ctan.bib"
.PHONY: getCTAN
getCTAN:
rm -f $(NAME)-ctan.bib
for pkg in $(PKG);\
do \
$(CTAN); \
done
$ make
for pkg in datatool biblatex archaeologie;\
do \
echo "ctanbib $pkg >> MyPackage-ctan.bib"; \
done
ctanbib datatool >> MyPackage-ctan.bib
ctanbib biblatex >> MyPackage-ctan.bib
ctanbib archaeologie >> MyPackage-ctan.bib
I am very tired because of a problem:
I want to run a bash script that automatically creates me another bash script, which in turn should already contain C code that just gets compiled and run on the fly while its output gets piped to aplay.
My example is based on the ComputerPhile bitshift variations code.
I have a C script like this (should get replaced and developed later):
g(i, x, t, o) {
return ((3 & x & (i * (
(3 & i >> 16 ? "BY}6YB6%" : "Qj}6jQ6%")[t%8] + 51
) >> o)) << 4);
};
main(i, n, s) {
for (i = 0; ;i++)
putchar(
g(i, 1, n = i >> 14, 12) +
g(i, s = i >> 17, n^i >> 13, 10) +
g(i, s/3, n + ((i >> 11) % 3), 10) +
g(i, s/5, 8 + n - ((i >> 10) % 3), 9)
);
}
and I want to
1. minify it and
2. escape the " characters to \"
to get a bash script like this:
#!/bin/bash
echo "g(i,x,t,o){return((3&x&(i*((3&i>>16?\"BY}6YB6%\":\"Qj}6jQ6%\")[t%8]+51)>>o))<<4);};main(i,n,s){for(i=0;;i++)putchar(g(i,1,n=i>>14,12)+g(i,s=i>>17,n^i>>13,10)+g(i,s/3,n+((i>>11)%3),10)+g(i,s/5,8+n-((i>>10)%3),9));}"|gcc -xc -&&./a.out|aplay
However, I cannot seem to come near that goal. Either it does make an empty string (echo ""|gcc ...) or it weirdly pastes the script parameters in (...return ((3 & x & (i c2sh.sh tmp waveform.c waveform.sh ( (3 & i >> 16 ? BY}6YB6%..., but I already had a version of this where the " got replaced correctly).
My code (without cleanup so you can see two other things I tried) is currently:
#!/bin/bash
echo '#!/bin/bash' > tmp
out=''
echo $(cat waveform.c | tr '"' ' ')
#while read line; do
# out+="$line"
#done < 'waveform.c'
#out=$(echo $out | sed 's/"/\\"/g')
out=${out/"/\\"}
echo 'echo "'$out'" | gcc -xc -&& ./a.out | aplay' >> tmp
mv tmp 'waveform.sh'
chmod +x 'waveform.sh'
What the heck is going on and how should this be done correctly?
Correctly? It shouldn't be done. In bash, code generation is a "smell", contrary to accepted best practices.
However, if you're going to do it anyway, let the shell do the escaping, rather than trying to write your own logic (which will almost without exception be prone to shell injection vulnerabilities):
#!/bin/bash
infile=${1:-waveform.c} # read name from command-line argument if given
content=$(<"$infile") # $(<foo) is more efficient than $(cat foo)
content=${content//[[:space:]]/} # trim spaces from variable contents
# put the arguments to the command you want to generate into an array
cmd=( printf '%s\n' "$content" )
# tell the shell to generate a string which has each element of that array escaped
printf -v cmd_q '%q ' "${cmd[#]}"
# put our generated contents into an array, one line per each;
# not essential, but makes it easy to comment each line we generate below.
output=(
'#!/bin/bash' # using bash ensures that printf %q-generated code works.
"$cmd_q | gcc -xc - || exit" # use our generated string for all non-constant code.
'./a.out | aplay'
)
printf '%s\n' "${output[#]}" # finally, emit output with all of the above
Items of note:
printf '%q' formats a given shell variable in such a way that, when evaluated by a shell, it will evaluate back to its original literal contents. This is essential not just for correctness, but also for security: If you were listening to waveforms from someone else's code, you wouldn't want /* $(rm -rf ~) */ to be evaluated by your shell.
echo is a very ambiguously-specified command, in order to allow wildly different implementations by various historical operating systems to all comply with the spec. As POSIX recommends, printf should be used instead in any case where the data is not hand-generated and known not to trigger any of the ambiguities.
There is a useful C Minifier *python script on GitHub, minifier.py, which might help:
code=`python minifier.py waveform.c`
echo "$code" ## `printf "%s\n" "$code"` is safer though...
Output:
g(i,x,t,o){return((3&x&(i*((3&i>>16 ? "BY}6YB6%" :
"Qj}6jQ6%")[t%8]+51)>>o))<<4);};main(i,n,s){for(i=0;;i++)putchar(g(i,1,n=i>>14,12)+g(i,s=i>>17,n^i>>13,10)+g(i,s/3,n+((i>>11)%3),10)+g(i,s/5,8+n-((i>>10)%3),9));}
For a more general fix C. Duffy's answer quotes every char, a 100% increase, using some kind of binary to hex encoding would be more efficient, albeit meaningless to humans, e.g.:
python minifier.py waveform.c | base64
ZyhpLHgsdCxvKXtyZXR1cm4oKDMmeCYoaSooKDMmaT4+MTYgPyAiQll9NllCNiUiIDogIlFqfTZq
UTYlIilbdCU4XSs1MSk+Pm8pKTw8NCk7fTttYWluKGksbixzKXtmb3IoaT0wOztpKyspcHV0Y2hh
cihnKGksMSxuPWk+PjE0LDEyKStnKGkscz1pPj4xNyxuXmk+PjEzLDEwKStnKGkscy8zLG4rKChp
Pj4xMSklMyksMTApK2coaSxzLzUsOCtuLSgoaT4+MTApJTMpLDkpKTt9Cg==
Which is unreadable, but so is escaped minified C...