I have the below Extended Choice Parameter in my Job:
In case I choose both values, how can I loop thought it in my jenkinsfile?
sh "docker run --rm --net=host -v ${WORKSPACE}:/app/ ${MyImage} --env ${ENV}"
This is what I used (with the help of this post: Single parameter with multiple values - referencing extended-choice parameter values):
str = env.ENV.split(',')
for (String values :str)
sh "docker run --rm --net=host -v ${WORKSPACE}:/app/ ${MyImage} --env $values"
Related
This question already has answers here:
What is the use of eval `opam config env` or eval $(opam env) and their difference?
(2 answers)
Closed 9 months ago.
I keep seeing
eval $(opam env)
e.g. when I try to install coq:
# brew install opam # for mac
# for ubuntu
conda install -c conda-forge opam
opam init
# if doing local env?
# eval $(opam env)
# - install coq
# local install
#opam switch create . 4.12.1
#eval $(opam env)
#opam repo add coq-released https://coq.inria.fr/opam/released
#opam install coq
# If you want a single global (wrt conda) coq installation (for say your laptop):
opam switch create 4.12.1
opam switch 4.12.1
opam repo add coq-released https://coq.inria.fr/opam/released
opam install coq
but I never know what it does (nor no one I talk to knows) and when I google it this comes up instead: What is the use of eval `opam config env`?
cross posted: https://discuss.ocaml.org/t/what-does-eval-opam-env-do-does-it-activate-a-opam-environment/9990
Clarification
Note that
eval `opam config env`
might be very similar. In the sense that I think the dashes '' are also command substitution as in $(...) but not sure. Which might make What is the use of eval `opam config env`? very related (though I don't know the difference for sure)
tldr: it activates your opam context according to your current switch -- similar to how a python virtual env is activated in python but for opam.
What eval $(opam env) does is command substitute $(opam env) i.e. run opam env in a subshell (that's what $( ) does) return the string it outputted and then give it to eval to evaluate as bash code. Command substitution is usually done to next the output of commands in bash. In python eval would interpret the input string to it as literal python code to evaluate, parse, run here similarly it would do the same but assume it's bash (is my guess).
What does opam env does? It returns the current bash env variables for the current swithc (i.e. siwth approximately euqal to opam environment). So therefore, doing:
eval $(opam env)
does this:
first runs in a subshell because thats what $(cmd) does. It's command substitution. Usually used to nest commands.
then it's output (so the output string of $(opam env)) is given to eval. opam env returns a string of env variables needed to "activate: the current opam env (similar to how you activate virtual envs in python).
finally eval evaluates the string it receives from the command substitution i.e. it parses the string as a bash command and runs it.
For compeleteness see the output of opam env:
(iit_synthesis) brandomiranda~ ❯ opam env
OPAM_SWITCH_PREFIX='/Users/brandomiranda/.opam/4.12.1'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='/Users/brandomiranda/.opam/4.12.1/lib/stublibs:/Users/brandomiranda/.opam/4.12.1/lib/ocaml/stublibs:/Users/brandomiranda/.opam/4.12.1/lib/ocaml'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='/Users/brandomiranda/.opam/4.12.1/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
PATH='/Users/brandomiranda/.opam/4.12.1/bin:/Users/brandomiranda/miniconda/envs/iit_synthesis/bin:/Users/brandomiranda/miniconda/condabin:/usr/local/bin:/Users/brandomiranda/.opam/4.12.1/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin'; export PATH;
Comment on backard ticks:
Please note : The back ticks shown around opam env after eval are essential. They change the order of application, which is very important. The back ticks tells the system to first evaluate opam env (which returns a string of commands) and then eval executes those commands in the string. Executing them doesn’t return anything, but it initializes the Opam environment behind the scenes.
ref: https://ocaml.org/docs/up-and-running
also see:
What does eval $(opam env) do, does it activate a opam environment?
What is the use of eval `opam config env` or eval $(opam env) and their difference?
https://ocaml.org/docs/up-and-running
https://discuss.ocaml.org/t/what-does-eval-opam-env-do-does-it-activate-a-opam-environment/9990
All details that lead to this answer:
From the man page for opam env:
This is most usefully used as eval $(opam env) to have further shell commands be evaluated in the proper opam context.
see: https://opam.ocaml.org/doc/man/opam-env.html
Slightly more details:
Finally, to get all the pieces together, what is the difference between command substitution (e.g. $( ) or ) and evaluating an expression (e.g. eval string or eval $(cmd) ) (in bash mainly). Ref: https://unix.stackexchange.com/a/23116/55620. My understanding is that $( ) runs a command in a subshells and outputs the string so that it's string is an input to another command e.g. eval. So to my understanding you can think of $(cmd) as replacing that expression and pasting the string of it back to to the command line for another commadn e.g. eval. While eval will take in the string and evaluate the expression in it instead. So what eval $(cmd) does is first evaluate $(cmd) replace it with the output string of running cmd in a subshell and then feeding it to eval -- where eval will pretend that input is code and evaluate it, parse and run it. So $(...) is usually done to nest command inside other commands.
The only puzzling thing to me is why $(cmd) without a cmd2 (eg eval) in the front interprets the strings of $(cmd) as bash commands to run instead of just evaluting e.g.
(iit_synthesis) brandomiranda~ ❯ $(echo v=1)
zsh: command not found: v=1
but
(iit_synthesis) brandomiranda~ ❯ v=1
(iit_synthesis) brandomiranda~ ❯ echo $v
1
does evaluate my string v=1 and then run it.
Thus what eval $(opam env) does is command substitute $(opam env) i.e. run opam env in a subshell return the string it outputted and then given to eval to evaluate as bash code. In python eval would interpret the input string to it as literal python code to evaluate, parse, run here similarly it would do the same but assume its bash (is my guess)
Long answer with all details:
toolchain = In software, a toolchain is a set of programming tools that is used to perform a complex software development task or to create a software product, which is typically another computer program or a set of related programs. https://www.google.com/search?q=toolchain&oq=toolchain&aqs=chrome..69i57j0i512l9.209j0j7&sourceid=chrome&ie=UTF-8
(iit_synthesis) brandomiranda~ ❯ opam env
OPAM_SWITCH_PREFIX='/Users/brandomiranda/.opam/4.12.1'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='/Users/brandomiranda/.opam/4.12.1/lib/stublibs:/Users/brandomiranda/.opam/4.12.1/lib/ocaml/stublibs:/Users/brandomiranda/.opam/4.12.1/lib/ocaml'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='/Users/brandomiranda/.opam/4.12.1/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
PATH='/Users/brandomiranda/.opam/4.12.1/bin:/Users/brandomiranda/miniconda/envs/iit_synthesis/bin:/Users/brandomiranda/miniconda/condabin:/usr/local/bin:/Users/brandomiranda/.opam/4.12.1/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin'; export PATH;
so opam env outputs a bunch of paths that are needed for the toolchain to work...perhaps here toolchain == opam? Based on What is the use of eval `opam config env` or eval $(opam env) and their difference? (question for eval opam config env ) I am inferring that what it means is that these are the bash env variables needed to "activate the ocaml/opam environment". Concluded that because opam env outputs a bunch of env variables.
So running opam env by itself only prints the output to the terminal the output of the command opam env.
$ usually refers that the identifier is a variable. e.g. $x is the variable x while x in the terminal is literally the string x (or command x if it exists).
$(command) or command is command substitution. So doing $(command) runs the output (e.g. the script that would have been printed to the terminal after running command) of command. I assume it's called command substitution because we substitute command with the output of it and try to run it. e.g. it says: In computing, command substitution is a facility that allows a command to be run and its output to be pasted back on the command line as arguments to another command in the wikipedia article so yes. https://en.wikipedia.org/wiki/Command_substitution
opam env = opam-env - Prints appropriate shell variable assignments to stdout. Returns the bindings for the environment variables set in the current switch, e.g. PATH, in a format intended to be evaluated by a shell. https://opam.ocaml.org/doc/man/opam-env.html note that same opam man page says This is most usefully used as eval $(opam env) to have further shell commands be evaluated in the proper opam context.
Finally, to get all the pieces together, what is the difference between command substitution (e.g. $( ) or ) and evaluating an expression (e.g. eval string or eval $(cmd) ) (in bash mainly). Ref: https://unix.stackexchange.com/a/23116/55620. My understanding is that $( ) runs a command in a subshells and outputs the string so that it's string is an input to another command e.g. eval. So to my understanding you can think of $(cmd) as replacing that expression and pasting the string of it back to to the command line for another commadn e.g. eval. While eval will take in the string and evaluate the expression in it instead. So what eval $(cmd) does is first evaluate $(cmd) replace it with the output string of running cmd in a subshell and then feeding it to eval -- where eval will pretend that input is code and evaluate it, parse and run it. So $(...) is usually done to nest command inside other commands.
The only puzzling thing to me is why $(cmd) without a cmd2 (eg eval) in the front interprets the strings of $(cmd) as bash commands to run instead of just evaluting e.g.
(iit_synthesis) brandomiranda~ ❯ $(echo v=1)
zsh: command not found: v=1
but
(iit_synthesis) brandomiranda~ ❯ v=1
(iit_synthesis) brandomiranda~ ❯ echo $v
1
does evaluate my string v=1 and then run it.
Thus what eval $(opam env) does is command substitute $(opam env) i.e. run opam env in a subshell return the string it outputted and then given to eval to evaluate as bash code. In python eval would interpret the input string to it as literal python code to evaluate, parse, run here similarly it would do the same but assume its bash (is my guess)
Eval vs $( ) details:
Playground 1
(iit_synthesis) brandomiranda~ ❯ clear
(iit_synthesis) brandomiranda~ ❯ $(echo v=1)
zsh: command not found: v=1
(iit_synthesis) brandomiranda~ ❯ v=1
(iit_synthesis) brandomiranda~ ❯ echo $v
1
Playground 2
(iit_synthesis) brandomiranda~ ❯ echo vv=2
vv=2
(iit_synthesis) brandomiranda~ ❯ $(echo vv=2)
zsh: command not found: vv=2
(iit_synthesis) brandomiranda~ ❯ eval $(echo vv=2)
(iit_synthesis) brandomiranda~ ❯ echo $vv
2
I have tried a number of different ways to assign an array inside a RUN command within a Dockerfile. None of them seem to work. I am running on Ubuntu-Slim, with bash as my default shell.
I've tried this (second line below)
RUN addgroup --gid 1000 node \
&& NODE_BUILD_PACKAGES=("binutils-gold" "g++" "gcc" "gnupg" "libgcc-7-dev" "linux-headers-generic" "make" "python3" ) \
...
But it fails with /bin/sh: 1: Syntax error: "(" unexpected.
I also tried assigning it as an ENV variable, as in:
ENV NODE_BUILD_PACKAGES=("binutils-gold" "g++" "gcc" "gnupg" "libgcc-7-dev" "linux-headers-generic" "make" "python3" )
but that fails as well.
Assigning and using arrays in Bash is completely supported. Yet, it appears that I can't use that feature of Bash when running in a Dockerfile. Can someone confirm/deny that you can assign arrays variables inside of shell commands in Dockerfile RUN syntax (or ENV variable syntax)?
The POSIX shell specification does not have arrays. Even if you're using non-standard shells like GNU bash, environment variables are always simple strings and never hold arrays either.
The default shell in Docker is usually /bin/sh, which should conform to the POSIX spec, and not bash. Alpine-based images don't have bash at all unless you go out of your way to install it. I'd generally recommend trying to stick to the POSIX syntax whenever possible.
A typical Dockerfile is fairly straightforward; it doesn't have a lot of parts that get reused multiple times and most of the things you can specify in a Dockerfile you don't need to be user-configurable. So for a list of OS packages, for example, I'd just list them out in a RUN command and not bother trying to package them into a variable.
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
binutils-gold \
g++ \
gcc \
...
Other things I see in Stack Overflow questions that do not need to be parameterized include the container path (set it once as the WORKDIR and refer to . thereafter), the process's port (needs to be a fixed number for the second docker run -p part), and user IDs (can be overridden with docker run -u, and you don't usually want to build an image that can only run on one system).
WORKDIR /app # not an ENV or ARG
COPY . . # into the WORKDIR, do not need to repeat
RUN adduser node # with no specific uid
EXPOSE 3000 # a fixed port number
RUN mkdir /data # also use a fixed path for potential mount points
You can have array NODE_BUILD_PACKAGES in RUN if you define SHELL :
SHELL ["/bin/bash", "-c"]
EDIT
I want to pass the last git short hash into my React app build with the following command:
git log -1 --pretty=%h
Thus in my Dockerfile I want something such as:
ARG REACT_APP_GIT_SHORTHASH
RUN git clone https://token#github.com/paulywill/repo.git && cd repo && export REACT_APP_GIT_SHORTHASH=`git log -1 --pretty=%h`
ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
In my Github Actions build I'm getting the following:
Step 6/14 : ARG REACT_APP_GIT_SHORTHASH
---> Running in f45f530d5c76
Removing intermediate container f45f530d5c76
---> 87a91c010aaf
Step 7/14 : RUN git clone https://***#github.com/paulywill/repo.git && cd repo && export REACT_APP_GIT_SHORTHASH=$(git log -1 --pretty=%h)
---> Running in b8a8fa3cd703
Cloning into 'repo'...
Removing intermediate container b8a8fa3cd703
---> 5bbf3a76b928
Step 8/14 : ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
---> Running in f624f2e59dc6
Removing intermediate container f624f2e59dc6
---> d15c3c276062
Are these command even visible or able to pass values if they're different intermediate containers?
ROM ubuntu:20.04
ARG REACT_APP_GIT_SHORTHASH
RUN apt-get update -y
RUN apt-get install git -y
RUN git clone https://github.com/pooya-mohammadi/deep_utils.git
WORKDIR deep_utils
RUN REACT_APP_GIT_SHORTHASH=`git log -1 --pretty=%h`
ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
Change deep_utils with your repo name. I found the cd <directory> to be problematic.
As per a previous answer and thanks in part for #david-maze for pointing me in the right direction, I can easily grab the git short hash before the docker build.
.github/deploy.yml
...
- name: Set outputs
id: vars
run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
- name: Check outputs
run: echo ${{ steps.vars.outputs.sha_short }}
...
docker build --build-arg REACT_APP_GIT_SHORTHASH=${{ steps.vars.outputs.sha_short }} -t $ECR_REPOSITORY_CLIENT .
Dockerfile
FROM node:16-alpine
ARG VERSION
ENV VERSION $VERSION
ARG REACT_APP_GIT_SHORTHASH
ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
...
I have a database for my app and I want to create it in run time docker
I have a file CreateDB.sh and it creates all the tables and stored procedure that I want.
I tried this :
FROM mcr.microsoft.com/mssql/server
ENV ACCEPT_EULA=Y \
SA_PASSWORD=qwe123QWE
USER root
RUN mkdir /home/db
COPY ./db /home/db
RUN chmod +x /home/db/DbScriptLinux.sh
WORKDIR /home/db/
CMD ["/bin/bash", "/home/db/DbScriptLinux.sh"]
but it returns an error :
LoginTimeout
is there any way to run my script after all services (sql-server) start?
You can use an if statement, for example, RUN if [[ -z "$arg" ]] ; then echo Argument not provided ; else echo Argument is $arg ; fi
Another way would be to use command1 && command2 so if the command 1 is successfull, then command 2 would run afterwards.
Your last line CMD ["/bin/bash", "/home/db/DbScriptLinux.sh"] if this is to start the database every time you start the container, as your default command to run, then should be alright, otherwise it would be better to use the RUN command.
I very new to ksh script but I have 2 ksh scripts each of them calling sybase stored procedure via isql. The issue I'm seeing is that when I execute the first script the stored procedure runs fine but when I execute the second it fails with error of (isql -b -S value -U value -P value: not found). Here are code snipets
Values for $SERVER, $DBO_USER and $DBO_PASSWORD are set earlier in the script.
test1.ksh:
ISQL_CMD="isql -b -S ${SERVER} -U ${DBO_USER} -P ${DBO_PASSWORD}"
VAR=`${ISQL_CMD} << EOF
set nocount on
go
set proc_return_status off
go
declare #var_id int
,#rtnval int
exec #rtnval = DB_NAME..MY_STORED_PROC_1
#parameter1 = ${VAR_IN}
,#parameter_output = #var_id output
go
EOF`
This executes fine and I get value in VAR variable
test2.ksh (VAR variable gets passed in from test1.ksh):
ISQL_CMD="isql -b -S ${DSQUERY} -U ${DBO_USER} -P ${DBO_PASSWORD}"
RETURN_VALUE=`${ISQL_CMD} << EOF
set nocount on
go
declare #rtnval int
exec #rtnval = DB_NAME..MY_STORED_PROC_2
#var_id = '${VAR}'
go
EOF`
I get the following error:
isql -b -S value -U value -P value: not found
These scripts can be run independent of each other so there is no guarantee that isql may have been called before test2.ksh and that is why I set the the ISQL_CMD variable in each script.
test1.ksh runs properly but test2.ksh does not, whether called from test1.ksh or run on it's own. Tried running the scripts in debug, but it didn't really provide any further information.
Ok I have figured this out after a full day of head scratching. test2.ksh is performing some file processing and setting the IFS value several lines prior to isql command. I had to reset or unset the IFS once I was done and isql command worked fine! the command to unset the IFS value is:
unset IFS
Thanks for those that were trying to help!!