How to pass arguments to Python script or read config files in Python scripts when using Flink? - apache-flink

For example, I'm submitting a python script my_driver.py to Flink by running bin/flink run --python my_driver.py. Can I pass any user-defined arguments to my_driver.py? Or can I open and read a config file in my_driver.py

Related

Copy a docker ARG into an Angularjs config file

I have a simple AngularJS application that is built through a Jenkins pipeline and a Docker file. When running the Jenkins job, the environment is set. Then it builds to one of two environments: dev or integration. What I need is a way to get that variable into the angular app.
The docker file uses the environment to build different config settings like:
ARG env
COPY build_config/${env} /opt/some/path...
I need to get that env into one of the controllers. Is there a way to copy env into a controller. I attempted something like the following:
COPY ${env} path/to/angular/file/controller
I have searched and tried different methods but cannot find a solution to work for the Jenkins with Docker pipeline.
You can just use RUN to write a string to a file:
RUN echo "$env" > path/to/angular/file/controller
If you want to append to the file instead of overwritting it, use
RUN echo "$env" >> path/to/angular/file/controller

Docker command not able to pass parameter at runtime to a appConfig.json file

Hi i am new to docker(version 19.03.8) and basically I have an angularjs project(dummyPoject) which contains appConfig.json file with the following path dummyPoject\src\assets\conf\appConfig.json. The json file contains the following variable:
{
"baseUrl": "MAPPED-URL"
}
Basically I want to override the MAPPED-URL properties with the one that i am sending while executing docker command.
Based on the online documentation I found out that it can be passed as environment variable while running the docker command please find below:
docker run -e baseUrl=http://localhost:8081/dummyUrl/ -p 8000:8080 -d --name cms test:1.0
I was expecting that MAPPED-URL will change to http://localhost:8081/dummyUrl/ but it is not the case.
Anything I am missing here please?
By adding -e baseUrl=http://localhost:8081/dummyUrl/ to docker run you have successfully added a environment variable to your docker container. But this value will not magically replace values in your appConfig.json file.
You will need some sort of script that extracts the baseUrl variable from environment and replaces the value in the script. This could be done using a bash script which runs when the container starts and replaces the line "baseUrl": "MAPPED-URL" using the environment variable you added.
Update:
This question inspired me to create a small Node.js package command line package that should help solve your issue. The package is called replace-env
You can add replace-env to your package.json dependencies. You can then run the command as part of your Dockerfile build process, or you can have it modify the file at runtime by customizing your CMD instruction.

Using Gitlab CI/CD to run command batch file

Using Gitlab CI/CD to run command batch file.
I have a batch file stored locally in my system. Now I want to run it from GitLab CI/CD YAML file.
I used the below format to run the YAML
Windows:
script:
- call: ci\CheckStatus.bat init
- call: ci\CheckStatus.bat build
tags:
- windows
But it is giving me below error
Found errors in your .gitlab-ci.yml:
jobs:windows:script config should be a string or an array containing strings and arrays of strings
How to resolve this and call my batch file to execute.
There is no call keyword under script so just enter executable commands as you would use them locally:
Windows:
script:
- ci\CheckStatus.bat init
- ci\CheckStatus.bat build
tags:
- windows

Search AWS CLI output and save to variable

I'm very new to batch files, but I'm trying to use one to automate some AWS CLI instance-creation.
What I have is pretty simple so far -- I have a command in my .bat file that will run the run-instances command:
aws ec2 run-instances --dry-run --image-id %ami_id% --key-name %keypair% --security-group-ids %security_group% --instance-type "r3.large" --subnet-id %az1b_subnet%
This command takes a little bit to run, but will eventually (without the --dry-run) return json about the created instance(s). I'd like to search that json output and save the instance-id to a variable so that I can use it to tag my newly created instance with the ec2 create-tags command.
Any thoughts on how I could do that? My first attempt was to add > test.txt to the end of the above command and then search through the json and set the variable. However, the test.txt is created instantly before the CLI command has finished and returned its output.
Thanks.
The AWS Command-Line Interface (CLI) has a --query parameter that can be used to specify the output fields. Combined with --output text, it can provide a list of Instance IDs.
Here's a script, assuming that only one instance is started per run-instances call (otherwise a loop would be required):
ID=`aws ec2 run-instances --image-id ami-xxxxxxxx --instance-type t1.micro --query 'Instances[0].InstanceId' --output text`
aws ec2 create-tags --resources $ID --tags Key=Name,Value=WebServer

Setting environment variables in Jenkins with a bat file?

Currently I have a batch file that sets all the environment variables needed before starting the build process.
It's a must to use the same bat for setting the env variables.
I tried to use EnvInject Plugin, didn't have any success.
Also tried "Execute Windows batch command" before running msbuild. e.g. start mybat.bat - this didn't seem to work either
How can I integrate the same bat file to set the variables?
Each Jenkins "build step" has it's own environment, I explained this in detail in this answer: Can not change Jenkins String Parameter Variable
If you are using the MSBuild plugin, it is its own build step, so using other build steps to change the environment is futile. If you are launching MSBuild through command line using "Execute Windows batch command", then just ran your bat file within the same build step, preceding the MSBuild command
In the case of MSBuild plugin, the only proper way is to use EnvInject plugin. Maybe you should try to figure out what isn't working for you with EnvInject plugin. From the example documentation, you want to be using "At job level" configuration, to populate your whole job with the variables from your .bat file.

Resources