I have a codebuild project with an environment variable named "GATEWAY_URI" set on aws console as plaintext.
I am using the following buildspec file;
version: 0.2
env:
variables:
AWS_REGION: "eu-central-1"
phases:
pre_build:
commands:
- echo logging in to ecr...
- >
aws ecr get-login-password --region $AWS_REGION \
| docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
- |
if expr "$CODEBUILD_WEBHOOK_TRIGGER" == "branch/main" >/dev/null && expr "$CODEBUILD_WEBHOOK_HEAD_REF" == "refs/heads/main" >/dev/null; then
DOCKER_TAG=prod
else
DOCKER_TAG=${CODEBUILD_RESOLVED_SOURCE_VERSION}
fi
- echo "Docker tag:" $DOCKER_TAG
- git_diff="$(git diff --name-only HEAD HEAD~1)"
- echo "$git_diff"
- buildWeb=false
- |
case $git_diff in
*Web*)
buildWeb=true
docker pull $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:builder || true
docker pull $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:$DOCKER_TAG || true
;;
esac
build:
commands:
- echo building images....
- |
if [ "$buildWeb" = true ]; then
docker build \
--target builder \
--cache-from $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:builder \
-f Web/Dockerfile.prod \
-t $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:builder \
--build-arg NODE_ENV=production \
--build-arg REACT_APP_GATEWAY_URI=$GATEWAY_URI \
./Web
docker build \
--cache-from $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:$DOCKER_TAG \
-f Web/Dockerfile.prod \
-t $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:$DOCKER_TAG \
./Web
fi
post_build:
commands:
- |
if [ "$buildWeb" = true ]; then
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:builder
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/web:$DOCKER_TAG
fi
- chmod +x ./deploy.sh
- bash deploy.sh enter code here
The "GATEWAY_URI" is reachable in buildspec I echo it before. I am passing it to docker build command as --build-arg and in docker file, I am setting it as env before "npm run build" so CRA will be able to catch it. I using the following multistage docker file
###########
# BUILDER #
###########
FROM public.ecr.aws/docker/library/node:17-alpine as builder
# set working directory
WORKDIR /usr/src/app
# add `/usr/src/app/node_modules/.bin` to $PATH
ENV PATH /usr/src/app/node_modules/.bin:$PATH
# install and cache app dependencies
COPY package.json .
COPY package-lock.json .
RUN npm ci
RUN npm install react-scripts#5.0.0 --silent
# set environment variables
ARG REACT_APP_GATEWAY_URI
ENV REACT_APP_GATEWAY_URI $REACT_APP_GATEWAY_URI
ARG NODE_ENV
ENV NODE_ENV $NODE_ENV
# create build
COPY . .
RUN npm run build
#########
# FINAL #
#########
FROM public.ecr.aws/docker/library/nginx:stable-alpine
# update nginx conf
RUN rm -rf /etc/nginx/conf.d
COPY conf /etc/nginx
# copy static files
COPY --from=builder /usr/src/app/build /usr/share/nginx/html
# expose port
EXPOSE 80
# run nginx
CMD ["nginx", "-g", "daemon off;"]
At the end of the day when the build passes, the "REACT_APP_GATEWAY_URI" variable under process.env is becoming an empty string.
Note: When I used the same dockerfile with docker-compose on my local system it's working fine and I can see "REACT_APP_GATEWAY_URI" on process.env. So my suspicion is that am I using the "- |" syntax right in the buildspec file. I mean, can I execute multiple docker build commands under it consecutively like that?
Edit: After adding the same --build-arg for "GATEWAY_URI" also to the docker build command for the FINAL stage, it is now working as expected. So, I think it was replaced with an empty string between the builder stage and the final stage. But I thought that I need that variable only on the build stage.
Related
Starting from my last commit yesterday I am encountering with a strange problem where GitLab CI fails continuously with an error as shown below:
$ ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no $SERVER_USER#$SERVER_IP "docker run -d -p 8010:80 --name my_project $TAG_LATEST"
docker: Error response from daemon: No command specified.
See 'docker run --help'.
Cleaning up project directory and file based variables
ERROR: Job failed: exit code 125
This is an React App which should be build and deployed to my server.
This is my Dockerfile
FROM node:16.14.0-alpine as build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM nginx:1.19-alpine
COPY --from=build /app/build /usr/share/nginx/html
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx/nginx.conf /etc/nginx/conf.d
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
In .gitlab-ci.yml file I have 2 stages build and deploy. For more clarity I would like to share it eaither:
stages:
- build
- deploy
variables:
TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
build_test:
image: docker:latest
stage: build
services:
- docker:19.03.0-dind
script:
- docker build -t $TAG_LATEST .
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
- docker push $TAG_LATEST
environment:
name: test
rules:
- if: $CI_COMMIT_BRANCH == "develop"
when: on_success
deploy_test:
image: alpine:latest
stage: deploy
only:
- develop
tags:
- frontend
script:
- chmod og= $SSH_PRIVATE_KEY
- apk update && apk add openssh-client
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no $SERVER_USER#$SERVER_IP "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY"
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no $SERVER_USER#$SERVER_IP "docker pull $TAG_LATEST"
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no $SERVER_USER#$SERVER_IP "docker container rm -f my_project || true"
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no $SERVER_USER#$SERVER_IP "docker run -d -p 8010:80 --name my_project $TAG_LATEST"
environment:
name: test
I did the following steps:
1st tried to run image using docker run command manually in my server. The result was the same!
2nd I pulled project from gitlab to the brand new folder and run first docker build -my_project . command then as I did it in server. It worked on my local machine.
3rd I re-verified my code base with ChatGPT and it found no error.
Currently there's an issue with latest docker release.
Use:
image: docker:20.10.22
instead of :
image: docker:latest
For more details check this link: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29593#note_1263383415
I got the same error since 3 days.
I also use gitLab CI.
I solved the issue by adding this line in my docker-compose.yml file (in the involved service section)
command: ["nginx", "-g", "daemon off;"]
I not yet find the root cause of the issue.
Hope that will help.
I have the React app. with 3 versions: for developement, testing and production.
They only differ in the URL that is used for the login (different WordPress site).
How do I make the react app agnostic/configurable at runtime
and save the need to generate 3 versions?
Just use
window.location.host // need to add http/s
to get the URL.
Many other parameters can be found using: URLSearchParams, see URLSearchParams
For those that use a Docker container, it can be done with environment variables.
My situation:
I made my react app in Visual Studio with template 'ASP.NET Core with React.js and Redux'. It is placed in a docker container which is deployed in Kubernetes.
It took me almost half a day but I managed to do it :)
First I found this post and especially the comment from Patrick Lee Scott is interesting:
https://levelup.gitconnected.com/handling-multiple-environments-in-react-with-docker-543762989783
Comment from Patrick Lee Scott:
https://patrickleet.medium.com/another-option-build-with-dummy-values-like-replace-api-url-and-then-use-an-entrypoint-sh-db053a799167
The comment is a good start but doesn't show the complete solution.
First I tested the script (and try to figure out what it was doing).
During the testing I found out that the 'cat /proc/self/environ' was not working, I replaced it with xargs -0 -L1 -a /proc/self/environ.
Second I had trouble getting the script to run via ENTRYPOINT, I figured out that the script needed to begin with: #!/bin/bash
Third, I added the original ENTRYPOINT at the bottom of the script.
Here is the modified script of Patrick Lee Scott:
appEntryPoint.sh:
#!/bin/bash
echo "Inserting env variables"
for file in ./ClientApp/build/static/js/*.js
do
echo "env sub for $file"
list="$(xargs -0 -L1 -a /proc/self/environ | awk -F= '{print $1}')"
echo "$list" | while read -r line; do
export REPLACE="REPLACE_$line"
export VALUE=$(eval "echo \"\$$line\"")
#echo "replacing ${REPLACE} with ${VALUE} in $file"
sed -i "s~${REPLACE}~${VALUE}~g" $file
unset REPLACE
unset VALUE
done
done
dotnet My.DotNet.ReactApp.dll
To make the answer complete, I will list here my Dockerfile:
Dockerfile:
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app/ClientApp
EXPOSE 80
EXPOSE 443
RUN echo "Acquire::Check-Valid-Until \"false\";\nAcquire::Check-Date \"false\";" | cat > /etc/apt/apt.conf.d/10no--check-valid-until && apt-get update -yq \
&& apt-get install -y curl \
&& apt-get install -y libpng-dev libjpeg-dev curl libxi6 build-essential libgl1-mesa-glx \
&& curl -sL https://deb.nodesource.com/setup_lts.x | bash - \
&& apt-get install -y nodejs
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
RUN echo "Acquire::Check-Valid-Until \"false\";\nAcquire::Check-Date \"false\";" | cat > /etc/apt/apt.conf.d/10no--check-valid-until && apt-get update -yq \
&& apt-get install -y curl \
&& apt-get install -y libpng-dev libjpeg-dev curl libxi6 build-essential libgl1-mesa-glx \
&& curl -sL https://deb.nodesource.com/setup_lts.x | bash - \
&& apt-get install -y nodejs
WORKDIR /app/ClientApp
COPY /My.DotNet.ReactApp/ClientApp/package*.json ./
RUN npm install --silent
COPY /My.DotNet.ReactApp/ClientApp ./
RUN npm run build
WORKDIR /app/publish/ClientApp
RUN cp -r /app/ClientApp/build .
WORKDIR /app
COPY /My.DotNet.ReactApp ./
RUN dotnet restore "My.DotNet.ReactApp.csproj"
RUN dotnet build "My.DotNet.ReactApp.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "My.DotNet.ReactApp.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
COPY ./appEntryPoint.sh ./
RUN chmod +x appEntryPoint.sh
ENTRYPOINT ["/app/appEntryPoint.sh"]
What you now have to do is put in your .env file placeholders:
.env.production
REACT_APP_API_ENDPOINT=REPLACE_REACT_APP_API_ENDPOINT
REACT_APP_API_SOME_OTHER_URL=REPLACE_REACT_APP_API_SOME_OTHER_URL
Now you can set the real values for the react variables as environment variables on the container, the script reads the environment variables from the container and will replace all values that begin with "REPLACE_"
So in this case we need to set these environment variables on the container used for production:
REACT_APP_API_ENDPOINT=https://prod.endpoint.com
REACT_APP_API_SOME_OTHER_URL=https://prod.url.com
And for the test environment:
REACT_APP_API_ENDPOINT=https://test.endpoint.com
REACT_APP_API_SOME_OTHER_URL=https://test.url.com
Use .env file. Check out this link for installation. At the end you will have such kind of structure in you app folder
I'm able to expose 2 services locally, on my computer with the following command docker run -d --name #containerName -e "PORT=8080" -p 8080:8080 -p 5000:5000 #imageName.
On port 5000, I expose my backend using a flask-restful api and on port 8080, I expose my frontend using nginx to serve my react.js application.
When I deploy that on Heroku platform I have 2 problems :
It seems Heroku try to bind Nginx on port 80 but the PORT env var is different, log's output :
Starting process with command /bin/sh -c gunicorn\ --bind\ 0.0.0.0:5000\ app:app\ --daemon\ \&\&\ sed\ -i\ -e\ \'s/\34352/\'\"\34352\"\'/g\'\ /etc/nginx/conf.d/default.conf\ \&\&\ nginx\ -g\ \'daemon\ off\;\'
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:1
[emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
How can I write the -p 8080:8080 -p 5000:5000 part inside the Dockerfile or hack around it since I can't specify the docker run [...] command on Heroku ?
I'm new to Docker and Nginx, so I would be very grateful if you know a better way to achieve my goal. Thanks in advance.
# ------------------------------------------------------------------------------
# Temporary image for react.js app using a multi-stage build
# ref : https://docs.docker.com/develop/develop-images/multistage-build/
# ------------------------------------------------------------------------------
FROM node:latest as build-react
# create a shared folder and define it as current dir
WORKDIR /usr/src/app
ENV PATH /usr/src/app/node_modules/.bin:$PATH
# copy the files required for node packages installation
COPY ./react-client/package.json ./
COPY ./react-client/yarn.lock ./
# install dependencies, copy code and build bundles
RUN yarn install
COPY ./react-client .
RUN yarn build
# ------------------------------------------------------------------------------
# Production image based on ubuntu:latest with nginx & python3
# ------------------------------------------------------------------------------
FROM ubuntu:latest as prod-react
WORKDIR /usr/src/app
# update, upgrade and install packages
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils
RUN apt-get upgrade -y
RUN apt-get install -y nginx curl python3 python3-distutils python3-apt
# install pip
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
RUN python3 get-pip.py
# copy flask-api requirements file and install modules
COPY ./flask-api/requirements.txt ./
RUN pip install -r requirements.txt
RUN pip install gunicorn
# copy flask code
COPY ./flask-api/app.py .
# copy built image and config onto nginx directory
COPY --from=build-react /usr/src/app/build /usr/share/nginx/html
COPY ./conf.nginx /etc/nginx/conf.d/default.conf
# ------------------------------------------------------------------------------
# Serve flask-api with gunicorn and react-client with nginx
# Ports :
# - 5000 is used for flask-api
# - 8080 is used by nginx to serve react-client
# You can change them but you'll have to change :
# - for flask-api : conf.nginx, axios calls (5000 -> #newApiPort)
# - for react-client : CORS origins (8080 -> #newClientPort)
#
# To build and run this :
# docker build -t #imageName .
# docker run -d --name #containerName -e "PORT=8080" -p 8080:8080 -p 5000:5000 #imageName
# ------------------------------------------------------------------------------
CMD gunicorn --bind 0.0.0.0:5000 app:app --daemon && \
sed -i -e 's/$PORT/'"$PORT"'/g' /etc/nginx/conf.d/default.conf && \
nginx -g 'daemon off;'
I am developing a python web server in Google App Engine.
I want to debug it in VScode so I want to get the Dockerfile for the latest python 3 version in the gcr.io/google-appengine/python
Where do I get it?
Here is the Dockerfile you can use:
FROM gcr.io/google-appengine/python
# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
# Use -p python3 or -p python3.7 to select python version. Default is version 2.
RUN virtualenv /env
# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
ADD requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# Add the application source code.
ADD . /app
WORKDIR /app
# Run a WSGI server to serve the application. gunicorn must be declared as
# a dependency in requirements.txt.
ENTRYPOINT ["gunicorn", "-b", ":8080", "server:app"]
You can also look at the Github Repository
This is the github repo of the Python Runtime for App Engine Flex, in that repository you can find the Dockerfile and all the Scripts to create an Docker container similar than the used on App Engine Flex
# The Google App Engine base image is debian (jessie) with ca-certificates
# installed.
# Source: https://github.com/GoogleCloudPlatform/debian-docker
FROM ${OS_BASE_IMAGE}
ADD resources /resources
ADD scripts /scripts
# Install Python, pip, and C dev libraries necessary to compile the most popular
# Python libraries.
RUN /scripts/install-apt-packages.sh
# Setup locale. This prevents Python 3 IO encoding issues.
ENV LANG C.UTF-8
# Make stdout/stderr unbuffered. This prevents delay between output and cloud
# logging collection.
ENV PYTHONUNBUFFERED 1
RUN wget https://storage.googleapis.com/python-interpreters/latest/interpreter-3.4.tar.gz && \
wget https://storage.googleapis.com/python-interpreters/latest/interpreter-3.5.tar.gz && \
wget https://storage.googleapis.com/python-interpreters/latest/interpreter-3.6.tar.gz && \
wget https://storage.googleapis.com/python-interpreters/latest/interpreter-3.7.tar.gz && \
tar -xzf interpreter-3.4.tar.gz && \
tar -xzf interpreter-3.5.tar.gz && \
tar -xzf interpreter-3.6.tar.gz && \
tar -xzf interpreter-3.7.tar.gz && \
rm interpreter-*.tar.gz
# Add Google-built interpreters to the path
ENV PATH /opt/python3.7/bin:/opt/python3.6/bin:/opt/python3.5/bin:/opt/python3.4/bin:$PATH
RUN update-alternatives --install /usr/local/bin/python3 python3 /opt/python3.7/bin/python3.7 50 && \
update-alternatives --install /usr/local/bin/pip3 pip3 /opt/python3.7/bin/pip3.7 50
# Upgrade pip (debian package version tends to run a few version behind) and
# install virtualenv system-wide.
RUN /usr/bin/pip install --upgrade -r /resources/requirements.txt && \
/opt/python3.4/bin/pip3.4 install --upgrade -r /resources/requirements.txt && \
rm -f /opt/python3.4/bin/pip /opt/python3.4/bin/pip3 && \
/opt/python3.5/bin/pip3.5 install --upgrade -r /resources/requirements.txt && \
rm -f /opt/python3.5/bin/pip /opt/python3.5/bin/pip3 && \
/opt/python3.6/bin/pip3.6 install --upgrade -r /resources/requirements.txt && \
rm -f /opt/python3.6/bin/pip /opt/python3.6/bin/pip3 && \
/opt/python3.7/bin/pip3.7 install --upgrade -r /resources/requirements.txt && \
rm -f /opt/python3.7/bin/pip /opt/python3.7/bin/pip3 && \
/usr/bin/pip install --upgrade -r /resources/requirements-virtualenv.txt
# Setup the app working directory
RUN ln -s /home/vmagent/app /app
WORKDIR /app
# Port 8080 is the port used by Google App Engine for serving HTTP traffic.
EXPOSE 8080
ENV PORT 8080
# The user's Dockerfile must specify an entrypoint with ENTRYPOINT or CMD.
CMD []
I'm trying to install api-platform with the docker images. The problem is that the containers will not start, exiting with code 127.
$ react-scripts start
/bin/sh: react-scripts: not found
error Command failed with exit code 127.
I'm using Docker for windows with Linux containers. I tried everything I could think of. The only thing I found in support was rebuilding the client and the admin but to no avail. Anybody an idea?
I use the provided Docker files from the api-plaform: https://github.com/api-platform/api-platform.
Dockerfile admin:
# https://docs.docker.com/develop/develop-images/multistage-build/#stop-at-a-specific-build-stage
# https://docs.docker.com/compose/compose-file/#target
# https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
ARG NODE_VERSION=13
ARG NGINX_VERSION=1.17
# "development" stage
FROM node:${NODE_VERSION}-alpine AS api_platform_admin_development
WORKDIR /usr/src/admin
# prevent the reinstallation of node modules at every changes in the source code
COPY package.json yarn.lock ./
RUN set -eux; \
apk add --no-cache --virtual .gyp \
g++ \
make \
python \
; \
yarn install; \
apk del .gyp
COPY . ./
VOLUME /usr/src/admin/node_modules
ENV HTTPS true
CMD ["yarn", "start"]
# "build" stage
# depends on the "development" stage above
FROM api_platform_admin_development AS api_platform_admin_build
ARG REACT_APP_API_ENTRYPOINT
RUN set -eux; \
yarn build
# "nginx" stage
# depends on the "build" stage above
FROM nginx:${NGINX_VERSION}-alpine AS api_platform_admin_nginx
COPY docker/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf
WORKDIR /usr/src/admin/build
COPY --from=api_platform_admin_build /usr/src/admin/build ./
Dockerfile: client
# https://docs.docker.com/develop/develop-images/multistage-build/#stop-at-a-specific-build-stage
# https://docs.docker.com/compose/compose-file/#target
# https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
ARG NODE_VERSION=13
ARG NGINX_VERSION=1.17
# "development" stage
FROM node:${NODE_VERSION}-alpine AS api_platform_client_development
WORKDIR /usr/src/client
RUN yarn global add #api-platform/client-generator
# prevent the reinstallation of node modules at every changes in the source code
COPY package.json yarn.lock ./
RUN set -eux; \
yarn install
COPY . ./
VOLUME /usr/src/client/node_modules
ENV HTTPS true
CMD ["yarn", "start"]
# "build" stage
# depends on the "development" stage above
FROM api_platform_client_development AS api_platform_client_build
ARG REACT_APP_API_ENTRYPOINT
RUN set -eux; \
yarn build
# "nginx" stage
# depends on the "build" stage above
FROM nginx:${NGINX_VERSION}-alpine AS api_platform_client_nginx
COPY docker/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf
WORKDIR /usr/src/client/build
COPY --from=api_platform_client_build /usr/src/client/build ./
I had the same problem.
I solved it by directly typing yarn install in both directory (/client and /admin). You can then start docker images (docker-compose up -d).
Hope it helps.