I have a Dockerfile to create a dev enviroment to develop a sailsJS app.
I just mount my source code into the container. I make my Git commit on my host machine but i would like to execute all my npm command in the container.
I have the following Dockerfile and i am running Docker (1.4.1) in ubuntu 14.10:
FROM ubuntu:14.04
### Utils ###
RUN apt-get update
RUN apt-get -y install build-essential git wget tar vim supervisor
### MongoDB ###
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | tee /etc/apt/sources.list.d/mongodb.list
RUN apt-get update
RUN apt-get install -y mongodb-org
RUN mkdir -p /data/db
### NodeJS ###
WORKDIR /tmp
RUN wget -O node http://nodejs.org/dist/v0.10.33/node-v0.10.33-linux-x64.tar.gz
RUN tar xf node
RUN mv node-v0.10.33-linux-x64 /usr/local/node
RUN ln -s /usr/local/node/bin/* /usr/local/bin
### Supervisord ###
RUN mkdir -p /var/log/supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
### Project ###
RUN npm install -g sails bower
WORKDIR /opt/sails
CMD ["/usr/bin/supervisord"]
EXPOSE 27017 1337
I run my container with the following command :
docker run -d -ti -p 1337:1337 -p 27017:27017 -v ~/dev/pinne:/opt/sails --name test-app loikg/sailsjs-mongo
The problem is that when I use command with npm inside the container that create files like sails genearet api I don't have the writing permission on them in the host machine.
How can i solve that ?
Users and Groups do not sync from host->container.
Your services in the container are running as root (UID:0 GID:0). Any files created by root in the container will need root access on the host.
One solution is to create a UID/GID inside the container that matches the UID/GID on the host. Then all your processes inside the container need to use that UID/GID so the files have the correct ownership/permissions.
Remember, it's UserID not user name. And GroupID not group name. The names need not match, only the numeric ID's.
It's kind of a pita. You will have to change your dockerfile to add the user, make sure your processes that create files are run with the correct uid, etc.
One of the workarounds is to use overlapping volumes, e.g.
... -v ~/dev/pinne:/opt/sails:ro -v /opt/sails/node_modules ...
would allow writing to /opt/sails/node_modules. The downside is that the changes will be lost upon the container termination (unless you copy the volumes data via --volumes-from). Another caveat AFAIR is that the path (~dev/pinne/node_modules -> /opt/sails/node_modules) should exist for this technique to work.
Related
I'm trying to get rid of Docker-In-Docker, therefore I'm replacing our Postgres images with new ones. For a use case we use a pre-filled Postgres image. The old workflow is to build the image, pull it in a pipeline and use Docker-In-Docker to fill it with data, then re-upload it to the Image registry again.
The new approach is to create the Postgres image with docker, and I've copied the .sql Dumps to /docker-entrypoint-initdb.d/. But this fills the image after the startup, I'd like to have a pre-filled image in the container registry because the filling takes up to 2 minutes.
This is my Dockerfile:
FROM postgres:11.12
LABEL maintainer="Hello Stackoverflow"
ARG POSTGRES_VERSION="11.12"
ARG TZ="Europe/Berlin"
ENV TZ ${TZ}
ENV LANG de_DE.UTF-8
ENV LANGUAGE de_DE.UTF-8
ENV LC_ALL de_DE.UTF-8
ENV POSTGRES_PASSWORD 'blabla'
ENV POSTGRES_HOST_AUTH_METHOD trust
RUN set -x && \
localedef -i de_DE -c -f UTF-8 -A /usr/share/locale/locale.alias de_DE.UTF-8
COPY test-data/. /docker-entrypoint-initdb.d/
CMD ["postgres"]
In the test-datafolder is a shell-script which executes the filling
#!/bin/sh
cd /docker-entrypoint-initdb.d
echo "read one.sql"
psql -v ON_ERROR_STOP=1 -U postgres < sql/one.sql
echo "read two.sql"
...
...
...
So the idea is to pre-fill the Postgres docker image with the schema and upload to the registry.
In theory you can rung postgres engine during docker build and execute whatever you need, here is not completely working example, i.e. postgres fails to start because there no configuration file.
if you spend more time on this i bet it should do the trick.
between your lines COPY test-data/. /docker-entrypoint-initdb.d/ and CMD ["postgres"] insert this
RUN adduser --disabled-password --gecos "" dbuser
RUN apt-get update
RUN apt-get install -y sudo
RUN echo "dbuser ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/dbuser && chmod 0440 /etc/sudoers.d/dbuser
USER dbuser:dbuser
RUN sudo chown -R dbuser:dbuser /docker-entrypoint-initdb.d
RUN sudo chown -R dbuser:dbuser /var/lib/postgresql/
RUN postgres
WORKDIR /docker-entrypoint-initdb.d
RUN psql -v ON_ERROR_STOP=1 -U postgres < sql/one.sql
at the moment that fails on RUN postgres - fails to find configuration on german, and i am not expert on postgres neither speak german so i wasn't able to solve right away.
also this part installs sudo and adds new dbuser into sudo group because postgress didn't want to start from root, so postgres runs from dbuser.
Hope this is going to help you moving into right direction :)
hope you might have some guidance for me on this.
Right now I have a React app that is part of a Django app (for the sake of ease of passing auth login tokens), which is now containerised in a single Dockerfile. Everything works as intended when it is run as a Docker instance locally, but the Docker Image is having issues, despite the fact that the webpages are visible when the Image is deployed on server.
Specifically, when the Docker image is accessed, the home page renders as expected, but then a number of fetch requests which usually go to localhost:8000/<path>/<to>/<url> return the following error:
GET http://localhost:8000/<path>/<to>/<url> net::ERR_CONNECTION_REFUSED
On a colleague's suggestion, I have tried changing localhost:8000 to the public IP address of the server the Docker Image is hosted on (eg 172.XX.XX.XXX:8000) but when I rebuild the React app, these changes do not remain, and it defaults back to localhost. Here are my questions:
Is this something I change from within the React application itself? Do I need manually assign an IP address? (This seems unlikely to me)
Or is this something to do with either the Django port settings, or the Dockerfile itself?
Here is the Dockerfile
FROM ubuntu:18.04
# ...
RUN apt-get update && apt-get install -y \
software-properties-common
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get update && apt-get install -y \
python3.7 \
python3-pip
RUN python3.7 -m pip install pip
RUN apt-get update && apt-get install -y \
python3-distutils \
python3-setuptools
RUN python3.7 -m pip install pip --upgrade pip
# ???
ENV PYTHONUNBUFFERD 1
# copy file form local machine to container
COPY ./requirement.txt /requirement.txt
# install dependency
# RUN pip install -r /requirement.txt
RUN pip install -r /requirement.txt
# create app folder in container
RUN mkdir /app
# set default working dictionary
WORKDIR /app
# copy local app folder to container folder
COPY ./app /app
CMD ["python", "test.py"]
Multiple technologies, multiple failure points - thanks in advance!
I have the React app. with 3 versions: for developement, testing and production.
They only differ in the URL that is used for the login (different WordPress site).
How do I make the react app agnostic/configurable at runtime
and save the need to generate 3 versions?
Just use
window.location.host // need to add http/s
to get the URL.
Many other parameters can be found using: URLSearchParams, see URLSearchParams
For those that use a Docker container, it can be done with environment variables.
My situation:
I made my react app in Visual Studio with template 'ASP.NET Core with React.js and Redux'. It is placed in a docker container which is deployed in Kubernetes.
It took me almost half a day but I managed to do it :)
First I found this post and especially the comment from Patrick Lee Scott is interesting:
https://levelup.gitconnected.com/handling-multiple-environments-in-react-with-docker-543762989783
Comment from Patrick Lee Scott:
https://patrickleet.medium.com/another-option-build-with-dummy-values-like-replace-api-url-and-then-use-an-entrypoint-sh-db053a799167
The comment is a good start but doesn't show the complete solution.
First I tested the script (and try to figure out what it was doing).
During the testing I found out that the 'cat /proc/self/environ' was not working, I replaced it with xargs -0 -L1 -a /proc/self/environ.
Second I had trouble getting the script to run via ENTRYPOINT, I figured out that the script needed to begin with: #!/bin/bash
Third, I added the original ENTRYPOINT at the bottom of the script.
Here is the modified script of Patrick Lee Scott:
appEntryPoint.sh:
#!/bin/bash
echo "Inserting env variables"
for file in ./ClientApp/build/static/js/*.js
do
echo "env sub for $file"
list="$(xargs -0 -L1 -a /proc/self/environ | awk -F= '{print $1}')"
echo "$list" | while read -r line; do
export REPLACE="REPLACE_$line"
export VALUE=$(eval "echo \"\$$line\"")
#echo "replacing ${REPLACE} with ${VALUE} in $file"
sed -i "s~${REPLACE}~${VALUE}~g" $file
unset REPLACE
unset VALUE
done
done
dotnet My.DotNet.ReactApp.dll
To make the answer complete, I will list here my Dockerfile:
Dockerfile:
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app/ClientApp
EXPOSE 80
EXPOSE 443
RUN echo "Acquire::Check-Valid-Until \"false\";\nAcquire::Check-Date \"false\";" | cat > /etc/apt/apt.conf.d/10no--check-valid-until && apt-get update -yq \
&& apt-get install -y curl \
&& apt-get install -y libpng-dev libjpeg-dev curl libxi6 build-essential libgl1-mesa-glx \
&& curl -sL https://deb.nodesource.com/setup_lts.x | bash - \
&& apt-get install -y nodejs
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
RUN echo "Acquire::Check-Valid-Until \"false\";\nAcquire::Check-Date \"false\";" | cat > /etc/apt/apt.conf.d/10no--check-valid-until && apt-get update -yq \
&& apt-get install -y curl \
&& apt-get install -y libpng-dev libjpeg-dev curl libxi6 build-essential libgl1-mesa-glx \
&& curl -sL https://deb.nodesource.com/setup_lts.x | bash - \
&& apt-get install -y nodejs
WORKDIR /app/ClientApp
COPY /My.DotNet.ReactApp/ClientApp/package*.json ./
RUN npm install --silent
COPY /My.DotNet.ReactApp/ClientApp ./
RUN npm run build
WORKDIR /app/publish/ClientApp
RUN cp -r /app/ClientApp/build .
WORKDIR /app
COPY /My.DotNet.ReactApp ./
RUN dotnet restore "My.DotNet.ReactApp.csproj"
RUN dotnet build "My.DotNet.ReactApp.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "My.DotNet.ReactApp.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
COPY ./appEntryPoint.sh ./
RUN chmod +x appEntryPoint.sh
ENTRYPOINT ["/app/appEntryPoint.sh"]
What you now have to do is put in your .env file placeholders:
.env.production
REACT_APP_API_ENDPOINT=REPLACE_REACT_APP_API_ENDPOINT
REACT_APP_API_SOME_OTHER_URL=REPLACE_REACT_APP_API_SOME_OTHER_URL
Now you can set the real values for the react variables as environment variables on the container, the script reads the environment variables from the container and will replace all values that begin with "REPLACE_"
So in this case we need to set these environment variables on the container used for production:
REACT_APP_API_ENDPOINT=https://prod.endpoint.com
REACT_APP_API_SOME_OTHER_URL=https://prod.url.com
And for the test environment:
REACT_APP_API_ENDPOINT=https://test.endpoint.com
REACT_APP_API_SOME_OTHER_URL=https://test.url.com
Use .env file. Check out this link for installation. At the end you will have such kind of structure in you app folder
I'd like to setup a docker container and share it with my co-workers so we all don't have to setup development environments individually.
Note that this does not include saving the container as an image and uploading to DockerHub (search around for that part). Its also worth noting that this set of instructions would be a good start for creating a "dockerfile" which would automatically run all of these commands to build up this container on-demand.
docker pull ubuntu
docker run --privileged=true -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:ro -v /dev/bus/usb:/dev/bus/usb ubuntu
// As root in the docker container:
apt-get update
apt-get install vim vifm ssh sshd iproute2 iputils-ping sshfs build-essential dos2unix git usbutils
adduser mydevuser
/etc/init.d/ssh start
ip a
// As mydevuser user in the docker container:
- Download and unzip gcc compilers from ARM (gcc-arm-none-eabi-8-2019-q3-update-linux.tar.bz2)
- Download and unzip nRF5SDK (nRF5SDK160098a08e2.zip)
- Download and unzip soft device (s113nrf52701.zip)
- Download and unzip command line tools (nRF-Command-Line-Tools_10_4_1_Linux-amd64.tar.gz)
// Configure for our compiler, here is my updated GNU_INSTALL_ROOT
~/nRF5SDK/components/toolchain/gcc ..) head Makefile.posix
GNU_INSTALL_ROOT ?= /home/mydevuser/gcc/gcc-arm-none-eabi-8-2019-q3-update/bin/
// Now lets compile some examples
~/nRF5SDK/external/micro-ecc/nrf52hf_armgcc/armgcc ..) make
~/nRF5SDK/examples/dfu/secure_bootloader/pca10100_s113_ble_debug/armgcc ..) make
~/nRF5SDK/examples/peripheral/spi/pca10056/blank/armgcc ..) make
// As root in the docker container:
mv /home/mydevuser/cli_nrf/mergehex /opt/
mv /home/mydevuser/cli_nrf/nrfjprog/ /opt/
ln -s /opt/nrfjprog/nrfjprog /usr/local/bin/nrfjprog
ln -s /opt/mergehex/mergehex /usr/local/bin/mergehex
cp -pv /home/mydevuser/cli_nrf/JLink_Linux_V650b_x86_64/libjlinkarm* /lib/x86_64-linux-gnu/
// As root in the docker container:
// Load the firmware over USB to the dev board:
nrfjprog -f NRF52 --program nrf52840_xxaa.hex --chiperase --log
We have an AngularJS application. We wrote a dockerfile for it so it's reusable on every system. The dockerfile isn't a best practice and it's maybe some weird build up (build and hosting in same file) for some but it's just created to run our angularjs app locally on each PC of every developer.
Dockerfile:
FROM nginx:1.10
... Steps to install nodejs-legacy + npm
RUN npm install -g gulp
RUN npm install
RUN gulp build
.. steps to move dist folder
We build our image with docker build -t myapp:latest .
Every developer is able to run our app with docker run -d -p 80:80 myapp:latest
But now we're developing other backends. So we have a backend in DEV, a backend in UAT, ...
So there are different URLS which we need to use in /config/xx.json
{
...
"service_base": "https://backend.test.xxx/",
...
}
We don't want to change that URL every time, rebuild the image and start it. We also don't want to declare some URLS (dev, uat, prod, ..) which can be used there. We want to perform our gulp build process with an environment variable instead of a hardcoded URL.
So we we can start our container like this:
docker run -d -p 80:80 --env URL=https://mybackendurl.com app:latest
Is there someone who has experience with this kind of issues? So we'll need an env variable in our json and building it and add the URL later on if that's possible.
EDIT : Better option is to use build args
Instead of passing URL at docker run command, you can use docker build args. It is better to have build related commands to be executed during docker build than docker run.
In your Dockerfile,
ARG URL
And then run
docker build --build-arg URL=<my-url> .
See this stackoverflow question for details
This was my 'solution'. I know it isn't the best docker approach but just for our developers it was a big help.
My dockerfile looks like this:
FROM nginx:1.10
RUN apt-get update && \
apt-get install -y curl
RUN sed -i "s/httpredir.debian.org/`curl -s -D - http://httpredir.debian.org/demo/debian/ | awk '/^Link:/ { print $2 }' | sed -e 's#<http://\(.*\)/debian/>;#\1#g'`/" /etc/apt/sources.list
RUN \
apt-get clean && \
apt-get update && \
apt-get install -y nodejs-legacy && \
apt-get install -y npm
WORKDIR /home/app
COPY . /home/app
RUN npm install -g gulp
RUN npm install
COPY start.sh /
CMD ["./start.sh"]
So after the whole include of the app + npm installation inside my nginx I start my container with the start.sh script.
The content of start.sh:
#!/bin/bash
sed -i 's#my-url#'"$DATA_ACCESS_URL"'#' configs/config.json
gulp build
rm -r /usr/share/nginx/html/
//cp right folders which are created by gulp build to /usr/share/nginx/html
...
//start nginx container
/usr/sbin/nginx -g "daemon off;"
So the build will happen when my container starts. Not the best way of course but it's all for the needs of the developers. Have an easy local frontend.
The sed command will perform a replace on the config file which contains something like:
{
"service_base": "my-url",
}
So my-url will be replaced by my the content of my environment variable which I willd define in my docker run command.
Than I'm able to perform.
docker run -d -p 80:80 -e DATA_ACCESS_URL=https://mybackendurl.com app:latest
And every developer can use the frontend locally and connect with their own backend URL.