The dockerfile builds locally but the GitLab pipeline fails, saying:
Step 3/8 : RUN gem install bundler rake
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the /usr/local/bundle directory.
The project strucutre is a Ruby Sinatra backend and a React frontend.
The Dockerfile looks like this
FROM ruby:3.0-alpine
# Install Dependencies
RUN apk update && apk add --no-cache build-base mysql-dev rrdtool
RUN gem install bundler rake
# Copy the files and build
WORKDIR /usr/server
COPY . .
RUN bundler install
# Run bundler
EXPOSE 443
CMD ["bundle", "exec", "puma"]
I thought Docker was meant to solve the problem of "it runs on my machine"...
What I've tried
As per this post, I tried adding -n /usr/local/bundle but it did not fix the issue.
Related
To get docker and yarn working on my corporate network, I needed to add a CA certificate to trust store (for docker) and set NODE_EXTRA_CA_CERTS for yarn (see here). The Dockerfile for my react application includes yarn install && yarn run build which gives a "self signed certificate in certificate chain" error. I am able to get around the error by running yarn install on my local machine before building in docker, remove yarn install from my Dockerfile and remove node_modules from my .dockerignore file.
How should I be resolving this error? Should I be transferring the .pem CA file to the Docker container and adding set NODE_EXTRA_CA_CERTS to the Dockerfile?
Dockerfile:
FROM node:15.13-alpine
WORKDIR /react
COPY . .
# RUN yarn config set cafile ./
RUN yarn install && yarn run build
.dockerignore:
node_modules
build
I had the same issue on my corporate network. What worked for me is copying the certificate into the image and allow the OS to recognize it by updating CA certificates.
I added this in my Dockerfile:
# Copy SSL certificates into the image
COPY *.crt /usr/local/share/ca-certificates/
# Update the certificate stores
RUN update-ca-certificates --verbose --fresh && \
npm config set cafile /usr/local/share/ca-certificates/my-custom-root-certificate.crt && \
yarn config set cafile /usr/local/share/ca-certificates/my-custom-root-certificate.crt
The *.crt files are in my docker build context (or same level as my Dockerfile)
hope you might have some guidance for me on this.
Right now I have a React app that is part of a Django app (for the sake of ease of passing auth login tokens), which is now containerised in a single Dockerfile. Everything works as intended when it is run as a Docker instance locally, but the Docker Image is having issues, despite the fact that the webpages are visible when the Image is deployed on server.
Specifically, when the Docker image is accessed, the home page renders as expected, but then a number of fetch requests which usually go to localhost:8000/<path>/<to>/<url> return the following error:
GET http://localhost:8000/<path>/<to>/<url> net::ERR_CONNECTION_REFUSED
On a colleague's suggestion, I have tried changing localhost:8000 to the public IP address of the server the Docker Image is hosted on (eg 172.XX.XX.XXX:8000) but when I rebuild the React app, these changes do not remain, and it defaults back to localhost. Here are my questions:
Is this something I change from within the React application itself? Do I need manually assign an IP address? (This seems unlikely to me)
Or is this something to do with either the Django port settings, or the Dockerfile itself?
Here is the Dockerfile
FROM ubuntu:18.04
# ...
RUN apt-get update && apt-get install -y \
software-properties-common
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get update && apt-get install -y \
python3.7 \
python3-pip
RUN python3.7 -m pip install pip
RUN apt-get update && apt-get install -y \
python3-distutils \
python3-setuptools
RUN python3.7 -m pip install pip --upgrade pip
# ???
ENV PYTHONUNBUFFERD 1
# copy file form local machine to container
COPY ./requirement.txt /requirement.txt
# install dependency
# RUN pip install -r /requirement.txt
RUN pip install -r /requirement.txt
# create app folder in container
RUN mkdir /app
# set default working dictionary
WORKDIR /app
# copy local app folder to container folder
COPY ./app /app
CMD ["python", "test.py"]
Multiple technologies, multiple failure points - thanks in advance!
I've build a React client application supported with a API written in Golang. I would like to use Docker to run these both apps using docker run.
I have the following project structure:
zid
|
|-web/ (my react folder)
main.go
Dockerfile
|
My goal is to run the main.go file in the zid folder and start the webapplication in the zid/web folder. The main.go file starts a API using Gin Gonic that will listen and serve on port 10000.
So I've tried the following:
# Build the Go API
FROM golang:latest as go_builder
RUN mkdir /zid
WORKDIR /zid
COPY . /zid
RUN GOOS=linux GOARCH=amd64 go build -a -ldflags "-linkmode external -extldflags '-static' -s -w" -o /go/bin/zid
# Build the React application
FROM node:alpine as node_builder
COPY --from=go_builder /zid/web ./
RUN npm install
RUN npm run build
# Final stage build, this will be the container with Go and React
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=go_builder /go/bin/zid /go/zid
COPY --from=go_builder /zid/ca /go/ca
COPY --from=node_builder /build ./web
EXPOSE 3000
WORKDIR /go
CMD ./zid
Next I did the following:
Build it with docker build -t zid . (no errors)
Run it with docker run -p 3000:3000 --rm zid
When I run this, it will startup the API, but when I go to http://localhost:3000/ then I get a Page does not work ERR: ERR_EMPTY_RESPONSE.
So the API starts up, but the npm build doens't. I am not sure what I am doing wrong, because the Docker container both contains the correct folders (go and web).
As you can see in the image it's all there I believe. What am I missing?
EDIT:
I am using the (*gin.Engine).Run() function to set the listen and serve on port 10000. In my local build my React application is sending request to localhost:10000. I always simply used npm start on the side of my React app (localhost:3000). My goal is to do the same but then all in one Dockerfile.
I am still a little unsure if I should EXPOSE ports 10000 & 3000 in my Dockerfile.
My HandleRequest function:
//Start the router and listen/serve.
func HandleRequests() {
router := SetupRouter()
router.Run(":10000")
}
My SetupRouter function:
//Setup the gin router
func SetupRouter() *gin.Engine {
router := gin.Default()
router.Use(CORSMiddleware())
router.POST("/auth/login", login)
router.POST("/component/deploy", deployComponent)
router.POST("/project/create", createProject)
router.POST("/diagram/create", createDiagram)
router.PATCH("/diagram/update", updateDiagram)
router.DELETE("/diagram/delete/:id", deleteDiagram)
router.GET("/diagram/:id", getDiagram)
router.GET("/project/list", getProjectsByUsername)
router.GET("/project/:id", getProject)
router.GET("/project/diagrams/:id", getDiagramsOfProject)
router.DELETE("/project/delete/:id", deleteProject)
router.GET("/application/list", applicationList)
router.GET("/instance/status/:id", getInstanceStatus)
router.GET("/user", getUser)
return router
}
Btw I just want to use the Docker container for Development and learning purpose only.
I've used the following multi-stage Docker build to create:
static VueJS UI HTML assets
compiled Go API http server (serving the above HTML assets)
Note: both Go and VueJS source is download from one git repo - but you could just as easily modify this to copy the two code-bases from local development directories.
#
# go build
#
FROM golang:1.16.5 AS go-build
#
# here we pull pkg source directly from git (and all it's dependencies)
#
RUN go get github.com/me/vue-go/rest
WORKDIR /go/src/github.com/me/vue-go/rest
RUN CGO_ENABLED=0 go build
#
# node build
#
FROM node:13.12.0 AS node-build
WORKDIR /app/vue-go
COPY --from=go-build go/src/github.com/me/vue-go/vue-go ./
# produces static html 'dist' here:
#
# /app/vue-go/dist
#
RUN npm i && npm run build
#
# final layer: include just go-binary and static html 'dist'
#
FROM scratch
COPY --from=go-build \
/go/src/github.com/me/vue-go/rest/rest \
/app/vue-go
COPY --from=node-build \
app/vue-go/dist \
/app/dist/
CMD ["/app/vue-go"]
I don't use Gin - but to use native net/http fileserver serving APIs and static HTML assets, use something like:
h := http.NewServeMux()
// serve static HTML directory:
if conf.StaticDir != "" {
log.Printf("serving on '/' static files from %q", conf.StaticDir)
h.Handle(
"/",
http.StripPrefix(
"/",
http.FileServer(
http.Dir(conf.StaticDir), // e.g. "../vue-go/dist" vue.js's html/css/js build directory
),
),
)
}
// handle API route(s)
h.Handle("/users",
authHandler(
http.HandlerFunc(handleUsers),
),
)
and start the service:
s := &http.Server{
Addr: ":3000", // external-facing IP/port
Handler: h,
}
log.Fatal(s.ListenAndServe())
then to build & run:
docker build -t zid .
docker run -p 3000:3000 --rm zid
I've found a solution! I've created a script on basis of multi-service container and then a run this script in my Dockerfile.
my script (start.sh):
#!/bin/sh
# Start the first process
./zid &
ZID_PID=$!
# Start the second process
cd /web
npm start &
WEB_PID=$!
# Naive check runs checks once a minute to see if either of the processes exited.
# This illustrates part of the heavy lifting you need to do if you want to run
# more than one service in a container. The container exits with an error
# if it detects that either of the processes has exited.
# Otherwise it loops forever, waking up every 60 seconds
while sleep 60; do
ps -fp $ZID_PID
ZID_PROCESS_STATUS=$?
if [ $ZID_PROCESS_STATUS -ne 0 ]; then
echo "ZID process has already exited."
exit 1
fi
ps -fp $WEB_PID
WEB_PROCESS_STATUS=$?
if [ $WEB_PROCESS_STATUS -ne 0 ]; then
echo "WEB process has already exited."
exit 1
fi
done
Here I first start my go executable and then I do a npm start
In my Dockerfile I do the following:
# Build the Go API
FROM golang:latest as go_builder
RUN mkdir /zid
WORKDIR /zid
COPY . /zid
RUN GOOS=linux GOARCH=amd64 go build -a -ldflags "-linkmode external -extldflags '-static' -s -w" -o /go/bin/zid
# Build the React application
FROM node:alpine as node_builder
COPY --from=go_builder /zid/web ./web
WORKDIR /web
RUN npm install
# Final stage build, this will be the container with Go and React
FROM node:alpine
RUN apk --no-cache add ca-certificates procps
COPY --from=go_builder /go/bin/zid /go/zid
COPY --from=go_builder /zid/static /go/static
COPY --from=go_builder /zid/ca /go/ca
COPY --from=node_builder /web /web
COPY --from=go_builder /zid/start.sh /go/start.sh
RUN chmod +x /go/start.sh
EXPOSE 3000 10000
WORKDIR /go
CMD ./start.sh
Here I am creating a Go executable, copy and npm install my /web folder and in de final stage build I start my ./start.sh script.
This will start my Golang application and the React development server. I hope it helps for others.
I'm trying to build an image for my React app. It's a pretty simply create-react-app setup. I'm aware that there are many questions regarding this topic, but the distinction here is that I am trying to deploy to Heroku and, because of Heroku not supporting EXPOSE, the setup is a little different.
I've managed to get my frontend up and running, but I'm having issues with my Express portion. Here is my Dockerfile.
FROM node:14.1-alpine AS builder
WORKDIR /opt/web
COPY package.json ./
RUN npm install
ENV PATH="./node_modules/.bin:$PATH"
COPY . ./
RUN npm run build
FROM nginx:1.17-alpine
RUN apk --no-cache add curl
RUN curl -L https://github.com/a8m/envsubst/releases/download/v1.1.0/envsubst-`uname -s`-`uname -m` -o envsubst && \
chmod +x envsubst && \
mv envsubst /usr/local/bin
COPY ./nginx/nginx.conf /etc/nginx/nginx.template
CMD ["/bin/sh", "-c", "envsubst < /etc/nginx/nginx.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"]
COPY --from=builder /opt/web/build /usr/share/nginx/html
It's pretty straightforward, but I'm not sure how to serve my server.js file up as an API.
I've tried many online tutorials to get nginx up and running with React and Express, but it either doesn't work with my current setup (locally) or it fails building on Heroku.
I've created a reproducible repo here. Not sure where to go from here.
PROBLEM
Hey, I have not used Docker much - I am trying to run my Jest tests through the Dockerfile. However, I'm getting this error when trying to build image:
ERROR
Step 13/16 : RUN if [ "$runTests" = "True" ]; then RUN npm test; fi
---> Running in ccdb3f89fb79
/bin/sh: RUN: not found
Dockerfile
FROM node:10-alpine as builder
ARG TOKEN
WORKDIR /app
ARG runTests
COPY .npmrc-pipeline .npmrc
COPY package*.json ./
RUN npm install
COPY . .
RUN rm -f .npmrc
ENV PORT=2000
ENV NODE_ENV=production
RUN if [ "$runTests" = "True" ]; then \
RUN npm test; fi
RUN npm run build
EXPOSE 2000
CMD ["npm", "start"]
The command I am using to build the image is this, and the idea is to be able to run the tests only when runTests=True.
docker build -t d-image --build-arg runTests="True" --build-arg "MY TOOOOOKEN"
Is this possible to do by just using the Dockerfile? Or is it necessary to use docker-compose as well?
The conditional statement seems to work good.
Not posssible to have two CMD commands
I have tried this as a workaround (but it did not work):
Dockerfile
FROM node:10-alpine as builder
ARG TOKEN
WORKDIR /app
ARG runTests
COPY .npmrc-pipeline .npmrc
COPY package*.json ./
RUN npm install
COPY . .
RUN rm -f .npmrc
ENV PORT=3000
ENV NODE_ENV=production
RUN npm run build
EXPOSE 3000
CMD if [ "$runTests" = "True" ]; then \
CMD ["npm", "test"] && ["npm", "start"] ;fi
Now I'm not getting any output from the test, but it looks to be successful.
PROGRESS
I have made some progress and the tests are actually running when I'm building the image. I also decided to use the RUN command for running the tests, so that they run on the build step.
Dockerfile:
FROM node:10-alpine as builder
ARG TOKEN
WORKDIR /app
COPY .npmrc-pipeline .npmrc
COPY package*.json ./
RUN npm install
COPY . .
RUN rm -f .npmrc
ENV PORT=3000
ENV NODE_ENV=production
RUN npm run build
RUN npm test
EXPOSE 3000
Error:
FAIL src/pages/errorpage/tests/accessroles.test.jsx
● Test suite failed to run
Jest encountered an unexpected token
This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.
By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".
Here's what you can do:
• If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/en/ecmascript-modules for how to enable it.
• To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
• If you need a custom transformation specify a "transform" option in your config.
• If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.
You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/en/configuration.html
Details:
It seems to me that the docker build process does not use the jest:{...} configurations in my package.json, even though it is copied and installed in the Dockerfile Any ideas?
RUN and CMD aren't commands, they're instructions to tell Docker what do when building your container. So e.g.:
RUN if [ "$runTests" = "True" ]; then \
RUN npm test; fi
doesn't make sense, RUN <command> runs a shell command but RUN isn't defined in the shell, it should just be:
ARG runTests # you need to define the argument too
RUN if [ "$runTests" = "True" ]; then \
npm test; fi
The cleaner way to do this is to set up npm as the entrypoint, and start as the specific command:
ENTRYPOINT [ "npm" ]
CMD [ "start" ]
This allows you to build the container normally, it doesn't require any build arguments, then run an NPM script other than start in the container, e.g. to run npm test:
docker run <image> test
However, note that this means all of the dev dependencies need to be in the container. It looks (from ENV NODE_ENV=production) like you intend this to be a production build, so you shouldn't be running the tests in the container at all. Also despite having as builder this isn't really a multi-stage build. The idiomatic script for this would be something like:
# stage 1: copy the source and build the app
FROM node:10-alpine as builder
ARG TOKEN
WORKDIR /app
COPY .npmrc-pipeline .npmrc
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
# stage 2: copy the output and run it in production
FROM node:10-alpine
WORKDIR /app
ENV PORT=3000
ENV NODE_ENV=production
COPY --from=builder /app/package*.json ./
RUN npm ci
COPY --from=builder /* your build output */
EXPOSE 3000
ENTRYPOINT [ "npm" ]
CMD [ "start" ]
See e.g. this Dockerfile I put together for a full-stack React/Express app.
A couple things on this:
2 commands are possible (but NOT recommended) , as I did here: https://hub.docker.com/repository/docker/djangofan/mountebank-with-ui-node
Also, I wouldn't recommend that your tests run while building your container image. Instead, build the container image so that it maps a folder to the location of your test files. Then, include your "temporary test image" in your compose file.
version: '3.4'
services:
service-api:
container_name: service-api
build:
context: .
dockerfile: Dockerfile-apibase
ports:
- "8083:8083"
e2e-tests:
container_name: e2e-tests
build:
context: .
dockerfile: Dockerfile-testbase
command: bash -c "wait-for-it.sh service-api:8083 && gradle -q clean test -Dorg.gradle.project.buildDir=/usr/src/example"
Then execute it like so to get a 0 or 1 exit code:
docker-compose up --exit-code-from e2e-tests
After running that, the service will remain running but the tests will shutdown when finished.
Hopefully that makes sense even though the example I gave is not exactly like your situation. Here is a LINK to my example from above, which you can try yourself. It should work similarly for Jest tests.