when I try to create the database with the command:
symfony console doctrine:database:create
i get this error :
SQLSTATE[08006] [7] FATAL: could not open file "global/pg_filenode.map": Permission denied
I'm trying to create a database in docker container postgres
Thanks for your help
docker-compose.yml :
version: '3'
services:
###> doctrine/doctrine-bundle ###
database:
image: postgres:${POSTGRES_VERSION:-13}-alpine
environment:
POSTGRES_DB: ${POSTGRES_DB:-app}
# You should definitely change the password in production
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ChangeMe}
POSTGRES_USER: ${POSTGRES_USER:-symfony}
volumes:
- db-data:/var/lib/postgresql/data:rw
# You may use a bind-mounted host directory instead, so that it is harder to accidentally remove the volume and lose all your data!
# - ./docker/db/data:/var/lib/postgresql/data:rw
###< doctrine/doctrine-bundle ###
volumes:
###> doctrine/doctrine-bundle ###
db-data:
###< doctrine/doctrine-bundle ###
it's good, I found !
I had a docker volume with same name.
After erasing the old volume, it worked !
Related
I am trying to dockerize a react app with postgres database.
I am new to docker, so I followed tutorials online to come up with Dockerfile and docker-compose as shown below.
Dockerfile
# pull the official base image
FROM node:13.12.0-alpine
# set working direction
WORKDIR /app
# add `/app/node_modules/.bin` to $PATH
EXPOSE 1338
ENV PATH /app/node_modules/.bin:$PATH
# install application dependencies
COPY package.json ./
COPY package-lock.json ./
RUN npm i
# add app
COPY . ./
# start app
CMD ["npm", "start"]
docker-compose.yml
version: '3.7'
services:
sample:
container_name: sample
build:
context: .
dockerfile: ./Dockerfile
volumes:
- '.:/app'
- '/app/node_modules'
ports:
- 1338:1338
environment:
- CHOKIDAR_USEPOLLING=true
- ASPNETCORE_URLS=https://+:1338
- ASPNETCORE_HTTPS_PORT=1338
depends_on:
- db
db:
container_name: db
image: postgres:14-alpine
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_DB: ###
POSTGRES_USER: ###
POSTGRES_PASSWORD: ###
# I hide these information for privacy purpose, but I am 100% sure I input these information correctly.
volumes:
- ./db-data/:/var/lib/postgresql/data/
adminer:
image: adminer
restart: always
ports:
- 8080:8080
volumes:
pgdata1:
so what happened is when I tried to run docker-compose up , I suppose the db part had no issue, since it wrote database system is ready to accept connections. However, the "sample" part ended up with an error:
Server wasn't able to start properly.
error Error: connect ECONNREFUSED 127.0.0.1:5432
at TCPConnectWrap.afterConnect [as oncomplete]
which does not make much sense to me since the database is already up so there should not be any issue with connection at all.
Feel free to share your view, any idea would be appreciated. Thank you.
Problem
$ go run cmd/syndicate/main.go
2021/01/25 16:37:25 error connecting to database: dial tcp: lookup db: no such host
Unable to connect to database when attempting to run:
$ go run cmd/syndicate/main.go
2021/01/25 16:37:25 error connecting to database: dial tcp: lookup db: no such host
&
$ migrate -source file://migrations -database postgres://postgres:secret#db:5432/syndicate?sslmode=disable up
error: dial tcp: lookup db on [2001:558:feed::1]:53: no such host
What do these two commands have in common?... Database URL. I am nearly certain my database URL is incorrect.
I have verified my postgres container is running:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4e578bf646c7 adminer "entrypoint.sh docke…" 3 days ago Up 3 days 0.0.0.0:8080->8080/tcp syndicate_adminer_1
729fc179aa6f postgres "docker-entrypoint.s…" 3 days ago Up 3 days 5432/tcp syndicate_db_1
Here's where I might be overlooking something...
$ docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------
syndicate_adminer_1 entrypoint.sh docker-php-e ... Up 0.0.0.0:8080->8080/tcp
syndicate_db_1 docker-entrypoint.sh postgres Up 5432/tcp
5432/tcp???
I see that my adminer container is clearly mapped to my local port (0.0.0.0:8080->8080/tcp), however my postgres container is only showing 5432/tcp (and not 0.0.0.0:5432->5432/tcp)
I am new to docker.. Can anyone explain why my postgres port isn't associated with my local port?
Am I on the right track?
Here's my docker-compose.yml:
version: "3.8"
services:
db:
image: postgres
environment:
POSTGRES_DB: $POSTGRES_DB
POSTGRES_USER: $POSTGRES_USER
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
migrate:
image: migrate/migrate
volumes:
- ./migrations:/migrations
depends_on:
- db
command: -source=file://migrations -database postgres://$POSTGRES_USER:$POSTGRES_PASSWORD#db:5432/$POSTGRES_DB?sslmode=disable up
adminer:
image: adminer
restart: always
ports:
- "8080:8080"
environment:
ADMINER_DEFAULT_SERVER: db
depends_on:
- db
PS. I tried adding port: "5432:5432" variable for db servicd
Browse my repository at this time in history
Thank you!
Connor
add to db service
ports:
- "5432:5432"
migrate -path D:/works/go/go-fiber-api-server/backend/platform/migrations -database "postgres://postgres:password#cgapp-postgres:5432/postgres?sslmode=disable" up
error: dial tcp: lookup cgapp-postgres: no such host
Then I have fixed this issue to change db-host name(cgapp-postgres) into "IP ADDRESS" or host.docker.internal.
migrate -path D:/works/go/go-fiber-api-server/backend/platform/migrations -database "postgres://postgres:password#100.100.100.100:5432/postgres?sslmode=disable" up
1/u create_init_tables (20.0987ms)
migrate -path D:/works/go/go-fiber-api-server/backend/platform/migrations -database "postgres://postgres:password#host.docker.internal:5432/postgres?sslmode=disable" up
1/u create_init_tables (20.0987ms)
"host.docker.internal" works.
I am trying to distribute a set of connected applications running in several linked containers that includes a mongo database that is required to:
be distributed containing some seed data;
allow users to add additional data.
Ideally the data will also be persisted in a linked data volume container.
I can get the data into the mongo container using a mongo base instance that doesn't mount any volumes (dockerhub image: psychemedia/mongo_nomount - this is essentially the base mongo Dockerfile without the VOLUME /data/db statement) and a Dockerfile config along the lines of:
ADD . /files
WORKDIR /files
RUN mkdir -p /data/db && mongod --fork --logpath=/tmp/mongodb.log && sleep 20 && \
mongoimport --db testdb --collection testcoll --type csv --headerline --file ./testdata.csv #&& mongod --shutdown
where ./testdata.csv is in the same directory (./mongo-with-data) as the Dockerfile.
My docker-compose config file includes the following:
mongo:
#image: mongo
build: ./mongo-with-data
ports:
- "27017:27017"
#Ideally we should be able to mount this against a host directory
#volumes:
# - ./db/mongo/:/data/db
#volumes_from:
# - devmongodata
#devmongodata:
# command: echo created
# image: busybox
# volumes:
# - /data/db
Whenever I try to mount a VOLUME it seems as if the original seeded data - which is stored in /data/db - is deleted. I guess that when a volume is mounted to /data/db it replaces whatever is there currently.
That said, the docker userguide suggests that: Volumes are initialized when a container is created. If the container’s base image contains data at the specified mount point, that existing data is copied into the new volume upon volume initialization? So I expected the data to persist if I placed the VOLUME command after the seeding RUN command?
So what am I doing wrong?
The long view is that I want to automate the build of several linked containers, and then distribute a Vagrantfile/docker-compose YAML file that will fire up a set of linked apps, that includes a pre-seeded mongo database with a (partially pre-populated) persistent data container.
I do this using another docker container whose only purpose is to seed mongo, then exit. I suspect this is the same idea as ebaxt's, but when I was looking for an answer to this, I just wanted to see a quick-and-dirty, yet straightforward, example. So here is mine:
docker-compose.yml
mongodb:
image: mongo
ports:
- "27017:27017"
mongo-seed:
build: ./mongo-seed
depends_on:
- mongodb
# my webserver which uses mongo (not shown in example)
webserver:
build: ./webserver
ports:
- "80:80"
depends_on:
- mongodb
mongo-seed/Dockerfile
FROM mongo
COPY init.json /init.json
CMD mongoimport --host mongodb --db reach-engine --collection MyDummyCollection --type json --file /init.json --jsonArray
mongo-seed/init.json
[
{
"name": "Joe Smith",
"email": "jsmith#gmail.com",
"age": 40,
"admin": false
},
{
"name": "Jen Ford",
"email": "jford#gmail.com",
"age": 45,
"admin": true
}
]
I have found useful to use Docker Custom Images and using volumes, instead of creating another container for seeding.
File Structure
.
├── docker-compose.yml
├── mongo
│ ├── data
│ ├── Dockerfile
│ └── init-db.d
│ └── seed.js
Every File location mentioned in Dockerfile/docker-compose.yml, is relative to location of docker-compose.yml
DOCKERFILE
FROM mongo:3.6
COPY ./init-db.d/seed.js /docker-entrypoint-initdb.d
docker-compose.yml
version: '3'
services:
db:
build: ./mongo
restart: always
volumes:
- ./mongo/data:/data/db #Helps to store MongoDB data in `./mongo/data`
environment:
MONGO_INITDB_ROOT_USERNAME: {{USERNAME}}
MONGO_INITDB_ROOT_PASSWORD: {{PWD}}
MONGO_INITDB_DATABASE: {{DBNAME}}
seed.js
// Since Seeding in Mongo is done in alphabetical order... It's is important to keep
// file names alphabetically ordered, if multiple files are to be run.
db.test.drop();
db.test.insertMany([
{
_id: 1,
name: 'Tensor',
age: 6
},
{
_id: 2,
name: 'Flow',
age: 10
}
])
docker-entrypoint-initdb.d can be used for creating different users and mongodb administration related stuffs, just create an alphabetical ordered named js-script to createUser etc...
For more details on how to customize MongoDB Docker service, read this
Also, it is good to keep your passwords and usernames secure from Public, DO NOT push credentials on public git, instead use Docker Secrets. Also read this Tutorial on Secrets
Do note, it is not necessary to go into docker-swarm mode to use secrets. Compose Files supports secrets as well. Check this
Secrets can also be used in MongoDB Docker Services
Current answer based on #Jeff Fairley answer and updated according to new Docker docs
docker-compose.yml
version: "3.5"
services:
mongo:
container_name: mongo_dev
image: mongo:latest
ports:
- 27017:27017
networks:
- dev
mongo_seed:
container_name: mongo_seed
build: .
networks:
- dev
depends_on:
- mongo
networks:
dev:
name: dev
driver: bridge
Dockerfile
FROM mongo:latest
COPY elements.json /elements.json
CMD mongoimport --host mongo --db mendeleev --collection elements --drop --file /elements.json --jsonArray
You probably need to rebuild current images.
You can use this image that provides docker container for many jobs ( import, export , dump )
Look at the example using docker-compose
You can use Mongo Seeding Docker image.
Why?
You have the Docker image ready to go
You are not tied to JSON files - JavaScript and TypeScript files are supported as well (including optional model validation with TypeScript)
Example usage with Docker Compose:
version: '3'
services:
database:
image: 'mongo:3.4.10'
ports:
- '27017:27017'
api:
build: ./api/
command: npm run dev
volumes:
- ./api/src/:/app/src/
ports:
- '3000:3000'
- '9229:9229'
links:
- database
depends_on:
- database
- data_import
environment:
- &dbName DB_NAME=dbname
- &dbPort DB_PORT=27017
- &dbHost DB_HOST=database
data_import:
image: 'pkosiec/mongo-seeding:3.0.0'
environment:
- DROP_DATABASE=true
- REPLACE_ID=true
- *dbName
- *dbPort
- *dbHost
volumes:
- ./data-import/dev/:/data-import/dev/
working_dir: /data-import/dev/data/
links:
- database
depends_on:
- database
Disclaimer: I am the author of this library.
Here is the working database seed mongodb docker compose use the below command to seed the database
Dockerfile
FROM mongo:3.6.21
COPY init.json /init.json
CMD mongoimport --uri mongodb://mongodb:27017/testdb --collection users --type json --file /init.json --jsonArray
docker-compose.yml
version: "3.7"
services:
mongodb:
container_name: mongodb
image: mongo:3.6.21
environment:
- MONGO_INITDB_DATABASE=testdb
volumes:
- ./data:/data/db
ports:
- "27017:27017"
mongo_seed:
build: ./db
depends_on:
- mongodb
To answer my own question:
simple YAML file to create simple mongo container linked to a data volume container, fired up by Vagrant docker compose.
in the Vagrantfile, code along the lines of:
config.vm.provision :shell, :inline => <<-SH
docker exec -it -d vagrant_mongo_1 mongoimport --db a5 --collection roads --type csv --headerline --file /files/AADF-data-minor-roads.csv
SH
to import the data.
Package the box.
Distribute the box.
For the user, a simple Vagrantfile to load the box and run a simple docker-compose YAML script to start the containers and mount the mongo db against the data volume container.
In golang-migrate's documentation, it is stated that you can run this command to run all the migrations in one folder.
docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
-path=/migrations/ -database postgres://localhost:5432/database up 2
How would you do this to fit the syntax of the new docker-compose, which discourages the use of --network?
And more importantly: How would you connect to a database in another container instead to one running in your localhost?
Adding this to your docker-compose.yml will do the trick:
db:
image: postgres
networks:
new:
aliases:
- database
environment:
POSTGRES_DB: mydbname
POSTGRES_USER: mydbuser
POSTGRES_PASSWORD: mydbpwd
ports:
- "5432"
migrate:
image: migrate/migrate
networks:
- new
volumes:
- .:/migrations
command: ["-path", "/migrations", "-database", "postgres://mydbuser:mydbpwd#database:5432/mydbname?sslmode=disable", "up", "3"]
links:
- db
networks:
new:
Instead of using the --network host option of docker run you set up a network called new. All the services inside that network gain access to each other through a defined alias (in the above example, you can access the db service through the database alias). Then, you can use that alias just like you would use localhost, that is, in place of an IP address. That explains this connection string:
"postgres://mydbuser:mydbpwd#database:5432/mydbname?sslmode=disable"
The answer provided by #Federico work for me at the beginning, nevertheless, I realised that I've been gotten a connect: connection refused the first time the docker-compose was run in brand new environment, but not the second one. This means that the migrate container runs before the Database is ready to process operations. Since, migrate/migrate from docker-hub runs the "migration" command whenever is ran, it's not possible to add a wait_for_it.sh script to wait for the db to be ready. So we have to add depends_on and a healthcheck tags to manage the order execution.
So this is my docker file:
version: '3.3'
services:
db:
image: postgres
networks:
new:
aliases:
- database
environment:
POSTGRES_DB: mydbname
POSTGRES_USER: mydbuser
POSTGRES_PASSWORD: mydbpwd
ports:
- "5432"
healthcheck:
test: pg_isready -U mydbuser -d mydbname
interval: 10s
timeout: 3s
retries: 5
migrate:
image: migrate/migrate
networks:
- new
volumes:
- .:/migrations
command: ["-path", "/migrations", "-database", "postgres://mydbuser:mydbpwd#database:5432/mydbname?sslmode=disable", "up", "3"]
links:
- db
depends_on:
- db
networks:
new:
As of Compose file formats version 2 you do not have to setup a network.
As stated in the docker networking documentation By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.
So in you case you could do something like:
version: '3.8'
services:
#note this databaseservice name is what we will use instead
#of localhost when using migrate as compose assigns
#the service name as host
#for example if we had another container in the same compose
#that wnated to access this service port 2000 we would have written
# databaseservicename:2000
databaseservicename:
image: postgres:13.3-alpine
restart: always
ports:
- "5432"
environment:
POSTGRES_PASSWORD: password
POSTGRES_USER: username
POSTGRES_DB: database
volumes:
- pgdata:/var/lib/postgresql/data
#if we had another container that wanted to access migrate container at let say
#port 1000
#and it's in the same compose file we would have written migrate:1000
migrate:
image: migrate/migrate
depends_on:
- databaseservicename
volumes:
- path/to/you/migration/folder/in/local/computer:/database
# here instead of localhost as the host we use databaseservicename as that is the name we gave to the postgres service
command:
[ "-path", "/database", "-database", "postgres://databaseusername:databasepassword#databaseservicename:5432/database?sslmode=disable", "up" ]
volumes:
pgdata:
I have noticed that when I try to run docker-compose up command for the first time I'm getting an error:
Starting mssql ...
Starting mssql ... done
Recreating api ...
Recreating api ... done
Attaching to mssql, api
api exited with code 1
Because api try to get data from DB but the MSSQL has not been started yet.
So, my question is it possible to somehow wait for DB and after that run API?
Here are my docker-compose and dockerfile
docker-compose:
version: '3.3'
services:
api:
image: api
container_name: api
build:
context: .
dockerfile: Dockerfile
ports:
- "8000:80"
depends_on:
- db
db:
image: "microsoft/mssql-server-linux"
container_name: mssql
environment:
SA_PASSWORD: "testtest3030!"
ACCEPT_EULA: "Y"
MSSQL_PID: "Express"
ports:
- "8001:1433"
dockerfile:
# Build Stage
FROM microsoft/aspnetcore-build as build-env
WORKDIR /source
COPY . .
RUN dotnet restore
RUN dotnet publish -o /publish --configuration Release
# Publish Stage
FROM microsoft/aspnetcore
WORKDIR /app
COPY --from=build-env /publish .
ENTRYPOINT ["dotnet", "Api.dll"]
I also noticed in logs:
2017-11-17 22:12:42.67 Logon Error: 18456, Severity: 14, State: 38.
2017-11-17 22:12:42.67 Logon Login failed for user 'sa'. Reason: Failed to open the explicitly specified database 'MyDb'. [CLIENT: 172.26.0.3]
You could use a simple entrypoint.sh script:
#!/bin/bash
set -e
run_cmd="dotnet your_app.dll"
sleep 10
exec $run_cmd
And docker file will change accordingly:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-bionic AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
COPY ["src/entrypoint.sh", ""]
RUN chmod +x entrypoint.sh
# .... here your copy/restore/build/publish
ENTRYPOINT [ "/bin/bash", "entrypoint.sh" ]