Currently, I have a docker-compose file for spinning up SQL Server database and it has a volume configured to persist the data on disk.
version: '3.4'
services:
sqlserver:
image: mcr.microsoft.com/mssql/server
container_name: MsSqlServer
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=Password_01
ports:
- "1433:1433"
volumes:
- /tmp/docker-volumes/db:/var/opt/mssql/data
Instead, I would like to have 2 docker images. One with SQL Server only and another one which would contain the data so that I could later regularly pull just the data image (not the whole server).
What is the best way to configure this? I believe it can be somehow configured using volumes_from attribute? Does somebody know how exactly? Is there any other way how to configure it, for docker compose v3?
Related
I created a docker compose file to run a spring-boot application (2.5.7) and a MS SQL Server:
version: '3.8'
services:
db:
ports:
- 1434:1433
build: ./db
api:
depends_on:
- db
build: ./mdm-web-api
restart: on-failure
env_file: ./.env
ports:
- $SPRING_LOCAL_PORT:$SPRING_DOCKER_PORT
environment:
- SPRING_DATASOURCE_URL=jdbc:sqlserver://db:1434;databaseName=dwh-demo;encrypt=true;logingTimeout=30
stdin_open: true
tty: true
I defined a Dockerfile for the mssql in the folder 'db' and successfully tested the SQL server alone.
I added my spring-boot application, which I also tested separately.
I have an application.properties file as follow (I replace sensible values):
logging.level.root=INFO
spring.datasource.username=<my-db-user>
spring.datasource.password=<my-ultra-strong-pw>
spring.datasource.driver-class-name=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.jpa.hibernate.ddl-auto=none
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.default_schema:entity_reference
spring.jpa.hibernate.use-new-id-generator-mappings=false
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
spring.data.rest.basePath=/api
springdoc.api-docs.path=/api-docs
I just added the spring.datasource.url properties in the docker-compose.yml to be able to point to the database server using the service name:
environment:
- SPRING_DATASOURCE_URL=jdbc:sqlserver://db:1434;databaseName=dwh-demo;encrypt=true;logingTimeout=30
It seems like the SPRING_DATASOURCE_URL value is completely ignored and I'm getting the following error:
Failed to determine suitable jdbc url
It shouldn't be a problem to use both places to add configuration, but maybe I'm wrong.
What could be the problem and which alternative do I have?
Additional details:
java openjdk 19
maven 3.8.5
Docker on Windows
SQL Server 2019-latest for Linux
I have a SQL Server running inside a Docker container. It is setup via docker-compose file and tied to my .NET Core API:
version: '3.4'
services:
appdb:
image: mcr.microsoft.com/mssql/server
container_name: ${SQL_SERVER_CONTAINER_NAME}
restart: unless-stopped
ports:
- "${SQL_SERVER_HOST_PORT}:1433"
environment:
SA_PASSWORD: ${SQL_SERVER_USER_PASSWORD}
ACCEPT_EULA: "Y"
volumes:
- ./../../.microsoft/data:/var/opt/mssql/data
- ./../../.microsoft/log:/var/opt/mssql/log
- ./../../.microsoft/secrets:/var/opt/mssql/secrets
demoapi.website:
image: ${DOCKER_REGISTRY-}demoapiwebsite
build:
context: .
dockerfile: DemoAPI.Website/Dockerfile
container_name: ${APP_NAME}-app
environment:
- ASPNETCORE_ENVIRONMENT=Development
- "ConnectionStrings__AppConnectionString=Server=${SQL_SERVER_CONTAINER_NAME},${SQL_SERVER_HOST_PORT}; Initial Catalog=${APP_NAME};User ID=${SQL_SERVER_USER_ID};Password=${SQL_SERVER_USER_PASSWORD}"
depends_on:
- appdb
ports:
- "8000:80"
I also have a bunch of SQL script files inside my .NET Core API project. These scripts basically define how to create and seed the database when the project is started.
What I would like to do is to execute these scripts somehow once the dockerized SQL Server is up and running. Is it possible to do this via the docker-compose file?
I see a lot of tutorials which use the actual .NET app and do this during the app start up with EF migrations, but I would prefer to get this done via docker-compose, if possible?
I have been researching how to connect multiple docker containers in the same compose file to a database (MySQL/MariaDB) on the local host. Currently, the database is containerized for development but production requires a separate database. Eventually, the database will be deployed to AWS or Azure.
There are lots of similar questions on SO, but none that seem to address this particular situation.
Given the existing docker-compose.yml
version: '3.1'
services:
db:
build:
image: mariadb:10.3
volumes:
- "~/data/lib/mysql:/var/lib/mysql:Z"
api:
image: t-api:latest
depends_on:
- db
web:
image: t-web:latest
scan:
image: t-scan:latest
proxy:
build:
context: .
dockerfile: nginx.Dockerfile
image: t-proxy
depends_on:
- web
ports:
- 80:80
All these services are reversed proxied behind nginx, with both api and scan services requiring access to the database. There are other services requiring database access not shown for simpliticy.
The production compose file would be:
version: '3.1'
api:
image: t-api:latest
depends_on:
- db
web:
image: t-web:latest
scan:
image: t-scan:latest
proxy:
build:
context: .
dockerfile: nginx.Dockerfile
image: t-proxy
depends_on:
- web
ports:
- 80:80
If there was a single container requiring database access, I could just open up the ports 3306:3306, which won't work for multiple containers.
Splitting up the containers breaks the reverse proxy and add's complexity to deployment and management. I've tried extra_hosts
extra_hosts:
- myhost: xx.xx.xx.xx
but this generate EAI_AGAIN DNS errors, which is strange because you can ping the host from inside containers. I realize this may not be possible
I can use Traefik for web sites since they use headers when they are connecting.
But I want to have multiple different instances of SQL Server running through docker which will be externally available (outside the docker host, potentially outside the local network)
So, is there anything which allows connecting to different sql server instances running on the same docker instance WITHOUT having to give them different ports or external ip addresses such that someone could access
sql01.docker.local,1433 AND sql02.docker.local,1433 from SQL Tools.
Start Additional Question
Since there has been no replies perhaps there is a way to have different instances like: sql.docker.local\instance1 and sql.docker.local\instance2 though I imagine that may also not be possible
End Additional Question
This is an example of the docker-compose file I was trying to use (before I realised that queries to sql server don't send through a host header - or am I wrong about that?)
version: '2.1'
services:
traefik:
container_name: traefik
image: stefanscherer/traefik-windows
command: --docker.endpoint=tcp://172.28.80.1:2375 --logLevel=DEBUG
ports:
- "8080:8080"
- "80:80"
- "1433:1433"
volumes:
- ./runtest:C:/etc/traefik
- C:/Users/mvukomanovic.admin/.docker:C:/etc/ssl
networks:
- default
restart: unless-stopped
labels:
- "traefik.enable=false"
whoami:
image: stefanscherer/whoami
labels:
- "traefik.backend=whoami"
- "traefik.frontend.entryPoints=http"
- "traefik.port=8080"
- "traefik.frontend.rule=Host:whoami.docker.local"
networks:
- default
restart: unless-stopped
sql01:
image: microsoft/mssql-server-windows-developer
environment:
- ACCEPT_EULA=Y
hostname: sql01
domainname: sql01.local
networks:
- default
restart: unless-stopped
labels:
- "traefik.frontend.rule=Host:sql01.docker.local,sql01,sql01.local"
- "traefik.frontend.entryPoints=mssql"
- "traefik.port=1433"
- "traefik.frontend.port=1433"
networks:
- default
restart: unless-stopped
sql02:
image: microsoft/mssql-server-windows-developer
environment:
- ACCEPT_EULA=Y
hostname: sql02
domainname: sql02.local
networks:
- default
restart: unless-stopped
labels:
- "traefik.frontend.rule=Host:sql02.docker.local,sql02,sql02.local"
- "traefik.frontend.entryPoints=mssql"
- "traefik.port=1433"
- "traefik.frontend.port=1433"
networks:
- default
restart: unless-stopped
networks:
default:
external:
name: nat
As mentionned earlier traefik is not the right solution since it's a HTTP only LoadBalancer.
I can think right now in 3 different ways to achieve what you want to do :
Use a TCP Load Balancer like HAproxy
Setup you server in Docker Swarm Mode (https://docs.docker.com/engine/swarm/), that will allow to bind the same port with a transparent routing between them
Use a service discovery service like consul and SRV records that can abstracts ports number (this might be overkill for your needs and complex to setup)
you can't use traefik, because it's a HTTP reverse proxy.
You're sql server listen and communicate via TCP.
I don't understand what's you're final goal.
Why are you using 2 differents sql-server ?
It depends on what's you want but you may have two solutions:
Can you use a simpler solution ? different databases, roles and permissions for separation.
You can search into the documentation of SQL Server Always On, but it doesn't seems easy to route queries to specific sever.
There is no "virtual" access to databases like for HTTP servers. So - no additional hostnames pointing to same IP can help you.
If you insist on port 1433 for all of your instances, then I see no way for you except to use two different external IPs.
If you were on a Linux box you may try some iptables magic, but it not elegant and would allow access to only one of your instances at any single moment. Windows may have iptables equivalent (I never heard of it) but still only-one-at-a-time you cannot escape.
My advice - use more than one port to expose your servers.
My Docker-Compose.yml:
version: "3"
services:
db:
image: microsoft/mssql-server-linux:2017-CU8
ports:
- 1433:1433
deploy:
mode: replicated
replicas: 1
environment:
- ACCEPT_EULA=Y
- MSSQL_SA_PASSWORD=SuperStrongSqlAdminPassword)(*£)($£)
volumes:
- /home/mssql/:/var/opt/mssql/
- /var/opt/mssql/data
As you can see I have a volume mapped to a directory on the host machine: /home/msqql:/var/opt/mssql/
If I do: docker stack deploy -c docker-compose.yml [stack name].
The server starts and I can see data is written to the hosts directory: /home/mssql/*.
I then connect to the server and create a database, tables and add some data.
If I then kill the stack using docker stack rm [stack name], or restart the host for maintenance reasons etc.
When SQL Server starts up again, although the /home/mssql/* still contains the files created by the server initially, if I connect to the server the database/tables/data is gone.
Do I have to re-attach the database when the server restarts somehow, or something else I'm missing maybe?
Thanks