I have a performance problems deploying SQL Server on Docker. I write as docker-compose like this :
version: '3'
services:
mssql:
restart: always
container_name: mssql
image: mcr.microsoft.com/mssql/server:2017-latest
user: root
environment:
- SA_PASSWORD=PASSWORD-SA
- ACCEPT_EULA=Y
- MSSQL_PID=Developer
volumes:
- /store/backup:/var/opt/mssql/backup:rw
- /store/data:/var/opt/mssql/data:rw
- /store/log:/var/opt/mssql/log:rw
- /store/secrets:/var/opt/mssql/secrets:rw
logging:
driver: "json-file"
options:
max-file: "1"
max-size: "1m"
deploy:
resources:
limits:
cpus: 0.75
memory: 80G
Everything work well, but in running production query very slow.
Compare to select a table with same data row - result very difference:
DOCKER : need 1 minutes 48 seconds
GCP SQL : 27 Seconds - In GCP just use 8 Core an 12Gb memory.
Any suggestion about tuning SQL Server on Docker?
Thanks
Thanks for comments from Mr #AlwaysLearning and #AaronBertrand.
I change docker-compose on resources
resources:
limits:
cpus: 40
memory: 100G
After that i change max memory on TSQL with :
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'max server memory', 100000;
GO
RECONFIGURE;
GO
And everthing fast right now.
Related
Problem
$ go run cmd/syndicate/main.go
2021/01/25 16:37:25 error connecting to database: dial tcp: lookup db: no such host
Unable to connect to database when attempting to run:
$ go run cmd/syndicate/main.go
2021/01/25 16:37:25 error connecting to database: dial tcp: lookup db: no such host
&
$ migrate -source file://migrations -database postgres://postgres:secret#db:5432/syndicate?sslmode=disable up
error: dial tcp: lookup db on [2001:558:feed::1]:53: no such host
What do these two commands have in common?... Database URL. I am nearly certain my database URL is incorrect.
I have verified my postgres container is running:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4e578bf646c7 adminer "entrypoint.sh docke…" 3 days ago Up 3 days 0.0.0.0:8080->8080/tcp syndicate_adminer_1
729fc179aa6f postgres "docker-entrypoint.s…" 3 days ago Up 3 days 5432/tcp syndicate_db_1
Here's where I might be overlooking something...
$ docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------
syndicate_adminer_1 entrypoint.sh docker-php-e ... Up 0.0.0.0:8080->8080/tcp
syndicate_db_1 docker-entrypoint.sh postgres Up 5432/tcp
5432/tcp???
I see that my adminer container is clearly mapped to my local port (0.0.0.0:8080->8080/tcp), however my postgres container is only showing 5432/tcp (and not 0.0.0.0:5432->5432/tcp)
I am new to docker.. Can anyone explain why my postgres port isn't associated with my local port?
Am I on the right track?
Here's my docker-compose.yml:
version: "3.8"
services:
db:
image: postgres
environment:
POSTGRES_DB: $POSTGRES_DB
POSTGRES_USER: $POSTGRES_USER
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
migrate:
image: migrate/migrate
volumes:
- ./migrations:/migrations
depends_on:
- db
command: -source=file://migrations -database postgres://$POSTGRES_USER:$POSTGRES_PASSWORD#db:5432/$POSTGRES_DB?sslmode=disable up
adminer:
image: adminer
restart: always
ports:
- "8080:8080"
environment:
ADMINER_DEFAULT_SERVER: db
depends_on:
- db
PS. I tried adding port: "5432:5432" variable for db servicd
Browse my repository at this time in history
Thank you!
Connor
add to db service
ports:
- "5432:5432"
migrate -path D:/works/go/go-fiber-api-server/backend/platform/migrations -database "postgres://postgres:password#cgapp-postgres:5432/postgres?sslmode=disable" up
error: dial tcp: lookup cgapp-postgres: no such host
Then I have fixed this issue to change db-host name(cgapp-postgres) into "IP ADDRESS" or host.docker.internal.
migrate -path D:/works/go/go-fiber-api-server/backend/platform/migrations -database "postgres://postgres:password#100.100.100.100:5432/postgres?sslmode=disable" up
1/u create_init_tables (20.0987ms)
migrate -path D:/works/go/go-fiber-api-server/backend/platform/migrations -database "postgres://postgres:password#host.docker.internal:5432/postgres?sslmode=disable" up
1/u create_init_tables (20.0987ms)
"host.docker.internal" works.
I'm migrating a local SQL Server development database to run in a Linux docker container (on the same dev machine). When running my integration tests in Visual Studio 2019 on Windows, I receive MSDTC errors:
Exception thrown:
'System.Transactions.TransactionManagerCommunicationException' in
System.Data.dll An exception of type
'System.Transactions.TransactionManagerCommunicationException'
occurred in System.Data.dll but was not handled in user code
Communication with the underlying transaction manager has failed.
Here's my latest iteration of SQL Server in my docker-compose:
services:
sqlserver:
image: mcr.microsoft.com/mssql/server:2019-latest
container_name: SqlServer
restart: always
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=verySecretPassword
- MSSQL_RPC_PORT=13500
- MSSQL_DTC_TCP_PORT=51000
ports:
- "1401:1433"
- "135:13500"
- "51000:51000"
volumes:
- sqldata:/var/opt/mssql
I've tried all sorts of ways to adjust the RPC port to get this working. This is the main MS article. I've tried port 135:135 but it gives the same error. The note in the article at the bottom appears to be related to my issue.
For SQL Server outside of a container or for non-root containers, a
different ephemeral port, such as 13500, must be used in the container
and traffic to port 135 must then be routed to that port. You would
also need to configure port routing rules within the container from
the container port 135 to the ephemeral port.
Also, if you decide to map the container's port 135 to a different
port on the host, such as 13500, then you have to configure port
routing on the host. This enables the docker container to participate
in distributed transactions with the host and with other external
servers.
SQL Server 2019 containers run as a non-root user. I've tried port routing using netsh in windows... and also the MS article links to how to perform port forwarding in Ubuntu... which I'm unable to do even when logged in as root in the SQL Server container... iptables is not installed, and it doesn't let me apt-get install it?? I also updated the DTC options in windows to make it as open as possible, but it had no effect. Not sure what the secret sauce is. Hoping someone else has a similar setup that works.
Thanks for the tip on msdtc config, I got mine working with this compose:
version: '3.4'
services:
sqlserver:
image: mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04
container_name: sqlserver
user: root
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=[yourPwd]
- MSSQL_RPC_PORT=135
- MSSQL_DTC_TCP_PORT=51000
ports:
- "1433:1433"
- "135:135"
- "51000:51000"
volumes:
- D:\DockerVolumes\sqlserver:/var/opt/mssql/data
I have simple docker-compose.yml which contains two services only, my-api and sql-server.
version: '3.0'
services:
sql-server:
image: mcr.microsoft.com/mssql/server:2019-latest
hostname: sql-server
container_name: sql-server
ports:
- "1433:1433"
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=MyPassword01*
- MSSQL_PID=Express
my-api:
ports:
- "8080:5000"
depends_on:
- sql-server
... ommited for clarity
When I docker-compose up --build the containers are ready (I can verify with docker ps)
a7a47b89a17a mcr.microsoft.com/mssql/server:2019-latest "/opt/mssql/bin/perm…" 12 minutes ago Up 11 minutes 0.0.0.0:1433->1433/tcp sql-server
but I cannot access my SQL Server using SSMS.
SSMS login window:
Server Name: localhost,1433
Authentication: SQL Server Authentication
Username: sa
Password: MyPassword01*
Error:
Cannot connect to localhost,1433.
Login failed for user 'sa'. (.Net SqlClient Data Provider)
PS: I also tried with
Server Name: sql-server,1433
but still cannot access
Execute the below code which will display the public ipaddress.
Instead of localhost use this ipaddress
docker inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}" mssqltrek-con1.
I followed the below link to achieve the same.
https://www.sqlshack.com/sql-server-with-a-docker-container-on-windows-server-2016/
I guess you have to debug further. The first thing I guess you can do is to open the bash inside the container and try to connect to your SQL database from the container.
docker exec -it sql-server "bash"
Once inside the container bash, then connect with sqlcmd,
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "<YourNewStrong#Passw0rd>"
If you fail to connect inside the container, then I have to assume your SA password is somehow different when you set up the SQL server. But if this is not an issue, that means, you can connect to SQL server inside the container, then you can be rest assured it is something wrong with SSMS on port 1433 in your computer. Make sure the server name is correct.
I am attempting to make a secured repo for our internal docker registry. Github has a ready to go docker-compose however it is using MariaDB and Postgres as highlighted below.
What would be the best practice to utilize the same informix container to run 2 databases for the frontend and backend support of Portus & Docker Registry.
I feel I have to post the entire docker-compose yaml for context. I am also not clear on if i really need Clair for anything.
I am running this on a Open SUSE Leap 15 system. Thank you!
I have been messing around with this and as its written the registry and portus will not connect for some reason, but the underlining Databases seem to work fine and those are a bigger concern at this moment.
version: '2'
services:
portus:
build: .
image: opensuse/portus:development
command: bundle exec rails runner /srv/Portus/examples/development/compose/init.rb
environment:
- PORTUS_MACHINE_FQDN_VALUE=${MACHINE_FQDN}
- PORTUS_PUMA_HOST=0.0.0.0:3000
- PORTUS_CHECK_SSL_USAGE_ENABLED=false
- PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
- CCONFIG_PREFIX=PORTUS
- PORTUS_DB_HOST=db
- PORTUS_DB_PASSWORD=portus
- PORTUS_DB_POOL=5
- RAILS_SERVE_STATIC_FILES=true
ports:
- 3000:3000
depends_on:
- db
links:
- db
volumes:
- .:/srv/Portus
background:
image: opensuse/portus:development
entrypoint: bundle exec rails runner /srv/Portus/bin/background.rb
depends_on:
- portus
- db
environment:
- PORTUS_MACHINE_FQDN_VALUE=${MACHINE_FQDN}
- PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
# Theoretically not needed, but cconfig's been buggy on this...
- CCONFIG_PREFIX=PORTUS
- PORTUS_DB_HOST=db
- PORTUS_DB_PASSWORD=portus
- PORTUS_DB_POOL=5
volumes:
- .:/srv/Portus
links:
- db
webpack:
image: kkarczmarczyk/node-yarn:latest
command: bash /srv/Portus/examples/development/compose/bootstrap-webpack
working_dir: /srv/Portus
volumes:
- .:/srv/Portus
clair:
image: quay.io/coreos/clair:v2.0.2
restart: unless-stopped
depends_on:
- postgres
links:
- postgres
ports:
- "6060-6061:6060-6061"
volumes:
- /tmp:/tmp
- ./examples/compose/clair/clair.yml:/clair.yml
command: [-config, /clair.yml]
**db:
image: library/mariadb:10.0.23
command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
environment:
MYSQL_ROOT_PASSWORD: portus**
**postgres:
image: library/postgres:10-alpine
environment:
POSTGRES_PASSWORD: portus**
registry:
image: library/registry:2.6
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /registry_data
REGISTRY_STORAGE_DELETE_ENABLED: "true"
REGISTRY_HTTP_ADDR: 0.0.0.0:5000
REGISTRY_HTTP_DEBUG_ADDR: 0.0.0.0:5001
REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /etc/docker/registry/portus.crt
REGISTRY_AUTH_TOKEN_REALM: http://${MACHINE_FQDN}:3000/v2/token
REGISTRY_AUTH_TOKEN_SERVICE: ${MACHINE_FQDN}:${REGISTRY_PORT}
REGISTRY_AUTH_TOKEN_ISSUER: ${MACHINE_FQDN}
REGISTRY_NOTIFICATIONS_ENDPOINTS: >
- name: portus
url: http://${MACHINE_FQDN}:3000/v2/webhooks/events
timeout: 2000ms
threshold: 5
backoff: 1s
volumes:
- /registry_data
- ./examples/development/compose/portus.crt:/etc/docker/registry/portus.crt:ro
ports:
- ${REGISTRY_PORT}:5000
- 5001:5001
links:
- portus
The databases seem to run fine but I am still what i would consider a novice with docker-compose and informix on the setup side.
Any pointers or documentations recommendations would be most helpful as well.
unfortunately, Portus does not support informix DB. see this link
My Docker-Compose.yml:
version: "3"
services:
db:
image: microsoft/mssql-server-linux:2017-CU8
ports:
- 1433:1433
deploy:
mode: replicated
replicas: 1
environment:
- ACCEPT_EULA=Y
- MSSQL_SA_PASSWORD=SuperStrongSqlAdminPassword)(*£)($£)
volumes:
- /home/mssql/:/var/opt/mssql/
- /var/opt/mssql/data
As you can see I have a volume mapped to a directory on the host machine: /home/msqql:/var/opt/mssql/
If I do: docker stack deploy -c docker-compose.yml [stack name].
The server starts and I can see data is written to the hosts directory: /home/mssql/*.
I then connect to the server and create a database, tables and add some data.
If I then kill the stack using docker stack rm [stack name], or restart the host for maintenance reasons etc.
When SQL Server starts up again, although the /home/mssql/* still contains the files created by the server initially, if I connect to the server the database/tables/data is gone.
Do I have to re-attach the database when the server restarts somehow, or something else I'm missing maybe?
Thanks