Resize Docker Container of SQL Server - sql-server

I want to alleviate memory issues I'm having with my containerized SQL Server. I'm running this on Windows 10, v 1709.
For testing, I created a SQL Server container with this command:
docker run -d -p 1433:1433 --name sql1 -e SA_PASSWORD=1Secure*Password1
-e ACCEPT_EULA=Y microsoft/mssql-server-windows-developer:2017-latest
Within this containerized SQL instance I created a database that worked until it became too large. I'm now getting insufficient memory errors. For example, when I run a query in SSMS I receive "There is insufficient system memory in resource pool 'default' to run this query." Likewise, when I check the SQL Server ring-buffer, I see RESOURCE_MEMPHYSICAL_LOW. When I get these errors, only 8 gigs of my laptop's available 16 gigs of RAM are in use. So this is not an issue with real physical memory.
Docker for Windows, when on Windows 10, runs containers in a Hyper-V VM which defaults to 1GIG of RAM. I can confirm this by starting my container, and then entering an interactive mode with it:
PS C:\repos\somefolder> docker exec -it sql1 powershell
Within interactive mode, following this other SO example, I see these results:
PS C:\> systeminfo | findstr "Memory"
Total Physical Memory: 1,023 MB
Available Physical Memory: 221 MB
Virtual Memory: Max Size: 1,023 MB
Virtual Memory: Available: 82 MB
Virtual Memory: In Use: 941 MB
Also, when I run docker stats I can see that my sql1 container is using PRIV WORKING SET in the range of 750MiB to 970MiB.
Ok, so I've clearly confirmed that I've got a Hyper-V VM 1GB limit I need to raise.
How do I do this, without losing the database that is inside this container? I see an answer that says "use -m" option, but I think that only works when the container is first being created.
Docker Version
$ docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:12:48 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.24)
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:21:42 2018
OS/Arch: windows/amd64
Experimental: false

The SQL Server images allow you to attach databases in a JSON set as the attach_dbs environment variable.
What I've done is something like this:
# escape=`
FROM microsoft/mssql-server-windows-express:2017-GA
LABEL MAINTAINER="Seba Gómez #sebagomez"
# SQL Databases
COPY data\ c:/data
ENV attach_dbs="[{'dbName':'MyDB','dbFiles':['C:\\data\\MyDB.mdf','C:\\data\\MyDB_log.ldf']}, {'dbName':'MyOtherDB','dbFiles':['C:\\data\\MyOtherDB.mdf','C:\\data\\MyOtherDB_log.ldf']}]" `
ACCEPT_EULA=Y `
sa_password="dbPassword!"
All the mdf and ldf files are in a data folder right next to my dockerfile.

Related

Docker Compose - Entrypoint permission error (mariadb)

I have tried many things and searched for this it, but nobody seems to have my exact problem.
I installed docker on my Ubuntu 20.04 LTS using these steps and the test run docker run hello-world works perfect.
My docker-compose.yml looks like this:
version: '3.7'
services:
db:
image: mariadb
environment:
MYSQL_ROOT_PASSWORD: 42
volumes:
- type: bind
source: ./dbsetup
target: /docker-entrypoint-initdb.d
adminer:
image: adminer
ports:
- 8080:8080
depends_on:
- db
The docker service is running and mariadb/adminer images download without error.
When I try to use docker compose up (sudo or not) I get the following error:
Attaching to dbexercise3-adminer-1, dbexercise3-db-1
dbexercise3-db-1 | 2022-11-26 01:01:54+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.10.2+maria~ubu2204 started.
dbexercise3-db-1 | 2022-11-26 01:01:54+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
dbexercise3-db-1 | 2022-11-26 01:01:54+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.10.2+maria~ubu2204 started.
dbexercise3-db-1 | ls: cannot open directory '/docker-entrypoint-initdb.d/': Permission denied
dbexercise3-adminer-1 | [Sat Nov 26 01:01:54 2022] PHP 7.4.33 Development Server (http://[::]:8080) started
dbexercise3-db-1 exited with code 2
Adminer starts as it should, however I can't log in (because the mariadb container doesn't exist).
I also tried installing Docker Desktop.
After I start the GUI application docker compose up works and I can work with the DB.
Of course I could simply use Docker Desktop and start it on boot, but I don't need the GUI and the need to manually update via DEB-package is annoying.
The answer was surprisingly easy.
I simply hat to run chmod 777 dbsetup to clear the permission error.
I still have no idea how Docker Desktop managed to circumvent this issue, maybe because it runs a task which gives it higher priority?

Unable to run SQL Server 2019 docker with volumes and get ERROR: Setup FAILED copying system data file

When I run latest sql server image from official documentation on linux host.
docker run -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=asdasdasdsad' -p 1433:1433 -v ./data:/var/opt/mssql/data -d mcr.microsoft.com/mssql/server:2019-latest
I get error:
ERROR: Setup FAILED copying system data file 'C:\templatedata\model_replicatedmaster.mdf' to '/var/opt/mssql/data/model_replicatedmaster.mdf': 5(Access is denied.)
This message occurs only on Linux host and with binded volumes.
I happen because lack of permission. On 2019 mssql docker move from root user images into not-root. It made that docker sql-server containers with binded volumes and run on Linux host has a permission issue (=> has no permission to write into binded volume).
There are few solution for this problem:
1. Run docker as root.
eg. compose:
version: '3.6'
services:
mssql:
image: mcr.microsoft.com/mssql/server:2019-latest
user: root
ports:
- 1433:1433
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=BLAH
volumes:
- ./data:/var/opt/mssql/data
Source: https://github.com/microsoft/mssql-docker/issues/13#issuecomment-641904197
2. Setup proper directory owner (mssql)
Check id for mssql user on docker image
sudo docker run -it mcr.microsoft.com/mssql/server id mssql
gives: uid=10001(mssql) gid=0(root) groups=0(root)
Change folder's owner
sudo chown 10001 VOLUME_DIRECTORY
Source in Spanish: https://www.eiximenis.dev/posts/2020-06-26-sql-server-docker-no-se-ejecuta-en-root/
3. Give a full access (not recommended)
Give full access to db files on host
sudo chmod 777 -R VOLUME_DIRECTORY
Unfortunately, the only way I found to fix this issue involves a few manual steps.
I used the following docker-compose file for this to work
version: '3.9'
services:
mssql:
image: mcr.microsoft.com/mssql/server:2019-latest
platform: linux
ports:
- 1433:1433
environment:
- ACCEPT_EULA=Y
- MSSQL_SA_PASSWORD=<testPASSWORDthatISlongENOUGH_1234>
volumes:
- ./mssql/data:/var/opt/mssql/data
- ./backups:/var/backups
(the data directory has to be mounted directly due to another issue with SQL server containers hosted on Windows machines)
Then you need to perform the following manual steps:
Connect to the database using SSMS
Find and select your .bak database backup file
Open a terminal in the container
In the directory that the .mdf and .ldf files are going to be created, touch files with the database name you are going to use
touch /var/opt/mssql/data/DATABASE_NAME.mdf
touch /var/opt/mssql/data/DATABASE_NAME_log.ldf
Toggle the option to replace any existing database with the restore
Restore your database
I tried to follow the instructions in this https://www.sqlservercentral.com/blogs/using-volumes-in-sql-server-2019-non-root-containers article but I could not get it to work.
This problem was also discussed in this github issue (which the bot un-helpfully closed without a proper solution).
I encoutered the same problem as you trying to run a container based on sql server on DigitalOcean. user: root also solved the issue.

MSSQL at Docker exits instantly

I'm trying to setup MSSQL at Docker at Windows 10, but for some reason it started shutting down my container
I've been using it like that for months, but now I have no idea what's happening
C:\Users\user\
λ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
C:\Users\user\
λ docker login
Authenticating with existing credentials...
Login Succeeded
C:\Users\user\
λ docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=<YourStrong!Passw0rd>123' -p 1433:1433 --name sql -d mcr.microsoft.com/mssql/server:2017-latest
337e5efb35f0bf4b465181a0f8be4851b12f353a3a8710ddf817d2f501e5fea
C:\Users\user\
λ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
347q5effb3cf0 mcr.microsoft.com/mssql/server:2017-latest "/opt/mssql/bin/sqls…" 3 seconds ago Up 2 seconds 0.0.0.0:1433->1433/tcp sql
C:\Users\user\
λ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
347q5effb3cf0 mcr.microsoft.com/mssql/server:2017-latest "/opt/mssql/bin/sqls…" 6 seconds ago Exited (1) 2 seconds ago sql
C:\Users\user\
λ docker start sql
sql
C:\Users\user\
λ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
347q5effb3cf0 mcr.microsoft.com/mssql/server:2017-latest "/opt/mssql/bin/sqls…" 14 seconds ago Up 2 seconds 0.0.0.0:1433->1433/tcp sql
C:\Users\user\
λ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
347q5effb3cf0 mcr.microsoft.com/mssql/server:2017-latest "/opt/mssql/bin/sqls…" 16 seconds ago Exited (1) 1 second ago sql
docker logs sql
shows
The SQL Server End-User License Agreement (EULA) must be accepted before SQL
Server can start. The license terms for this product can be downloaded from
http://go.microsoft.com/fwlink/?LinkId=746388.
You can accept the EULA by specifying the --accept-eula command line option,
setting the ACCEPT_EULA environment variable, or using the mssql-conf tool.
The SQL Server End-User License Agreement (EULA) must be accepted before SQL
Server can start. The license terms for this product can be downloaded from
http://go.microsoft.com/fwlink/?LinkId=746388.
You can accept the EULA by specifying the --accept-eula command line option,
setting the ACCEPT_EULA environment variable, or using the mssql-conf tool.
Anybody has an idea what may be wrong?
When running Linux containers from Windows command line/Powershell, the environmental options (-e) require double quotes
docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=<YourStrong!Passw0rd>123" -p 1433:1433 --name sql -d mcr.microsoft.com/mssql/server:2017-latest

How to restore a docker container with a SQL server database

I'm using MacOS Sierra with the latest version of the mssql docker file for linux.
I had built a database which grew to a size of ~69 GB. I started getting an error "Could not allocate a new page for database because of insufficient disk space in filegroup". I attempted to solve this problem by running this code:
USE [master]
GO
ALTER DATABASE [db]
MODIFY FILE ( NAME = N'db', FILEGROWTH = 512MB )
GO
ALTER DATABASE [db]
MODIFY FILE
(NAME = N'db_log', FILEGROWTH = 256MB )
GO
After doing this, I was no longer able to startup the the mssql container. I then manually replaced a backup copy of the container folder which in MacOs is called "com.docker.docker" and which contained the prior working version of the database.
After doing this, I stated getting the following error: "The extended event engine has been disabled by startup options. Features that depend on extended events may fail to start."
At this point I re-installed the docker container using the procedure mentioned in this post. the command I used was:
docker create -v /var/opt/mssql --name mssql microsoft/mssql-server-linux /bin/true
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Test#123' -p 1433:1433 --volumes-from mssql -d --name sql-server microsoft/mssql-server-linux
Although now I'm able to start the server with the new container, I would like restore the original SQL server database (~69 GB). I tried doing so by again manually copying the file named "Docker.qcow2" into the docker container folder. This is obviously not working.
How can I restore my database?

Docker Keep Exiting (Deploying MS SQL on MAC osx)

I'm trying to deploy an MS SQL server on my MAC. There are several alternatives for that.
Here, I'm using docker: I've checked the MS-SQL website, and I executed this code:
docker run -e
'ACCEPT_EULA=Y'
-e 'SA_PASSWORD=<YourStrong!Passw0rd>' -p 1433:1433
-d microsoft/mssql-server-linux
However, The container keeps stopping by itself.
Did I miss something here?
The docker Version I'm using:
Version 1.13.0 (15072)
I had a similar problem. I finally realized the issue was that I was using a dummy password for local dev that didn't adhere to SQL Server's password policy. I used a more complex password and that fixed it up.
I faced this issue recently on Windows. Changing the ' quotes to " fixed the issue.
If you are using MacOS Ventura and/or using a Mac with M1/M2 (Apple Silicon) chip, you will need to enable the Rosetta Emulation to get this to work.
Go to Docker > Settings > Features in development and enable the option 'Use Rosetta for x86/amd64' emulation on Apple Silicon' and restart Docker.
Also, make sure the password obeys the Password Policy set by Microsoft and create a strong password.
Delete the container and re-run the docker run command. An example is below:
docker run -d -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Ithink%Th5r5f0re$Iam' --name sql_server --platform linux/amd64 -p 1433:1433 mcr.microsoft.com/mssql/server:2022-latest
This should get you run the container without the Exited(1) error.
This link explains the details:
https://devblogs.microsoft.com/azure-sql/development-with-sql-in-containers-on-macos/
When running this on Mac you need to bump up your Docker for Mac's RAM. SQL Server needs minimum 4GB RAM, Docker for Mac by default only allocates about 1-2GB for all containers.
To increase Docker for Mac's RAM:
Open Docker for Mac's preferences
Click "Resources"
Move the RAM slider up, in my case I moved it to 6GB (4GB for SQL Server and 2GB for everything else)
You also need to allocate 4GB to the container when starting it up:
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=<YourStrong!Passw0rd>' --memory=4096M -p 1433:1433 -d microsoft/mssql-server-linux
To confirm you memory limits were applied run: docker stats. The MEM USAGE / LIMIT column for the SQL Server container should have output similar to: 536.7MiB / 4GiB
The other thing to watch out for on Mac is that you cannot mount volumes this will cause issues.

Resources