I'm trying to create a container with SQL Server running on with this command:
docker run -e "ACCEPT_EULA=Y" -e MSSQL_SA_PASSWORD="MyPassword1"
-e MSSQL_COLLATION="Polish_CI_AS" -p 1434:1433
-v C:/Users/User1/sql-server/data:/var/opt/mssql/data
-d mcr.microsoft.com/mssql/server:2019-latest
Everything is working fine, the env variable is set but the server collation is still the default - SQL_Latin1_General_CP1_CI_AS.
Any ideas?
Note that at the time of this writing, MSSQL_COLLATION only works when initializing the server for the first time (i.e. creating the master database). In order to change the server collation:
Create backups of all user databases.
Stop the old container.
Start container with -e MSSQL_COLLATION=Polish_CI_AS. Ensure the data volume is empty except for the backups.
Restore from the backups.
Related
My committed image of SQL Server does not seem to work when using a mount. When not mounted, the image contains the changes I made but when when I mount the data directory I don't see the committed changes.
Step to reproduce:
Run a container with SQL server 2019.
docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=P#ssword123" -p 14350:1433 -d mcr.microsoft.com/mssql/server:2019-latest
Connect to this SQL Server and create a database named TestDB.
Commit the container, tagged as testDB, and delete the container. (My contianer ID is cabc3ac26dd0)
docker commit cabc3ac26dd0 testdb
docker container stop cabc3ac26dd0
docker rm cabc3ac26dd0
Run a container with this new image.
docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=P#ssword123" -p 14350:1433 -d testdb
Connect to this SQL sServer container and you will see the new TestDB database. This is your new database, saved as part of the commit. This shows that the commit worked.
Remove the container, and run a new container for testdb with the data drive mounted to your local filesystem.
docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=P#ssword123" -p 14350:1433 --mount type=bind,source=C:\Users\me\testdbfiles\mssql\data,target=/var/opt/mssql/data -d testdb
Connect to this sql server conatiner and you will not see the new TestDB.
It seems that the commit has worked when I run without the mount. When I run with the mount, I don't see the changes for my committed image.
This docker run command is the same one I have used for months.
This problem started about a week ago, near when I upgraded to docker
desktop 4.12.0.
The mounted location (C:\Users\me\testdbfiles\mssql\data) has the
sql server database files (such as master.mdf), therefore the mount
the syntax must be correct.
Running a select from sys.databases shows the new TestDB when I am not mounted. When mounted, this select doesn't show this database. This indicates that the master database itself is different when mounted.
Listing the contents of the /var/opt/mssql/data shows the TestDB file when not mounting, but does not show them when mounting.
Has anyone else experienced this problem?
How can I troubleshoot what happens during the mounting phase?
Update: I also added a new sql server login to the image. When I mount I don't see the login, with no bind mount the login is there.
The following docker file creates a custom SQL server image with a database restored from a backup (rmsdev.bak).
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV MSSQL_PID=Developer
ENV SA_PASSWORD=Password1?
ENV ACCEPT_EULA=Y
USER mssql
COPY rmsdev.bak /var/opt/mssql/backup/
# Launch SQL Server, confirm startup is complete, restore the database, then terminate SQL Server.
RUN ( /opt/mssql/bin/sqlservr & ) | grep -q "Service Broker manager has started" \
&& /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q 'RESTORE DATABASE rmsdev FROM DISK = "/var/opt/mssql/backup/rmsdev.bak" WITH MOVE "rmsdev" to "/var/opt/mssql/data/rmsdev.mdf", MOVE "rmsdev_Log" to "/var/opt/mssql/data/rmsdev_log.ldf", NOUNLOAD, STATS = 5' \
&& pkill sqlservr
CMD ["/opt/mssql/bin/sqlservr"]
The issue is that, once the restore is complete, the backup file is not required anymore and I would like to remove it from the image.
Unfortunately, due to how docker images are formed (layers) I cannot simply 'rm' the file as I would like to.
Multistage Dockerfile is not easily applicable in this case as in a build scenario.
Another way would be to run the container, restore the backup and then commit a new image, but what I am looking to do is to use only docker build with the proper Dockerfile.
Does anyone know a way?
If you know where the data directory is in the image, and the image does not declare that directory as a VOLUME, then you can use a multi-stage build for this. The first stage would set up the data directory as you show. The second stage would copy the populated data directory from the first stage but not the backup file. This trick might depend on the two stages running identical builds of the underlying software.
For SQL Server, the Docker Hub page and GitHub repo are both tricky to find, and surprisingly neither talks to the issue of data storage (as #HansKillian notes in a comment, you would almost always want to store the database data in some sort of volume). The GitHub repo does include a Helm chart built around a Kubernetes StatefulSet and from that we can discover that a data directory would be mounted on /var/opt/mssql.
So I might write a multi-stage build like so:
# Put common setup steps in an initial stage
FROM mcr.microsoft.com/mssql/server:2019-latest AS setup
ENV MSSQL_PID=Developer
ENV SA_PASSWORD=Password1? # (weak password, easily extracted with `docker inspect`)
ENV ACCEPT_EULA=Y # (legally probably the end user needs to accept this not the image builder)
# Have a stage specifically to populate the data directory
FROM setup AS data
# (copy-and-pasted from the question)
USER mssql
COPY rmsdev.bak / # not under /var/opt/mssql
RUN ( /opt/mssql/bin/sqlservr & ) | grep -q "Service Broker manager has started" \
&& /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q 'RESTORE DATABASE rmsdev FROM DISK = "/rmsdev.bak" WITH MOVE "rmsdev" to "/var/opt/mssql/data/rmsdev.mdf", MOVE "rmsdev_Log" to "/var/opt/mssql/data/rmsdev_log.ldf", NOUNLOAD, STATS = 5' \
&& pkill sqlservr
# Final stage that actually will actually be run.
FROM setup
# Copy the prepopulated data tree, but not the backup file
COPY --from=data /var/opt/mssql /var/opt/mssql
# Use the default USER, CMD, etc. from the base SQL Server image
The standard Docker Hub open-source database images like mysql and postgres generally declare a VOLUME in their Dockerfile for the database data, which forces the data to be stored in a volume. The important thing this means is that you can't set up data in the image like this; you have to populate the data externally, and then copy the data tree outside of the Docker image system.
I'm trying to add a volume to a docker container but when I commit it and run with the new volume none of the sql services run on this copy?? Why would that be.
https://learn.microsoft.com/en-us/sql/linux/quickstart-install-connect-docker?view=sql-server-ver15&pivots=cs1-powershell
I am adding the initial one as above and it works.All fine. Services running. I can connect to it, run SQL but I need to share a drive.
Seems I cant add one directly to an existing instance??
docker commit 5a8f89adeead newimagename
docker run -ti -v "C:/dir1":/dir1 newimagename /bin/bash
I do the above to clone it and add a volume. WORKS. But the sql services just arent running on this new one. Ill accept it either way I just want SQL running and a share in there.
Can anyone help.
Manged it:
docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=Pa55word1" `
-v C:/db:/dir1 `
-p 1433:1433 --name sql3 `
-d mcr.microsoft.com/mssql/server:2019-CU3-ubuntu-18.04
Had an issue with having no drive or no services but this has done it.
On Docker for Windows and working with windows containers, I cannot get my persistent volume to work on the main database directory of the windows container. This would be C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\DATA
How can I get the benefits of persistent volumes for databases without having to mess with backups and restores into the mssql-server-container?
This may be because of the data directory having the master- and system-dbs stored inside this folder where I try to mount the persistent volume.
In SQL Server for linux containers this simply works, you can connect the persistent volume to /var/opt/mssql and have your database persistent.
I know I can recover a database from a backup into the container, but this has two major drawbacks: I have to have a big container size because I am working with a big database. So I extended the 20 GB limit of the container to 60 GB but... rebuilding the database each time from a backup is time consuming.
The second drawback is, if the mssql-dev container is killed, the database is lost, too. Any work on this database is then gone. This would be different if the database could reside on the persistent volume.
docker run -d -e sa_password=<Password> -e ACCEPT_EULA=Y -v "C:\mylocalfolder:C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\DATA" microsoft/mssql-server-windows-developer
The error message is 'failure in a Windows system call: the virtual computer or container was shutdown unexpectetly. (0xc0370106)
Workaround 1
connect persistent volume to another location like
c:\mydata to prevent the error message from above.
Then get the database connected to the server while not using the standard database folder.
Extract database .bak file, so there are mdf and log files
--Get the name of your DB
RESTORE FILELISTONLY
FROM DISK = 'c:\mydata'
GO
--do the extraction of the bak file to certain folder
RESTORE DATABASE mydatabase
FROM DISK = 'c:\mydata'
WITH REPLACE,
MOVE 'mydatabase' TO 'c:\mydata\extractedDb.mdf',
MOVE 'mydatabase_log' TO 'c:\mydata\extractedLog.ldf'
GO
With this done you should now have your database files ready on your persistent volume. Now attach the database to the server. This has to be done by creating a new db but this procedure only takes nanoseconds to complete!
CREATE DATABASE StackoverflowIsGreat
ON (FILENAME = 'c:\mydata\extractedDb.mdf'),
(FILENAME = 'c:\mydata\extractedLog.ldf')
FOR ATTACH;
Now the database is safe in a persistent volume. If the db-server container goes down or is rebuild, you simply run this last statement again (or even better implement it in your docker-compose or dockerfile):
CREATE DATABASE StackoverflowIsGreat
ON (FILENAME = 'c:\mydata\extractedDb.mdf'),
(FILENAME = 'c:\mydata\extractedLog.ldf')
FOR ATTACH;
Workaround 2
the -attach_dbs parameter seems to work the same way.
Docker run:
docker run -p 1433:1433 --name mssql-dev -e sa_password=<yourpassword> -e ACCEPT_EULA=Y -e attach_dbs="[{'dbName':'PowerSlide_SQLDB','dbFiles':['C:\\your\\path\\database.mdf','C:\\sqldata\\databaselog.ldf']}]" -v "d:\sqldata:C:\sqldata" microsoft/mssql-server-windows-developer
or if you prefer Docker-Compose, it is a little bit tricky. I had to omit the leading and closing ' outside of the brackets and replace the double quotation marks inside the brackets with ' to make it work.
version: '3.2'
services:
mssql-dev:
container_name: mssql-dev
image: 'microsoft/mssql-server-windows-developer'
volumes:
- "d:\\sqldata:C:\\sqldata"
ports:
- "1433:1433"
restart: always
environment:
- "ACCEPT_EULA=Y"
- "sa_password=yourpassword"
- attach_dbs=[{"dbName":"<yourDbName>","dbFiles":["C:\\<your>\\path\\database.mdf","C:\\your\\path\\databaselog.ldf"]}]
volumes:
mssql-dev-data:
It seems this question can be answered with workaround 1 and 2 from above.
Connect persistent volume to another location like c:\mydata to prevent the error message from above. Then get the database connected to the server while not using the standard database folder.
Extract database .bak file, so there are mdf and log files
--Get the name of your DB
RESTORE FILELISTONLY
FROM DISK = 'c:\mydata'
GO
--do the extraction of the bak file to certain folder
RESTORE DATABASE mydatabase
FROM DISK = 'c:\mydata'
WITH REPLACE,
MOVE 'mydatabase' TO 'c:\mydata\extractedDb.mdf',
MOVE 'mydatabase_log' TO 'c:\mydata\extractedLog.ldf'
GO
Attach the database to the server in one of the following three ways:
Docker run example:
docker run -p 1433:1433 --name mssql-dev -e sa_password=<yourpassword> -e ACCEPT_EULA=Y -e attach_dbs="[{'dbName':'PowerSlide_SQLDB','dbFiles':['C:\\your\\path\\database.mdf','C:\\sqldata\\databaselog.ldf']}]" -v "d:\sqldata:C:\sqldata" microsoft/mssql-server-windows-developer
if you prefer Docker-Compose, it is a little bit tricky. I had to omit the leading and closing ' outside of the brackets and replace the double quotation marks inside the brackets with ' to make it work. Example for docker-compose:
version: '3.2'
services:
mssql-dev:
container_name: mssql-dev
image: 'microsoft/mssql-server-windows-developer'
volumes:
- "d:\\sqldata:C:\\sqldata"
ports:
- "1433:1433"
restart: always
environment:
- "ACCEPT_EULA=Y"
- "sa_password=yourpassword"
- attach_dbs=[{"dbName":"<yourDbName>","dbFiles":["C:\\<your>\\path\\database.mdf","C:\\your\\path\\databaselog.ldf"]}]
volumes:
mssql-dev-data:
Or attach DB with SQL Command
CREATE DATABASE StackoverflowIsGreat
ON (FILENAME = 'c:\mydata\extractedDb.mdf'),
(FILENAME = 'c:\mydata\extractedLog.ldf')
FOR ATTACH;
I'm using MacOS Sierra with the latest version of the mssql docker file for linux.
I had built a database which grew to a size of ~69 GB. I started getting an error "Could not allocate a new page for database because of insufficient disk space in filegroup". I attempted to solve this problem by running this code:
USE [master]
GO
ALTER DATABASE [db]
MODIFY FILE ( NAME = N'db', FILEGROWTH = 512MB )
GO
ALTER DATABASE [db]
MODIFY FILE
(NAME = N'db_log', FILEGROWTH = 256MB )
GO
After doing this, I was no longer able to startup the the mssql container. I then manually replaced a backup copy of the container folder which in MacOs is called "com.docker.docker" and which contained the prior working version of the database.
After doing this, I stated getting the following error: "The extended event engine has been disabled by startup options. Features that depend on extended events may fail to start."
At this point I re-installed the docker container using the procedure mentioned in this post. the command I used was:
docker create -v /var/opt/mssql --name mssql microsoft/mssql-server-linux /bin/true
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Test#123' -p 1433:1433 --volumes-from mssql -d --name sql-server microsoft/mssql-server-linux
Although now I'm able to start the server with the new container, I would like restore the original SQL server database (~69 GB). I tried doing so by again manually copying the file named "Docker.qcow2" into the docker container folder. This is obviously not working.
How can I restore my database?