I would like to copy db.bak to mounted volume_backup in order to run restore query basically. But it seems the file cannot be copied even though there message is shown up CACHED [2/2] COPY db.bak /var/opt/mssql/backup/
Unluckily, the server run but no db.bak file found
Msg 3201, Level 16, State 2, Server 3039ef6f2621, Line 1
myssql_1 | Cannot open backup device '/var/opt/mssql/backup/db.bak'. Operating system error 2(The system cannot find the file specified.).
myssql_1 | Msg 3013, Level 16, State 1, Server 3039ef6f2621, Line 1
myssql_1 | RESTORE DATABASE is terminating abnormally.
Here is my folder structure:
|docker-compose.yml
|SqlV1
|db.bak
|Dockerfile
|volume_backup
|[Expected db.bak would be here]
docker-compose.yml
version: '3.8'
services:
myssql:
build: ./SqlV1
restart: always
ports:
- '1433:1433'
volumes:
- ./volume_backup:/var/opt/mssql/backup
networks:
- data-pipeline
volumes:
db_data:
networks:
data-pipeline:
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA=Y
ENV MSSQL_SA_PASSWORD=Abc#123456
USER root
COPY db.bak /var/opt/mssql/backup/
CMD /opt/mssql-tools/bin/sqlcmd \
-S localhost -U 'sa' -P $MSSQL_SA_PASSWORD \
-Q 'RESTORE DATABASE Lion FROM DISK = "/var/opt/mssql/backup/db.bak" WITH MOVE "SRLM" TO "/var/opt/mssql/data/SRLM.mdf", MOVE "SRLM_Log" TO "/var/opt/mssql/data/SRLM.ldf"' & /opt/mssql/bin/sqlservr
If I manually copy to volume_backup, it worked fine. I just wonder that why this COPY seems doing nothing?
Related
I create a new image base on the "mcr.microsoft.com/mssql/server" image.
Then I have a script to create a new database with some tables with seeded data within the Dockerfile.
FROM mcr.microsoft.com/mssql/server
USER root
# CreateDb
COPY ./CreateDatabaseSchema.sql ./opt/scripts/
ENV ACCEPT_EULA=true
ENV MSSQL_SA_PASSWORD=myP#ssword#1
# Create database
RUN /opt/mssql/bin/sqlservr & sleep 60; /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P ${MSSQL_SA_PASSWORD} -d master -i /opt/scripts/CreateDatabaseSchema.sql
I can see the database created by my script if I don't attach it to a persistent volume, and DO NOT see the new database if I attach it to a persistent volume. I check the log and don't see any error. Looks like the system skip to process that file. What is the problem that might cause the environment to skip processing the SQL script whci defined in Dockerfile?
thanks,
Austin
The problem with using persistent volume is all the data in that directory is replaced by the base image. I need to learn how to create the database after volume mounts
volumeMounts:
- mountPath: /var/opt/mssql
You can use docker-compose.yml and Dockerfile. Both can work together.
version: '3.9'
services:
mysqlserver:
build:
context: ..
dockerfile: Dockerfile
restart: always
volumes:
- make/my/db/persistent:/var/opt/mssql
Then you can run it with:
docker-compose -f docker-compose.yml up
Have fun
I have a problem with connection to MSSQL Docker database created from docker-compose.yml file and additional .sql file with ready-to-go empty database configuration. To connect to the MSSQL database i have used DBeaver. During the connection this error occured:
Login failed for user 'system'. ClientConnectionId:23649321-6526-4ec2-a6ed-9943e64ca019
Probably i have wrong docker-compose.yml file or wrong .sql configuration file (.sql file should work with MSSQL format). .sql file is mounted in scipts folder inside mssql folder with .yml file.
docker-compose.yml file:
version: "3.7"
services:
sql-server-db:
container_name: sql-server-db
image: mcr.microsoft.com/mssql/server:2019-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "secret123new!"
ACCEPT_EULA: "Y"
TRUSTED_CONNECTION: "TRUE"
volumes:
- ./data/mssql:/scripts/
.sql file:
CREATE DATABASE probna
GO
USE probna
GO
CREATE LOGIN system WITH PASSWORD='system'
GO
CREATE USER system FOR LOGIN system
GO
ALTER ROLE [db_owner] ADD MEMBER system
GO
CREATE TABLE Products (ID int, ProductName nvarchar(max))
GO
If you have any ideas or fixes to existing code in .yml or .sql file please write a comment :)
Have a nice day !
PS. do you think this added to .yml file will work ?
mssql:
image: mcr.microsoft.com/mssql/server:2017-latest
environment:
- SA_PASSWORD=Admin123
- ACCEPT_EULA=Y
volumes:
- ./data/mssql:/scripts/
command:
- /bin/bash
- -c
- |
# Launch MSSQL and send to background
/opt/mssql/bin/sqlservr &
# Wait 30 seconds for it to be available
# (lame, I know, but there's no nc available to start prodding network ports)
sleep 30
# Run every script in /scripts
# TODO set a flag so that this is only done once on creation,
# and not every time the container runs
for foo in /scripts/*.sql
do /opt/mssql-tools/bin/sqlcmd -U sa -P $$SA_PASSWORD -l 30 -e -i $$foo
done
# So that the container doesn't shut down, sleep this thread
sleep infinity
if i have setup.sql file in scripts folder is this enough ? i had an error
sql-server-db | Sqlcmd: '/scripts/*.sql': Invalid filename.
what to change in that .yml file or should i add en extra file to scripts folder ?
When I run latest sql server image from official documentation on linux host.
docker run -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=asdasdasdsad' -p 1433:1433 -v ./data:/var/opt/mssql/data -d mcr.microsoft.com/mssql/server:2019-latest
I get error:
ERROR: Setup FAILED copying system data file 'C:\templatedata\model_replicatedmaster.mdf' to '/var/opt/mssql/data/model_replicatedmaster.mdf': 5(Access is denied.)
This message occurs only on Linux host and with binded volumes.
I happen because lack of permission. On 2019 mssql docker move from root user images into not-root. It made that docker sql-server containers with binded volumes and run on Linux host has a permission issue (=> has no permission to write into binded volume).
There are few solution for this problem:
1. Run docker as root.
eg. compose:
version: '3.6'
services:
mssql:
image: mcr.microsoft.com/mssql/server:2019-latest
user: root
ports:
- 1433:1433
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=BLAH
volumes:
- ./data:/var/opt/mssql/data
Source: https://github.com/microsoft/mssql-docker/issues/13#issuecomment-641904197
2. Setup proper directory owner (mssql)
Check id for mssql user on docker image
sudo docker run -it mcr.microsoft.com/mssql/server id mssql
gives: uid=10001(mssql) gid=0(root) groups=0(root)
Change folder's owner
sudo chown 10001 VOLUME_DIRECTORY
Source in Spanish: https://www.eiximenis.dev/posts/2020-06-26-sql-server-docker-no-se-ejecuta-en-root/
3. Give a full access (not recommended)
Give full access to db files on host
sudo chmod 777 -R VOLUME_DIRECTORY
Unfortunately, the only way I found to fix this issue involves a few manual steps.
I used the following docker-compose file for this to work
version: '3.9'
services:
mssql:
image: mcr.microsoft.com/mssql/server:2019-latest
platform: linux
ports:
- 1433:1433
environment:
- ACCEPT_EULA=Y
- MSSQL_SA_PASSWORD=<testPASSWORDthatISlongENOUGH_1234>
volumes:
- ./mssql/data:/var/opt/mssql/data
- ./backups:/var/backups
(the data directory has to be mounted directly due to another issue with SQL server containers hosted on Windows machines)
Then you need to perform the following manual steps:
Connect to the database using SSMS
Find and select your .bak database backup file
Open a terminal in the container
In the directory that the .mdf and .ldf files are going to be created, touch files with the database name you are going to use
touch /var/opt/mssql/data/DATABASE_NAME.mdf
touch /var/opt/mssql/data/DATABASE_NAME_log.ldf
Toggle the option to replace any existing database with the restore
Restore your database
I tried to follow the instructions in this https://www.sqlservercentral.com/blogs/using-volumes-in-sql-server-2019-non-root-containers article but I could not get it to work.
This problem was also discussed in this github issue (which the bot un-helpfully closed without a proper solution).
I encoutered the same problem as you trying to run a container based on sql server on DigitalOcean. user: root also solved the issue.
I'm trying to bind a host directory to a mssql docker container and have the following [bash] script:
#!/bin/sh
IMPORTDIR=$(cd $(dirname $0); pwd)
DATADIR=${DATADIR:-/tmp/db/}
set -xe
docker run --detach \
-e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=Passw0rd" -e "MSSQL_PID=Express" \
-v ${DATADIR}:/mnt/db \
-v ${IMPORTDIR}:/mnt/importing \
-p 127.0.0.1:1433:1433 \
--name mssql-db \
mcr.microsoft.com/mssql/server:latest
sleep 30
docker exec mssql-db /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P Passw0rd -Q "
RESTORE FILELISTONLY FROM DISK = '/mnt/db/mssql_db.bak'"
For which I receive the following error message when I run the script:
RESTORE FILELISTONLY FROM DISK = '\''/mnt/db/mssql_db.bak'\'''
Msg 3201, Level 16, State 2, Server cd17fd41ce82, Line 2
Cannot open backup device '/mnt/db/mssql_db.bak'. Operating system error 2(The system cannot find the file specified.).
Msg 3013, Level 16, State 1, Server cd17fd41ce82, Line 2
RESTORE FILELIST is terminating abnormally.
When I log into the container I see that indeed the file, which is in the host directory, isn't present is in the container's directory.
The following is my setup info:
Operating System: Mac Os 10.13.6
Docker Version:
File perms (on the host): -rw-r--r--#
I solved this because by adding the real path to /tmp to the file sharing resource path in docker.
In my OS version (can't comment on the later versions), the /tmp is a symlink to it's real path /private/tmp once I added that to the FILE SHARING resources, it worked.
I had just recently upgraded to docker as this wasn't an issue I previously experienced pre-upgrade
On Docker for Windows and working with windows containers, I cannot get my persistent volume to work on the main database directory of the windows container. This would be C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\DATA
How can I get the benefits of persistent volumes for databases without having to mess with backups and restores into the mssql-server-container?
This may be because of the data directory having the master- and system-dbs stored inside this folder where I try to mount the persistent volume.
In SQL Server for linux containers this simply works, you can connect the persistent volume to /var/opt/mssql and have your database persistent.
I know I can recover a database from a backup into the container, but this has two major drawbacks: I have to have a big container size because I am working with a big database. So I extended the 20 GB limit of the container to 60 GB but... rebuilding the database each time from a backup is time consuming.
The second drawback is, if the mssql-dev container is killed, the database is lost, too. Any work on this database is then gone. This would be different if the database could reside on the persistent volume.
docker run -d -e sa_password=<Password> -e ACCEPT_EULA=Y -v "C:\mylocalfolder:C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\DATA" microsoft/mssql-server-windows-developer
The error message is 'failure in a Windows system call: the virtual computer or container was shutdown unexpectetly. (0xc0370106)
Workaround 1
connect persistent volume to another location like
c:\mydata to prevent the error message from above.
Then get the database connected to the server while not using the standard database folder.
Extract database .bak file, so there are mdf and log files
--Get the name of your DB
RESTORE FILELISTONLY
FROM DISK = 'c:\mydata'
GO
--do the extraction of the bak file to certain folder
RESTORE DATABASE mydatabase
FROM DISK = 'c:\mydata'
WITH REPLACE,
MOVE 'mydatabase' TO 'c:\mydata\extractedDb.mdf',
MOVE 'mydatabase_log' TO 'c:\mydata\extractedLog.ldf'
GO
With this done you should now have your database files ready on your persistent volume. Now attach the database to the server. This has to be done by creating a new db but this procedure only takes nanoseconds to complete!
CREATE DATABASE StackoverflowIsGreat
ON (FILENAME = 'c:\mydata\extractedDb.mdf'),
(FILENAME = 'c:\mydata\extractedLog.ldf')
FOR ATTACH;
Now the database is safe in a persistent volume. If the db-server container goes down or is rebuild, you simply run this last statement again (or even better implement it in your docker-compose or dockerfile):
CREATE DATABASE StackoverflowIsGreat
ON (FILENAME = 'c:\mydata\extractedDb.mdf'),
(FILENAME = 'c:\mydata\extractedLog.ldf')
FOR ATTACH;
Workaround 2
the -attach_dbs parameter seems to work the same way.
Docker run:
docker run -p 1433:1433 --name mssql-dev -e sa_password=<yourpassword> -e ACCEPT_EULA=Y -e attach_dbs="[{'dbName':'PowerSlide_SQLDB','dbFiles':['C:\\your\\path\\database.mdf','C:\\sqldata\\databaselog.ldf']}]" -v "d:\sqldata:C:\sqldata" microsoft/mssql-server-windows-developer
or if you prefer Docker-Compose, it is a little bit tricky. I had to omit the leading and closing ' outside of the brackets and replace the double quotation marks inside the brackets with ' to make it work.
version: '3.2'
services:
mssql-dev:
container_name: mssql-dev
image: 'microsoft/mssql-server-windows-developer'
volumes:
- "d:\\sqldata:C:\\sqldata"
ports:
- "1433:1433"
restart: always
environment:
- "ACCEPT_EULA=Y"
- "sa_password=yourpassword"
- attach_dbs=[{"dbName":"<yourDbName>","dbFiles":["C:\\<your>\\path\\database.mdf","C:\\your\\path\\databaselog.ldf"]}]
volumes:
mssql-dev-data:
It seems this question can be answered with workaround 1 and 2 from above.
Connect persistent volume to another location like c:\mydata to prevent the error message from above. Then get the database connected to the server while not using the standard database folder.
Extract database .bak file, so there are mdf and log files
--Get the name of your DB
RESTORE FILELISTONLY
FROM DISK = 'c:\mydata'
GO
--do the extraction of the bak file to certain folder
RESTORE DATABASE mydatabase
FROM DISK = 'c:\mydata'
WITH REPLACE,
MOVE 'mydatabase' TO 'c:\mydata\extractedDb.mdf',
MOVE 'mydatabase_log' TO 'c:\mydata\extractedLog.ldf'
GO
Attach the database to the server in one of the following three ways:
Docker run example:
docker run -p 1433:1433 --name mssql-dev -e sa_password=<yourpassword> -e ACCEPT_EULA=Y -e attach_dbs="[{'dbName':'PowerSlide_SQLDB','dbFiles':['C:\\your\\path\\database.mdf','C:\\sqldata\\databaselog.ldf']}]" -v "d:\sqldata:C:\sqldata" microsoft/mssql-server-windows-developer
if you prefer Docker-Compose, it is a little bit tricky. I had to omit the leading and closing ' outside of the brackets and replace the double quotation marks inside the brackets with ' to make it work. Example for docker-compose:
version: '3.2'
services:
mssql-dev:
container_name: mssql-dev
image: 'microsoft/mssql-server-windows-developer'
volumes:
- "d:\\sqldata:C:\\sqldata"
ports:
- "1433:1433"
restart: always
environment:
- "ACCEPT_EULA=Y"
- "sa_password=yourpassword"
- attach_dbs=[{"dbName":"<yourDbName>","dbFiles":["C:\\<your>\\path\\database.mdf","C:\\your\\path\\databaselog.ldf"]}]
volumes:
mssql-dev-data:
Or attach DB with SQL Command
CREATE DATABASE StackoverflowIsGreat
ON (FILENAME = 'c:\mydata\extractedDb.mdf'),
(FILENAME = 'c:\mydata\extractedLog.ldf')
FOR ATTACH;