I create my MSSQL database docker container with only docker-compose.yml file and setup.sql file. My .yml file looks like that:
version: "3.7"
services:
sql-server-db:
container_name: sql-server-db
image: mcr.microsoft.com/mssql/server:2019-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "secret123new!"
ACCEPT_EULA: "Y"
volumes:
- ./data/mssql:/scripts/
command:
- /bin/bash
- -c
- |
# Launch MSSQL and send to background
/opt/mssql/bin/sqlservr &
pid=$$!
# Wait for it to be available
echo "Waiting for MS SQL to be available ⏳"
/opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P secret123new! -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , ##servername"
is_up=$$?
while [ $$is_up -ne 0 ] ; do
echo -e $$(date)
/opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P secret123new! -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , ##servername"
is_up=$$?
sleep 5
done
# Run every script in /scripts
# TODO set a flag so that this is only done once on creation,
# and not every time the container runs
cd /scripts
for foo in /scripts/"*.sql"
do echo "Processing $foo";
done
echo "All scripts have been executed. Waiting for MS SQL(pid $$pid) to terminate."
# Wait on the sqlserver process
wait $$pi
when i try to connect with Dbeaver to set in file user credentials there in an error looks like that:
Login failed for user 'system'. ClientConnectionId:17e53706-8242-4bca-974b-3648b5ba7f13
there in an extra warning:
The foo variable is not set. Defaulting to a blank string
setup.sql file in /scripts folder mounted on localhost directory mssql on desktop:
CREATE DATABASE probna
GO
USE probna
GO
CREATE LOGIN system WITH PASSWORD='system'
GO
CREATE USER system FOR LOGIN system
GO
ALTER ROLE [db_owner] ADD MEMBER system
GO
CREATE TABLE Products (ID int, ProductName nvarchar(max))
GO
sql-server-db | ServiAll scripts have been executed. Waiting for MS SQL(pid 8) to terminate.
sql-server-db | ce Broker endpoint is in disabled or stopped state.
What could be wrong with this ? my setup.sql file is in /scripts folder. I don t have another files. Should i replace some of the code or some $$ lines ? Please give me some tips how to start docker mssql database with startup file on windows without node.js :) Have a nice day !
Related
I am working on a MacBook Pro with M1 CPU so I can't use the "normal" mssql docker image. I am using azure-sql-edge that doesn't have sqlcmd to initialize the database (create schema, database, login).
I have created a sql script that I would like to run once the container starts but I can't find any alternative to sqlcmd.
Is there any other way to do it?
I had same issue, I used mssql-tools docker image from Microsoft registry.
Sample docker-compose:
---
version: '3.8'
services:
mssql:
image: mcr.microsoft.com/azure-sql-edge:latest
command: /opt/mssql/bin/sqlservr
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: "SA_Passw0rd"
stdin_open: true
ports:
- 1433:1433
networks:
- db_net
sqlcmd:
image: mcr.microsoft.com/mssql-tools:latest
command: /opt/mssql_scripts/run-initialization.sh
stdin_open: true
volumes:
- ./mssql_scripts:/opt/mssql_scripts
networks:
- db_net
networks:
db_net:
name: db_net
To use this docker-compose you need to have a shell script named run-initialization.sh with execute rights inside mssql_scripts folder.
The run-initialization.sh script waits for database to start up and then execute sql commands:
/opt/mssql-tools/bin/sqlcmd -S mssql -U SA -P SA_Passw0rd -d master -Q "SELECT version()"
or if you want to execute from test.sql file:
/opt/mssql-tools/bin/sqlcmd -S mssql -U SA -P SA_Passw0rd -d master -i /opt/mssql_scripts/test.sql
The solution above worked for me using Mac M1 chip, don't need to create a shell script can run the commands direct.
sqlcmd:
image: mcr.microsoft.com/mssql-tools:latest
stdin_open: true
environment:
- MSSQL_SA_PASSWORD=Xxx
- MSSQL_DATABASE=test
- MSSQL_BACKUP="/opt/mssql/test.bak"
volumes:
- ./test_data.bak:/opt/mssql/test.bak
command: /bin/bash -c '/opt/mssql-tools/bin/sqlcmd -S mssql -U sa -P $$MSSQL_SA_PASSWORD -d tempdb -q "EXIT(RESTORE DATABASE $$MSSQL_DATABASE FROM DISK = $$MSSQL_BACKUP)"; wait;'
mssql:
image: mcr.microsoft.com/azure-sql-edge:latest
environment:
- ACCEPT_EULA=Y
- MSSQL_SA_PASSWORD=Xxxx
- MSSQL_DATABASE=test
- MSSQL_SLEEP=7
ports:
- 1433:1433
Since I am starting a new project I looked into this issue again and found a good solution for me.
I found go-sqlcmd, a new implementation of sqlcmd using golang and it's compatible with M1 chips.
So I am running azure-sql-edge as before using docker compose:
version: "3.9"
services:
mssql:
image: mcr.microsoft.com/azure-sql-edge:latest
command: /opt/mssql/bin/sqlservr
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: ${DATABASE_SA_PASSWORD}
stdin_open: true
ports:
- 1433:1433
When the database container is up and in idle I run this bash script (in my case I am reading the environmnet variables from a .NET appsettings.json file):
cat <appsetting.json> | jq -r 'to_entries|map("\(.key)=\(.value|tostring)")|.[]' > temp
# Show env vars
grep -v '^#' temp
# Export env vars
export $(grep -v '^#' temp | xargs)
export SQLCMDPASSWORD=$DATABASE_SA_PASSWORD
sqlcmd -U sa \
-v DATABASE_SCHEMA=$DATABASE_SCHEMA \
-v DATABASE_DB_NAME=$DATABASE_DB_NAME \
-v DATABASE_LOGIN_NAME=$DATABASE_LOGIN_NAME \
-v DATABASE_LOGIN_PASSWORD=$DATABASE_LOGIN_PASSWORD \
-i sql/init-db.sql,sql/init-user.sql
I had to split the database and schema creation in a script, then I create the user and assign it to the database.
The sql scripts, init-db.sql:
USE master
IF NOT EXISTS (SELECT name FROM sys.schemas WHERE name = N'$(DATABASE_SCHEMA)')
BEGIN
EXEC sys.sp_executesql N'CREATE SCHEMA [$(DATABASE_SCHEMA)] AUTHORIZATION [dbo]'
END
IF NOT EXISTS (SELECT name FROM sys.databases WHERE name = N'$(DATABASE_DB_NAME)')
BEGIN
CREATE DATABASE $(DATABASE_DB_NAME)
END
init-user.sql:
USE $(DATABASE_DB_NAME)
IF NOT EXISTS(SELECT principal_id FROM sys.server_principals WHERE name = '$(DATABASE_LOGIN_NAME)') BEGIN
CREATE LOGIN $(DATABASE_LOGIN_NAME)
WITH PASSWORD = '$(DATABASE_LOGIN_PASSWORD)'
END
IF NOT EXISTS(SELECT principal_id FROM sys.database_principals WHERE name = '$(DATABASE_LOGIN_NAME)') BEGIN
CREATE USER $(DATABASE_LOGIN_NAME) FOR LOGIN $(DATABASE_LOGIN_NAME)
END
I have a problem with connection to MSSQL Docker database created from docker-compose.yml file and additional .sql file with ready-to-go empty database configuration. To connect to the MSSQL database i have used DBeaver. During the connection this error occured:
Login failed for user 'system'. ClientConnectionId:23649321-6526-4ec2-a6ed-9943e64ca019
Probably i have wrong docker-compose.yml file or wrong .sql configuration file (.sql file should work with MSSQL format). .sql file is mounted in scipts folder inside mssql folder with .yml file.
docker-compose.yml file:
version: "3.7"
services:
sql-server-db:
container_name: sql-server-db
image: mcr.microsoft.com/mssql/server:2019-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "secret123new!"
ACCEPT_EULA: "Y"
TRUSTED_CONNECTION: "TRUE"
volumes:
- ./data/mssql:/scripts/
.sql file:
CREATE DATABASE probna
GO
USE probna
GO
CREATE LOGIN system WITH PASSWORD='system'
GO
CREATE USER system FOR LOGIN system
GO
ALTER ROLE [db_owner] ADD MEMBER system
GO
CREATE TABLE Products (ID int, ProductName nvarchar(max))
GO
If you have any ideas or fixes to existing code in .yml or .sql file please write a comment :)
Have a nice day !
PS. do you think this added to .yml file will work ?
mssql:
image: mcr.microsoft.com/mssql/server:2017-latest
environment:
- SA_PASSWORD=Admin123
- ACCEPT_EULA=Y
volumes:
- ./data/mssql:/scripts/
command:
- /bin/bash
- -c
- |
# Launch MSSQL and send to background
/opt/mssql/bin/sqlservr &
# Wait 30 seconds for it to be available
# (lame, I know, but there's no nc available to start prodding network ports)
sleep 30
# Run every script in /scripts
# TODO set a flag so that this is only done once on creation,
# and not every time the container runs
for foo in /scripts/*.sql
do /opt/mssql-tools/bin/sqlcmd -U sa -P $$SA_PASSWORD -l 30 -e -i $$foo
done
# So that the container doesn't shut down, sleep this thread
sleep infinity
if i have setup.sql file in scripts folder is this enough ? i had an error
sql-server-db | Sqlcmd: '/scripts/*.sql': Invalid filename.
what to change in that .yml file or should i add en extra file to scripts folder ?
I'm currently trying to set up a SQL Server in docker compose
and I want to create the database on build with the RUN instruction. This doesn't work, however when I execute the same command on the running container with sh, it works
my compose file looks like this:
version: "3.7"
services:
mssql:
build: ./mssql
environment:
SA_PASSWORD: "Password12345!"
ACCEPT_EULA: "Y"
container_name: mssqlDB
ports:
- "1433:1433"
restart: always
And here my Dockerfile:
FROM mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04
COPY ./prod.sql /
RUN ./opt/mssql-bin/sqlcmd -S localhost -U SA -P "Password12345!" -Q "Create Database HelloWorld"
CMD ["/opt/mssql/bin/sqlservr"]
This is because the SQL Server instance is not started and you must wait for it.
From the Docker Hub official page of SQL Server there are a link to a GitHub Repository where show how to run a sql script on Docker container.
Below I have re-adapted the GitHub code for you case
initialize.sh
# Typically SQL Server takes about 5-10 seconds to start up
# Wait for the SQL Server to come up (90 sec) You can reduce to 20sec and see
sleep 90s
#run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P password -d master -i prod.sql
entrypoint.sh
#start SQL Server, start the script to create the DB and import the data
/opt/mssql/bin/sqlservr & initialize.sh
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04
COPY ./prod.sql /
# Grant permissions for the import-data script to be executable
RUN chmod +x ./initialize.sh
CMD /bin/bash ./entrypoint.sh
Another solution that I personally made is to run the SQL Server service and wait until the service came up.
create.sh
/opt/mssql-tools/bin/sqlcmd -U sa -P $1 -Q 'CREATE DATABASE [MyNewDatabase]'
/opt/mssql-tools/bin/sqlcmd -U sa -P $1 -d 'MyNewDatabase' -i /src/script.sql
script.sql
CREATE TABLE MyTable (..)
DockerFile
FROM mcr.microsoft.com/mssql/server:2017-latest-ubuntu
EXPOSE 1433
WORKDIR /
COPY ./create.sh /src/
COPY ./script.sql /src/
ENV ACCEPT_EULA Y
ENV SA_PASSWORD P#ssw0rd
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN ( /opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Service Broker manager has started" \
&& /src/create.sh P#ssw0rd \
&& pkill sqlservr
I'm trying to run a setup script on a Docker SQL Server image
For this I have created a Dockerfile from the mssql image
FROM microsoft/mssql-server-linux:2017-CU8
# Create directory to place app specific files
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Copy setup scripts
COPY entrypoint.sh \
./
RUN chmod +x ./entrypoint.sh
CMD /bin/bash ./entrypoint.sh
In entrypoint.sh I'm starting SQL Server and I want to run some setup commands.
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr &
echo 'Sleeping 20 seconds before running setup script'
sleep 20s
echo 'Starting setup script'
#run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P <MyPassWd> -d master -i setup.sql
echo 'Finished setup script'
When I run this script, the database starts, the setup runs, and after the
setup is finished, the container shuts down.
So I thought something in the script makes the container shut down, therefore I stripped the script down to a bare minimum
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr &
echo 'Sleeping 20 seconds before running setup script'
sleep 20s
That also stops the container after sleep 20s finished.
Moving on...
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr &
Which stops the container right away
And then...
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr
Now the container runs, but I can't do any initialization
Does someone know how to get this working?
Change the password of the sql server to be complex enough.
docker run -d -p 1433:1433 -e "sa_password=ComplexPW2019!" -e "ACCEPT_EULA=Y" <sqlserverimageid>
Root cause of this issue is PID 1 allocation for docker container.
PID 1 will be allocated to command given in CMD in Dockerfile (in our case ./entrypoint.sh)
Container has a life spam according to PID 1(as soon as PID 1 is stop/killed container will be stopped)
1) In case of /opt/mssql/bin/sqlservr &
a child process ID will be allocated to sqlserver cmd and will be executed in background and as soon as rest of the script is executed, container will stop.
2) In case of /opt/mssql/bin/sqlservr
script will not proceed from here until this execution will complete.
so the solution is to assign PID 1 to CMD /opt/mssql/bin/sqlservr and rest of the script should be executed as child process.
I have done below changes and it is working for me.
in Dockerfile
replace CMD /bin/bash ./entrypoint.sh to CMD exec /bin/bash entrypoint.sh
in entrypoint.sh
#!/bin/bash
#start SQL Server
sh -c "
echo 'Sleeping 20 seconds before running setup script'
sleep 20s
echo 'Starting setup script'
#run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P \"YourStrong!Passw0rd\" -Q
\"ALTER LOGIN SA WITH PASSWORD='NewStrong!Passw0rd'\"
echo 'Finished setup script'
exit
" &
exec /opt/mssql/bin/sqlservr
To create the database on startup, try the approach below.
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA Y
ENV DB_NAME test
COPY startup.sh /var/opt/mssql/startup.sh
CMD ["bash", "/var/opt/mssql/startup.sh"]
startup.sh
#!/usr/bin/env bash
if ! [ -f /var/opt/mssql/.initialized ] && [ -n "$DB_NAME" ]; then
while ! </dev/tcp/localhost/1433 2>/dev/null; do
sleep 2
done
echo "Creating $DB_NAME database..."
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "$SA_PASSWORD" -d master \
-Q "CREATE DATABASE $DB_NAME"
touch /var/opt/mssql/.initialized
fi &
/opt/mssql/bin/sqlservr
SQL Server has to be the right most command.
I know it does not make sense as you want SQL Server to run first and then run your scripts to create/restore databases. I guess this is because of the way SQL Server runs on Linux ( Sql server process creates a SQL server process as part of startup).
MSDN documentation makes the order of execution clear at: https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-configure-docker?view=sql-server-ver15#customcontainer
So for your example, you would have to write something like:
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P <MyPassWd> -d master -i setup.sql & /opt/mssql/bin/sqlservr
There is a pull request to allow to run an init SQL script on first time run.
The purpose of this PR is to add into the start.ps1 the ability to check the folder docker-entrypoint-initdb and run all sql scripts inside. Once it is done, the script creates a flag file to avoid running the setup phase on the next startup after a stop.
By mounting a volume from local folder scripts to c:/docker-entrypoint-initdb, the container will execute all .sql scripts files. The volume should be a directory.
E.g.
version: "3.8"
services:
sqlserver:
platform: windows/amd64
environment:
- sa_password=<YourPassword>
- ACCEPT_EULA=Y
image: microsoft/mssql-server-windows-developer
volumes:
- ./dockerfiles/sqlserver/initdb:c:/docker-entrypoint-initdb:ro
ports:
- "1433:1433"
I need sql server container with some database. I've prepared the following dockerfile:
FROM microsoft/mssql-server-linux:latest
ENV ACCEPT_EULA Y
ENV SA_PASSWORD yourStrong(!)Password
WORKDIR sqlserver
COPY load_db.sh load_db.sh
COPY /resources/sql/ sql
ENTRYPOINT ./load_db.sh
So it runs load_db.sh:
#!/usr/bin/env bash
SERVER_OUT=/var/log/sqlserver.out
TIMEOUT=90
/opt/mssql/bin/sqlservr &>${SERVER_OUT} &
function server_ready() {
grep -q -F 'Recovery is complete.' ${SERVER_OUT}
}
echo 'Wait until Microsoft SQL Server is up'
for (( i=0; i<${TIMEOUT}; i++ )); do
sleep 1
if server_ready; then
break
fi
done
echo 'Microsoft SQL Server is up'
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P ${SA_PASSWORD} -Q "CREATE DATABASE MarketDataService;"
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P ${SA_PASSWORD} -d MarketDataService -i sql/2018.0100.000000001.Add_Layout_table.sql
To run it I've prepared the followinf docker-compose file:
services:
db:
build:
context: .
dockerfile: Dockerfile-db
ports:
- 1433:1433
When I try to run docker-compose up --build it looks okay.
I have output:
db_1 | Wait until Microsoft SQL Server is up
db_1 | Microsoft SQL Server is up
db_1 |
db_1 | (3 rows affected)
But after it is exited...
amptest_db_1 exited with code 0
To solve it I've tried to add tty: true but it doesn't make sense. The same output. How can I keep alive my container by docker-compose? Here I've found tail -F anything. It works but looks terrible. Is there a better way?
Upd: I've stayed with tail -F '/var/log/sqlserver.out'