docker-compose for mssql not using .env file? - sql-server

I have a .env file named .mssql filled with the basic MSSQL options:
ACCEPT_EULA=Y
MSSQL_SA_PASSWORD=12Password34
My docker-compose.yml looks like this:
version: '3'
services:
db:
image: "mcr.microsoft.com/mssql/server:2017-latest"
volumes:
- ./db-data:/var/opt/mssql
- ./sql_scripts:/sql_scripts
env_file:
- .envs/.local/.mssql
healthcheck:
test: [ "CMD", "/opt/mssql-tools/bin/sqlcmd", "-S", "localhost", "-U", "sa", "-P", "$SA_PASSWORD", "-Q", "SELECT 1" ]
interval: 30s
timeout: 30s
retries: 3
# This command runs the sqlservr process and then runs the sql scripts in the sql_scripts folder - it uses init_done.flag to determine if it has already run
command: /bin/bash -c '/opt/mssql/bin/sqlservr; if [ ! -f /sql_scripts/init_done.flag ]; then apt-get update && apt-get install -y mssql-tools unixodbc-dev; /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P $SA_PASSWORD -i /sql_scripts/db_setup.sql; /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P $SA_PASSWORD -i /sql_scripts/second_db_setup.sql; /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P $SA_PASSWORD -i /sql_scripts/third_db_setup.sql; touch /sql_scripts/init_done.flag; fi; tail -f /dev/null'
ports:
- 1433:1433
volumes:
db-data:
When I run the command docker-compose up -d --build
It gives me a warning in the terminal where I ran the docker-compose command:
WARN[0000] The "SA_PASSWORD" variable is not set. Defaulting to a blank string.
Then when the container boots I see it begin the process without issue. Then I see the following in the container logs:
Logon Error: 18456, Severity: 14, State: 8.
Logon Login failed for user 'sa'. Reason: Password did not match that for the login provided.

You set MSSQL_SA_PASSWORD in your env file. But as far as I see, in your Dockerfile you try to log in with $SA_PASSWORD.
So, either you change MSSQL_SA_PASSWORD to SA_PASSWORD in your env file or you change the Dockerfile (in the "test" and "command" section) from $SA_PASSWORD to $MSSQL_SA_PASSWORD. The second option could be preferred, since the image might additionally require MSSQL_SA_PASSWORD to be correctly set internally. I can't tell you about that specifically, since I don't know the MSSQL image enough.

Related

Initialize SQL Server database in docker container without sqlcmd

I am working on a MacBook Pro with M1 CPU so I can't use the "normal" mssql docker image. I am using azure-sql-edge that doesn't have sqlcmd to initialize the database (create schema, database, login).
I have created a sql script that I would like to run once the container starts but I can't find any alternative to sqlcmd.
Is there any other way to do it?
I had same issue, I used mssql-tools docker image from Microsoft registry.
Sample docker-compose:
---
version: '3.8'
services:
mssql:
image: mcr.microsoft.com/azure-sql-edge:latest
command: /opt/mssql/bin/sqlservr
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: "SA_Passw0rd"
stdin_open: true
ports:
- 1433:1433
networks:
- db_net
sqlcmd:
image: mcr.microsoft.com/mssql-tools:latest
command: /opt/mssql_scripts/run-initialization.sh
stdin_open: true
volumes:
- ./mssql_scripts:/opt/mssql_scripts
networks:
- db_net
networks:
db_net:
name: db_net
To use this docker-compose you need to have a shell script named run-initialization.sh with execute rights inside mssql_scripts folder.
The run-initialization.sh script waits for database to start up and then execute sql commands:
/opt/mssql-tools/bin/sqlcmd -S mssql -U SA -P SA_Passw0rd -d master -Q "SELECT version()"
or if you want to execute from test.sql file:
/opt/mssql-tools/bin/sqlcmd -S mssql -U SA -P SA_Passw0rd -d master -i /opt/mssql_scripts/test.sql
The solution above worked for me using Mac M1 chip, don't need to create a shell script can run the commands direct.
sqlcmd:
image: mcr.microsoft.com/mssql-tools:latest
stdin_open: true
environment:
- MSSQL_SA_PASSWORD=Xxx
- MSSQL_DATABASE=test
- MSSQL_BACKUP="/opt/mssql/test.bak"
volumes:
- ./test_data.bak:/opt/mssql/test.bak
command: /bin/bash -c '/opt/mssql-tools/bin/sqlcmd -S mssql -U sa -P $$MSSQL_SA_PASSWORD -d tempdb -q "EXIT(RESTORE DATABASE $$MSSQL_DATABASE FROM DISK = $$MSSQL_BACKUP)"; wait;'
mssql:
image: mcr.microsoft.com/azure-sql-edge:latest
environment:
- ACCEPT_EULA=Y
- MSSQL_SA_PASSWORD=Xxxx
- MSSQL_DATABASE=test
- MSSQL_SLEEP=7
ports:
- 1433:1433
Since I am starting a new project I looked into this issue again and found a good solution for me.
I found go-sqlcmd, a new implementation of sqlcmd using golang and it's compatible with M1 chips.
So I am running azure-sql-edge as before using docker compose:
version: "3.9"
services:
mssql:
image: mcr.microsoft.com/azure-sql-edge:latest
command: /opt/mssql/bin/sqlservr
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: ${DATABASE_SA_PASSWORD}
stdin_open: true
ports:
- 1433:1433
When the database container is up and in idle I run this bash script (in my case I am reading the environmnet variables from a .NET appsettings.json file):
cat <appsetting.json> | jq -r 'to_entries|map("\(.key)=\(.value|tostring)")|.[]' > temp
# Show env vars
grep -v '^#' temp
# Export env vars
export $(grep -v '^#' temp | xargs)
export SQLCMDPASSWORD=$DATABASE_SA_PASSWORD
sqlcmd -U sa \
-v DATABASE_SCHEMA=$DATABASE_SCHEMA \
-v DATABASE_DB_NAME=$DATABASE_DB_NAME \
-v DATABASE_LOGIN_NAME=$DATABASE_LOGIN_NAME \
-v DATABASE_LOGIN_PASSWORD=$DATABASE_LOGIN_PASSWORD \
-i sql/init-db.sql,sql/init-user.sql
I had to split the database and schema creation in a script, then I create the user and assign it to the database.
The sql scripts, init-db.sql:
USE master
IF NOT EXISTS (SELECT name FROM sys.schemas WHERE name = N'$(DATABASE_SCHEMA)')
BEGIN
EXEC sys.sp_executesql N'CREATE SCHEMA [$(DATABASE_SCHEMA)] AUTHORIZATION [dbo]'
END
IF NOT EXISTS (SELECT name FROM sys.databases WHERE name = N'$(DATABASE_DB_NAME)')
BEGIN
CREATE DATABASE $(DATABASE_DB_NAME)
END
init-user.sql:
USE $(DATABASE_DB_NAME)
IF NOT EXISTS(SELECT principal_id FROM sys.server_principals WHERE name = '$(DATABASE_LOGIN_NAME)') BEGIN
CREATE LOGIN $(DATABASE_LOGIN_NAME)
WITH PASSWORD = '$(DATABASE_LOGIN_PASSWORD)'
END
IF NOT EXISTS(SELECT principal_id FROM sys.database_principals WHERE name = '$(DATABASE_LOGIN_NAME)') BEGIN
CREATE USER $(DATABASE_LOGIN_NAME) FOR LOGIN $(DATABASE_LOGIN_NAME)
END

Is it possible to create a volume with 'Microsoft SQL Server' docker container?

I run the following command on my linux machine, and want to attach a volume, like in Mysql
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=12345678' -p 1433:1433 -d mcr.microsoft.com/mssql/server:2017-latest
You can attach a volume on the Microsoft SQL Server too, as described on the Microsoft Documentation:
docker run -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=12345678' -p 1433:1433 -v <host directory>/data:/var/opt/mssql/data -v <host directory>/log:/var/opt/mssql/log -v <host directory>/secrets:/var/opt/mssql/secrets -d mcr.microsoft.com/mssql/server:2017-latest
Using the above command you mount three folders of the container to a host directory:
<host directory>/data:/var/opt/mssql/data
<host directory>/log:/var/opt/mssql/log
<host directory>/secrets:/var/opt/mssql/secrets
You can also use a data volume container instead of a mounted host directory:
docker run -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=12345678' -p 1433:1433 -v sqlvolume:/var/opt/mssql -d mcr.microsoft.com/mssql/server:2017-latest
You can use this as docker-compose.yaml file:
services:
mssql:
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- 1433:1433
volumes:
- ./data:/var/opt/mssql/data
- ./log:/var/opt/mssql/log
- ./secrets:/var/opt/mssql/secrets
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=123456789
Then, to prevent permission denied error create data,log and secrets folder and change their permission:
mkdir data && sudo chown 10001 data
mkdir log && sudo chown 10001 log
mkdir secrets && sudo chown 10001 secrets
And finally, use docker-compose up -d to start the container.
inspect the file /var/opt/mssql/mssql.conf and you will get the location of all data under [filelocation] section:
[filelocation]
#defaultbackupdir # Default directory for backup files
#defaultdatadir # Default directory for data files
#defaultdumpdir # Default directory for crash dump files
#defaultlogdir # Default directory for log files
#errorlogfile # Error log file location
#masterdatafile # Master database data file location
#masterlogfile # Master database log file location

SQL Server Docker Compose SQLCMD does not execute

I'm currently trying to set up a SQL Server in docker compose
and I want to create the database on build with the RUN instruction. This doesn't work, however when I execute the same command on the running container with sh, it works
my compose file looks like this:
version: "3.7"
services:
mssql:
build: ./mssql
environment:
SA_PASSWORD: "Password12345!"
ACCEPT_EULA: "Y"
container_name: mssqlDB
ports:
- "1433:1433"
restart: always
And here my Dockerfile:
FROM mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04
COPY ./prod.sql /
RUN ./opt/mssql-bin/sqlcmd -S localhost -U SA -P "Password12345!" -Q "Create Database HelloWorld"
CMD ["/opt/mssql/bin/sqlservr"]
This is because the SQL Server instance is not started and you must wait for it.
From the Docker Hub official page of SQL Server there are a link to a GitHub Repository where show how to run a sql script on Docker container.
Below I have re-adapted the GitHub code for you case
initialize.sh
# Typically SQL Server takes about 5-10 seconds to start up
# Wait for the SQL Server to come up (90 sec) You can reduce to 20sec and see
sleep 90s
#run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P password -d master -i prod.sql
entrypoint.sh
#start SQL Server, start the script to create the DB and import the data
/opt/mssql/bin/sqlservr & initialize.sh
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04
COPY ./prod.sql /
# Grant permissions for the import-data script to be executable
RUN chmod +x ./initialize.sh
CMD /bin/bash ./entrypoint.sh
Another solution that I personally made is to run the SQL Server service and wait until the service came up.
create.sh
/opt/mssql-tools/bin/sqlcmd -U sa -P $1 -Q 'CREATE DATABASE [MyNewDatabase]'
/opt/mssql-tools/bin/sqlcmd -U sa -P $1 -d 'MyNewDatabase' -i /src/script.sql
script.sql
CREATE TABLE MyTable (..)
DockerFile
FROM mcr.microsoft.com/mssql/server:2017-latest-ubuntu
EXPOSE 1433
WORKDIR /
COPY ./create.sh /src/
COPY ./script.sql /src/
ENV ACCEPT_EULA Y
ENV SA_PASSWORD P#ssw0rd
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN ( /opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Service Broker manager has started" \
&& /src/create.sh P#ssw0rd \
&& pkill sqlservr

Create SQL Server database from a script in docker

Simple question I hope. I cannot find anything anywhere.
How do you create a database in a Microsoft SQL Server Docker container?
Dockerfile:
I am looking at the following Dockerfile:
FROM microsoft/mssql-server-windows-developer:latest
ENV sa_password ab873jouehxaAGR
WORKDIR /
COPY /db-scripts/create-db.sql .
# here be the existing run commnd in the image microsoft/mssql-server-windows-developer:latest
#CMD ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';", ".\\start -sa_password $env:sa_password -ACCEPT_EULA $env:ACCEPT_EULA -attach_dbs \\\"$env:attach_dbs\\\" -Verbose" ]
RUN (sqlcmd -S localhost -U SA -P ab873jouehxaAGR -i create-db.sql)
docker-compose.yml:
I have put together the following docker compose file:
version: '3.4'
services:
sql.data:
image: ${DOCKER_REGISTRY}myfirst-mssql:latest
container_name: myfirst-mssql_container
build:
context: .
dockerfile: ./Dockerfile
environment:
SA_PASSWORD: ab873jouehxaAGR
ACCEPT_EULA: Y
Bringing it together
I am running the command docker-compose up against the above. And assuming create-db.sql file will simply create a database which is not relevant here to keep things minimal.
Errors
The error I get above is that the login for SA is invalid when it runs the .sql script:
Step 7/7 : RUN (sqlcmd -S localhost -U SA -P ab873jouehxaAGR -i create-db.sql)
---> Running in 2ac5644d0bd9
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : Login failed for user 'SA'..
It looks like this runs before the password has been changed to ab873jouehxaAGR which typically looks like the command from mssql-server-windows-developer:latest.json from inspecting the image in vscode - \start -sa_password $env:sa_password -ACCEPT_EULA $env:ACCEPT_EULA -attach_dbs actually does.
Environment
I am running docker Docker version 18.06.1-ce, build e68fc7a on Windows 10.
Attach or script
I am not specifying attaching a database using the environment variable attach_dbs of which I see in many examples.
I am trying to find a best practice for managing a sql container from a point of view of end to end testing and a lot of articles seem to not cover the data aspect part - ie Development Workflow
I would be interested to hear in comments thoughts on these two approaches in the Docker world.
using following commands can solve your problem
docker-compose up --build -d
version: '3.4'
services:
sql.data:
image: ${DOCKER_REGISTRY}myfirst-mssql:latest
container_name: myfirst-mssql_container
environment:
SA_PASSWORD: ab873jouehxaAGR
ACCEPT_EULA: Y
and after that:
docker exec myfirst-mssql_container sqlcmd
-d master
-S localhost
-U "sa"
-P "ab873jouehxaAGR"
-Q 'select 1'

Docker-compose stops the container after entrypoint is done

I need sql server container with some database. I've prepared the following dockerfile:
FROM microsoft/mssql-server-linux:latest
ENV ACCEPT_EULA Y
ENV SA_PASSWORD yourStrong(!)Password
WORKDIR sqlserver
COPY load_db.sh load_db.sh
COPY /resources/sql/ sql
ENTRYPOINT ./load_db.sh
So it runs load_db.sh:
#!/usr/bin/env bash
SERVER_OUT=/var/log/sqlserver.out
TIMEOUT=90
/opt/mssql/bin/sqlservr &>${SERVER_OUT} &
function server_ready() {
grep -q -F 'Recovery is complete.' ${SERVER_OUT}
}
echo 'Wait until Microsoft SQL Server is up'
for (( i=0; i<${TIMEOUT}; i++ )); do
sleep 1
if server_ready; then
break
fi
done
echo 'Microsoft SQL Server is up'
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P ${SA_PASSWORD} -Q "CREATE DATABASE MarketDataService;"
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P ${SA_PASSWORD} -d MarketDataService -i sql/2018.0100.000000001.Add_Layout_table.sql
To run it I've prepared the followinf docker-compose file:
services:
db:
build:
context: .
dockerfile: Dockerfile-db
ports:
- 1433:1433
When I try to run docker-compose up --build it looks okay.
I have output:
db_1 | Wait until Microsoft SQL Server is up
db_1 | Microsoft SQL Server is up
db_1 |
db_1 | (3 rows affected)
But after it is exited...
amptest_db_1 exited with code 0
To solve it I've tried to add tty: true but it doesn't make sense. The same output. How can I keep alive my container by docker-compose? Here I've found tail -F anything. It works but looks terrible. Is there a better way?
Upd: I've stayed with tail -F '/var/log/sqlserver.out'

Resources