I have a Dockerfile that builds a SQL Server image. I am copying a SQL create table script into the image and an execs.sh that invokes this SQL script using docker exec -it "execs.sh".
What I am seeing is that it errors out if I invoke execs.sh using "docker exec -it" but if I shell into the container and run execs.sh it works. What am I doing wrong?
Dockerfile:
FROM microsoft/mssql-server-linux
COPY create_tables.sql .
COPY execs.sh .
# set environment variables
ENV MSSQL_SA_PASSWORD=P#ssw0rd
ENV ACCEPT_EULA=Y
execs.sh contents (this gets invoked from docker exec -it):
echo "Executing SQL scripts"
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P 'P#ssw0rd' -i ./create_tables.sql
Script build container, run it, and execute commands:
docker build -t mssql_test .
docker run -p 1433:1433 --name mssql2 -d mssql_test
docker exec -it mssql2 "/execs.sh"
docker exec -it mssql2 "ls"
Output:
OCI runtime exec failed: exec failed: container_linux.go:349: starting container process
caused "exec format error": unknown
bin create_tables.sql execs.sh lib mnt root srv usr
boot dev home lib64 opt run sys var
etc install.sh media proc sbin tmp
You're missing an interpreter at the top of the script. When you run that from a shell like bash or sh, it assumes you want to run it as a shell script with the current shell. But when you run it with an OS exec there's no shell assuming how you want to run it. Therefore you want the following in the first line of your script:
#!/bin/sh
And be sure that's done with Linux linefeeds, not Windows linefeeds, else it will give you a file not found.
The problem is, that the command which your are try to force when the docker container starts (in your case: ./create_tables.sql) will be run at the same second the docker starts.
The SQL Server hasnt started yet, so the sql file could not even run against this service.
You have to work with either a great waiting time within your script or build a loop (e.g. check every second if the SQL server is reachable 100 times before exit).
Related
I run SQL Server in container
docker run --network=bridge --name sql29 -h sql29 -it --rm -v h:/sql219data:/var/opt/mssql -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=sQL_19[pwd]" -p 12433:1433 -d mcr.microsoft.com/mssql/server:2019-latest
as described here:
https://learn.microsoft.com/en-us/sql/linux/tutorial-restore-backup-in-sql-server-container?view=sql-server-ver15
I see the active docker
docker ps
But if I try to create the new folder :
docker exec -it sql29 mkdir /var/opt/mssql/bkp22
then the docker disappeared.
docker ps
.........
How to understand: why the docker disappeared? Maybe the volume was mapped incorrectly?
As #Zeitounator commented, the tutorial you linked has a note saying that bind mounts don't work on Windows with the /var/opt/mssql directory:
Host volume mapping for Docker on Windows does not currently support mapping the complete /var/opt/mssql directory. However, you can map a subdirectory, such as /var/opt/mssql/data to your host machine.
You commented that your goal was to keep or restore databases between docker runs. You don't need a bind mount to do that, this is the primary purpose of a volume.
You can create a volume using docker volume create:
docker volume create sql219data
Then run your container using this volume:
docker run -v sql219data:/var/opt/mssql # ...
For debugging purposes you can remove the --rm option from your docker run command so the container won't be removed when stopped. You will then be able to read the logs of the container (even if it stopped):
docker logs sql29
# Then remove it to run the same `docker run` command again
docker rm sql29
docker run # ...
I installed oracle db version 12c in my docker environment.
I used the following command:
docker run -d --name oracle -p 8080:8080 -p 1521:1521 quay.io/maksymbilenko/oracle-12c
I connected to the DB and everything went well but I wanted to enable unified audit.
In order to do that, at first you must shutdown the Database and in all the instructions that I found it says to use sqlplus as following:
sqlplus / as sysoper
SQL> shutdown immediate
SQL> exit
I connected successfully to the DB using the next command:
docker exec -it oracle "bash"
and then I ran the sqlplus command and I received "command not found"
[root#f30cc670f85f /]# sqlplus / as sysoper
bash: sqlplus: command not found
Am I doing it wrong?
What should I do in order to have sqlplus on my oracle DB?
I looked for it and didn't find anything that helped me.
I have mac if its relevant
I think that Docker image is just the database and enough of the OS to run the database. I don't think it includes client software such as SQL*Plus.
You need to have SQL*Plus installed on your Mac. If you haven't already, download the Oracle Instant Client for MacOS including the SQL*Plus extension. Or why not treat yourself and install the new-fangled sqlCL tool? It is easier to install and has all the SQL*Plus capabilities and a whole bunch more features. Find it here.
Whatever client you choose, once it's installed on your Mac you run it like any other app: when prompted for connection you give the string Maksym provides:
system/oracle#//localhost:1521/xe
If you need to connect as sys that would look like this:
sys/oracle#//localhost:1521/xe as sysdba
Sourcing the .bashrc should work to connect to sqlplus as sysdba.
docker-compose exec db bash -c "source /home/oracle/.bashrc; sqlplus sys/Oradoc_db1#ORCLCDB as sysdba;"
with this, you enter the image:
docker exec -it oracle /bin/bash
after that, you can use:
sqlplus sys as sysdba
When using the docker image store/oracle/database-enterprise:12.2.0.1-slim sqlplus and sqlldr tools are only available after the container has started.
You can't do the following in a Dockerfile:
RUN sqlplus sys/password AS SYSDBA #create_database.sql
The container images can be configured to run scripts after setup and on startup. Currently sh and sql extensions are supported.
In your Dockerfile, copy the SQL script into the startup directory:
COPY create_database.sql /opt/oracle/scripts/setup/01_create_database.sql
The database will be created on first startup of the container.
I don't have any experience with docker, but it looks for all the world like you are getting to a bash environment, so there we are on solid ground. The returned error ("bash: sqlplus: command not found") simply means that the executable (sqlplus) was not found in any directory listed in your PATH environment variable, as it exists within your shell environment. You actually need to set three variables: ORACLE_SID needs to be set to the value of your database name. ORACLE_HOME needs to be set to the value of the directory where your oracle binaries are installed. And PATH needs to have $ORACLE_HOME/bin added to it:
export PATH=$ORACLE_HOME/bin:$PATH
Obviously, since you are using the value of ORACLE_HOME in setting PATH, ORACLE_HOME needs to be set first.
For Windows OS:
Type docker ps in command line to show running containers and check container id.
Type docker exec -it container_id //bin/bash
Login via sqlplus command
Or the simplest way
docker exec -it container_id bash -c "source /home/oracle/.bashrc; sqlplus sys/Oradoc_db1#ORCLCDB as sysdba;"
More info is here: https://hub.docker.com/u/cgmmathaw/content/sub-90f0c051-b514-4b7b-a0fe-fc9d6f2172fa
I'm creating a custom Docker image from the existing mcr.microsoft.com/mssql/server:2017-latest-ubuntu by adding some databases with Flyway and then creating backups of the databases within the container.
The idea is to start the Docker container and have it import all databases from the backup files it finds in a certain directory.
If my Docker entrypoint just starts sqlservr and afterwards I exec the shell scripts that restore the DBs from outside, everything works (except that sometimes SQL Server seems to be up, but really isn't).
According to https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-configure-docker?view=sql-server-2017#customcontainer, I should start my shell scripts before sqlservr starts, since it's the foreground process which defines whether the container is up or not.
But it's a chicken/egg situation: How can I execute SQL commands within my shell scripts before SQL Server is running?
I'm aware that normally one would do this with a volume, but this is a test database, and I don't care what happens to the data once the test has finished.
Here's my Docker entrypoint script (I have some custom scripts in /opt/mssql-tools/bin/):
#!/bin/sh
if ! whoami &> /dev/null; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-sqlservr}:x:$(id -u):0:${USER_NAME:-sqlservr} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi
/opt/mssql-tools/bin/restore-databases.sh & sqlservr
Thanks for any hints and suggestions what "the Docker way" should be.
Here is what I do on my side it's not ideal as it involves "sleep" function to wait for SQL server to be up and running before executing the SQL... but it works.
I have the following folder structure:
Dockerfile
init/
database.sql
entrypoint.sh
setup.sh
My Dockerfile calls my entrypoint.sh as follow:
Dockerfile
FROM microsoft/mssql-server-linux:latest
# Project files
ARG PROJECT_DIR=/srv/db-sql-server
RUN mkdir -p $PROJECT_DIR
WORKDIR $PROJECT_DIR
COPY ./init/ ./
# Grant permissions for scripts to be executable
RUN chmod +x $PROJECT_DIR/entrypoint.sh
RUN chmod +x $PROJECT_DIR/setup.sh
CMD ["/bin/bash", "entrypoint.sh"]
My entrypoint.sh starts SQL Server, calls another custom shell script setup.sh to setup the database and then keeps the container alive:
entrypoint.sh
#start SQL Server, start the script to create the DB and import the data, start the app
/opt/mssql/bin/sqlservr & ./setup.sh & sleep infinity & wait
My custom shell script setup.sh waits 30s for SQL Server to be up and running and then executes my SQL file database.sql to create the database / restore the data:
setup.sh
# Wait for SQL Server to be started
sleep 30s
# Run the setup script to create the database
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -d master -i database.sql
I bet sp_procoption could work in your case. This allows you to designate a stored procedure to run each time the instance starts and after all databases are mounted and the relational engine begins listening for commands.
I want to create a shell script which automates the creation and running and creating database in a postgres database using docker.
I want to use the docker postgres official package for postgres in docker.
The script that I use is as follows:
docker network create --subnet=172.18.0.0/16 shared_network;
docker kill postgres_linker;
docker rm postgres_linker;
docker run --name postgres_linker -e POSTGRES_PASSWORD=blahblahblah -d --net shared_network --ip 172.18.0.2 postgres:10-alpine;
docker exec -it postgres_linker psql -U postgres -c "create database linker;";
But when I run this I get the following output without any database being created:
Error response from daemon: network with name shared_network already exists
postgres_linker
postgres_linker
b2a9fd4d6e25b62d60adb05c8b6b653a1b55ec7a869c4728677d6289f5cddd63
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
The first line of this log is OK, the second and third are too, The problem is that psql does not run the command on the postgres container althogh the command I am trying to run is correct. If I run the last command seperately from my shell script:
docker exec -it postgres_linker psql -U postgres -c "create database linker;";
It does not give me error! and it works!
Why is this behavior happening?
I found the solution to my problem.
Unfortunately I thought that when postgres container is run the server is up and running immediately.
It was not true and it takes some time to come up. so I have needed to add some delay.
So the resulting script file should look like this:
docker network create --subnet=172.18.0.0/16 shared_network;
docker kill postgres_linker;
docker rm postgres_linker;
docker run --name postgres_linker -e POSTGRES_PASSWORD=blahblahblah -d --net shared_network --ip 172.18.0.2 postgres:10-alpine;
sleep 5;
docker exec -it postgres_linker psql -U postgres -c "create database linker;";
I'm trying to run a setup script on a Docker SQL Server image
For this I have created a Dockerfile from the mssql image
FROM microsoft/mssql-server-linux:2017-CU8
# Create directory to place app specific files
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Copy setup scripts
COPY entrypoint.sh \
./
RUN chmod +x ./entrypoint.sh
CMD /bin/bash ./entrypoint.sh
In entrypoint.sh I'm starting SQL Server and I want to run some setup commands.
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr &
echo 'Sleeping 20 seconds before running setup script'
sleep 20s
echo 'Starting setup script'
#run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P <MyPassWd> -d master -i setup.sql
echo 'Finished setup script'
When I run this script, the database starts, the setup runs, and after the
setup is finished, the container shuts down.
So I thought something in the script makes the container shut down, therefore I stripped the script down to a bare minimum
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr &
echo 'Sleeping 20 seconds before running setup script'
sleep 20s
That also stops the container after sleep 20s finished.
Moving on...
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr &
Which stops the container right away
And then...
#!/bin/bash
#start SQL Server
/opt/mssql/bin/sqlservr
Now the container runs, but I can't do any initialization
Does someone know how to get this working?
Change the password of the sql server to be complex enough.
docker run -d -p 1433:1433 -e "sa_password=ComplexPW2019!" -e "ACCEPT_EULA=Y" <sqlserverimageid>
Root cause of this issue is PID 1 allocation for docker container.
PID 1 will be allocated to command given in CMD in Dockerfile (in our case ./entrypoint.sh)
Container has a life spam according to PID 1(as soon as PID 1 is stop/killed container will be stopped)
1) In case of /opt/mssql/bin/sqlservr &
a child process ID will be allocated to sqlserver cmd and will be executed in background and as soon as rest of the script is executed, container will stop.
2) In case of /opt/mssql/bin/sqlservr
script will not proceed from here until this execution will complete.
so the solution is to assign PID 1 to CMD /opt/mssql/bin/sqlservr and rest of the script should be executed as child process.
I have done below changes and it is working for me.
in Dockerfile
replace CMD /bin/bash ./entrypoint.sh to CMD exec /bin/bash entrypoint.sh
in entrypoint.sh
#!/bin/bash
#start SQL Server
sh -c "
echo 'Sleeping 20 seconds before running setup script'
sleep 20s
echo 'Starting setup script'
#run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P \"YourStrong!Passw0rd\" -Q
\"ALTER LOGIN SA WITH PASSWORD='NewStrong!Passw0rd'\"
echo 'Finished setup script'
exit
" &
exec /opt/mssql/bin/sqlservr
To create the database on startup, try the approach below.
Dockerfile
FROM mcr.microsoft.com/mssql/server:2019-latest
ENV ACCEPT_EULA Y
ENV DB_NAME test
COPY startup.sh /var/opt/mssql/startup.sh
CMD ["bash", "/var/opt/mssql/startup.sh"]
startup.sh
#!/usr/bin/env bash
if ! [ -f /var/opt/mssql/.initialized ] && [ -n "$DB_NAME" ]; then
while ! </dev/tcp/localhost/1433 2>/dev/null; do
sleep 2
done
echo "Creating $DB_NAME database..."
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "$SA_PASSWORD" -d master \
-Q "CREATE DATABASE $DB_NAME"
touch /var/opt/mssql/.initialized
fi &
/opt/mssql/bin/sqlservr
SQL Server has to be the right most command.
I know it does not make sense as you want SQL Server to run first and then run your scripts to create/restore databases. I guess this is because of the way SQL Server runs on Linux ( Sql server process creates a SQL server process as part of startup).
MSDN documentation makes the order of execution clear at: https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-configure-docker?view=sql-server-ver15#customcontainer
So for your example, you would have to write something like:
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P <MyPassWd> -d master -i setup.sql & /opt/mssql/bin/sqlservr
There is a pull request to allow to run an init SQL script on first time run.
The purpose of this PR is to add into the start.ps1 the ability to check the folder docker-entrypoint-initdb and run all sql scripts inside. Once it is done, the script creates a flag file to avoid running the setup phase on the next startup after a stop.
By mounting a volume from local folder scripts to c:/docker-entrypoint-initdb, the container will execute all .sql scripts files. The volume should be a directory.
E.g.
version: "3.8"
services:
sqlserver:
platform: windows/amd64
environment:
- sa_password=<YourPassword>
- ACCEPT_EULA=Y
image: microsoft/mssql-server-windows-developer
volumes:
- ./dockerfiles/sqlserver/initdb:c:/docker-entrypoint-initdb:ro
ports:
- "1433:1433"