Can't Connect to External SQL Server From Docker Container - sql-server

I've developed a SpringBoot(Java) application that calls out to an external SQL Server on port 1433. The SQL Server instance is located on-premises (not local SQL Server instances). However, it's reachable from my desktop using either IntelliJ or SQL Clients.
I am using the Microsoft SQL Server JDBC connector to communicate with the instances.
If I run the app from IntelliJ all is well, the app can call the SQL Server, execute the command and returns a resultsset.
However, now I'm trying to Dockerize the api app. The container does the usual SpringBoot initialization but when it tries to call the SQL Server I get the following error:
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host myexternalsqlserver.domain, port 1433 has failed. Error: "myexternalsqlserver.domain. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
at com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:285)
at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2434)
at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:659)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2546)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2216)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2067)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1204)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:825)
at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
at com.symetra.SdsApi.SqlConnector.getResultSet(SqlConnector.java:26)
This is my Docker file
FROM openjdk:11.0.4-jre-slim-buster
VOLUME /tmp
COPY target/myapi-1.0-SNAPSHOT.jar app.jar
EXPOSE 8080
EXPOSE 1433
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.profiles.active=docker -jar /app.jar" ]
And these are the Docker commands I use to build and run the container:
docker build -t myapi . && docker run -p 8080:8080 -p 1433:1433 --name myapi myapi "java","-jar","myapi-1.0-SNAPSHOT.jar"
I don't think the mapping of port 1433 is necessary, it's just an experiment. I should be able to communicate outside of my container through to port 1433 on the host. Port 1433 isn't being blocked on the host because I have no problems running this outside the container.
Finally, this is the connection string I'm using:
//Create Connection Url.
String connectionUrl="jdbc:sqlserver://myexternalsqlserver.domain:1433;database=mydb;user=MyUser;password=MyPassword";
I'm not sure what I'm doing wrong here. I'm wondering if I need to set up Docker networking.
Thanks for your help!

It should work.
NB I'm assuming myexternalsqlserver.domain is not what you're using.
One way to test your container is to shell into it (or create a variant) and try resolving the SQL Server's host name:
docker run --interactive --tty openjdk:11.0.4-jre-slim-buster /bin/bash
# then from within the container's shell
apt update && apt install -y dnsutils
nslookup ${SQL_SERVER}
If that succeeds, it's your code.
If not, it's the network.
NB Your container need not publish 1443 (--publish=1433:1433) since it's consuming that port (on the SQL Server) not exposing the port itself.

The same issue bugged me for days and finally was able to solve it by following the steps below.
In my case the SQL server was hosted on Azure managed instance and exposed a private endpoint. The docker was able to resolve the DNS name of the Azure managed instance to a private IP address (as expected ). But this resolved IP was considered by the docker network/bridge as a locally running application because upon inspecting docker's bridge (docker network inspect bridge)The subnet of the bridge was found to be a range of IP addresses where the resolved IP of the SQL server also falls. Hence, the docker assumed the SQL server to be hosted in the same local network and not outside (here enterprise network). In my case the Windows VM I was working was already on the private network of the enterprise.
Solution:
In the docker desktop -> settings -> Docker Engine json file, add a new key value pair
"bip": "<local IP address range which is outside of the resolved sql server IP>".
Restart the docker desktop and the issue should be solved.

Related

Docker Containers cannot communicate with Host Database

I know this question has been asked before, but I haven't managed to solve it from those answers.
System spec: I am running the server on Ubuntu 22.10 docker version is 20.10.16 and docker-compose version is 1.29.2
What I want to achieve: I have nextcloud running as a docker container and have MariaDB installed on the host machine uncontainerized. I want to use a database I created in MariaDB by nextcloud in docker container. But Hostname is always incorrect, and I get the following error.
Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] Connection refused
Troubleshooting I've tried so far
Option 1: I added an extra host to my docker compose file so it can listen to the host system. Here's how my docker compose file like
version: '2'
services:
app:
image: nextcloud
restart: always
ports:
- 8080:80
volumes:
- /home/ritzz/nextcloud:/var/www/html
extra_hosts:
- host.docker.internal:host-gateway
However, when adding database if I use host.docker.internal as hostname I still get the error mentioned above.
Option2: Using docker host IP as database hostname. I used the following command to find out my host ip which was 172.17.0.1
ip addr show docker0
However, again adding 172.17.0.1 or 172.17.0.1:3306 returns with the same error.
Option 3: I saw an option on the Internet to use network_mode: host to make the container use the same network as host. However, since docker container uses port 80 and on my host I can't use port 80. This method won't work for me I assume.
Additional Troubleshooting
I made sure Mariadb is with command sudo systemctl status mariadb as well as checked that it's listening to port 3306 using command sudo netstat -tlnp I also logged in to the database with the user and pass using command sudo mysql -u<username> -p<password> <database> and I can login successfully.
I am at the edge with this. Hopefully someone else can help me out

Docker container can't connect to Redis

I have a Docker container running a C application using hiredis, which should write data to a Redis server exposed at the default address and port running locally on the same Linux device at 127.0.0.1:6379 .
The Redis server is running in a different Docker container. I start this container running, exposing port 6379 as follows : sudo docker run --name redis_container -d -p 6379:6379 40c68ed3a4d2
redsi-cli can connect to this via 127.0.0.1:6379 without issues.
However, no matter what I try, my container which should write to the Redis gets a Redis connection refused error from the C code all the time. This was my last attempt at running the container : sudo docker run --expose=6379 -i 7340dfee8ea5
What exactly am I missing here? Thanks
The C client is running inside a container, that means 127.0.0.1 points to the container itself, not to your host. You should configure the redis client to redis_container:6379 as that is the name you have used when docker run the redis container. More about this here
Besides, both containers need to be inside the same docker network. Use the following command to create a simple network
docker network create my-net
and add --network my-net to both docker run commands (redis client and redis server)
You can read more about docker network here

How to connect sql server using docker on mac?

I am trying to connect to sql server using docker. I have successfully enabled the container using this command
sudo docker run -d --name aakash -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=Maximus6hero#" -p 1434:1434 microsoft/mssql-server-linux
docker ps
3e41723b93d5 microsoft/mssql-server-linux "/opt/mssql/bin/sqls…" About an hour ago Up About an hour 1433/tcp, 0.0.0.0:1434->1434/tcp aakash
It shows that my container is running.
But when i try to connect using db visualizer it throws an error.
The TCP/IP connection to the host localhost, port 1434 has failed. Error: "The driver received an unexpected pre-login response. Verify the connection properties and check that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. This driver can be used only with SQL Server 2000 or later.". ClientConnectionId:6a802ae0-7203-411d-a599-4c1de997d843.
I also have windows pc with me, and i can connect easily using sql server management studio. Even i can connect others pc using that windows pc. But when i try to connect my mac to windows pc using its ip it also gives the same error.
I have enabled tcp/ip connection on windows using sql server configuration manager. But there is no such thing on mac.
You might be getting this error on your MAC because when you run the docker in the background, it uses only 2GB of memory by default which is insufficient to run the SQL server as it needs minimum of 3.25 GB and ideally, we should point it to 4GB of memory. Update the preferences section with above details on your docker. Save and restart the docker and you may check the below steps to see if that might help you for your MAC. I did this using Azure Data Studio and Docker.
Once you have Saved and restart the docker with 4GB of Memory Allocation for docker to run, all you'd need to do is pull the docker image of the sql server and download it. this can be done by below commands on your terminal . FYI, I am using bash commands below:
Command 1:
sudo docker pull mcr.microsoft.com/mssql/server:2017-latest
This will pull the latest vesion docker image and download. Once done, you need to set your SQL authentication on the server for your database. Follow below commands:
Command 2:
sudo docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=<SetYourPasswordHere>' \
-p 1433:1433 --name sql1 \
-d mcr.microsoft.com/mssql/server:2017-latest
This sets your password and uses the port 1433 for SQL server (which is the default port). To confirm if the image has been created and the SQL server is running on docker, execute the below command to check log(s).
Command 3:
docker ps
To check all instances in your history of dockers( i.e. if you already had dockers installed before you are attempting this SQL connection/execution), run the below command and it will give you all the logs of all instances you have created
Command 4:
docker ps -a
or
docker ps -all
Once, you have completed above steps and see that the docker has created SQL instance, you need to go to Azure Data Studio and set the below credentials to access the server that you just created above using Docker.
Server: localhost
Authentication Type: SQL Authentication
Username: sa
Password: <Check Command 2 to see what you entered in the password where it says SetYourPasswordHere>
Hope this helps in your tryst with running SQL server on your MAC. All the Best!

How to communicate between two docker containers (mssql and .net core app) got Connection refused 127.0.0.1:1433

I have a .net core 2.0 project which uses mssql server. I have Created a docker image and container for my .net core 2.0 and running on 9090:9090. I created it like below.
docker container run --name mytestapp --publish 9090:9090 --detach my_.netapp_image_name
and below is my connection string in .net core 2.0 app.
"DefaultConnection": "Server=127.0.0.1;Database=mydatabase;UserId=SA;Password=mydbpassword"
before this, I created a container for mssql server with below,
docker container run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=<YourStrong!Passw0rd>' \
-p 1433:1433 --name sql1 \
-d microsoft/mssql-server-linux:2017-latest
my .net core app has seeds for database. each time it gives me an error says
Unhandled Exception: System.Data.SqlClient.SqlException: A
network-related or instance-specific error occurred while establishing
a connection to SQL Server. The server was not found or was not
accessible. Verify that the instance name is correct and that SQL
Server is configured to allow remote connections. (provider: TCP
Provider, error: 35 - An internal exception was caught) --->
System.AggregateException: One or more errors occurred. (Connection
refused 127.0.0.1:1433) --->
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException:
Connection refused 127.0.0.1:1433
NOTE: this works fine when I run my .net app via IDE(visual studio) and use db as docker mssql container. I ran these two containers separately. then I tried to run using docker-compose, but didn't work.
What am I doing wrong here. hope your help with this.
Containers each have their own network namespace by default. Compose will place all containers on a shared network and set an alias in DNS for the service name. So to connect between containers, all you need to do is point to your service name instead of the 127.0.0.1 (assuming mysql is your service name):
"DefaultConnection": "Server=mysql;Database=mydatabase;UserId=SA;Password=mydbpassword"
This is more portable and handles containers scaling/updating better than to attaching containers to the same network namespace.
So the issue here is the docker sandboxing. For each container that you run you can think of as different virtual environment that has its own host name, IP address and network. While using -p only forwards port form that internal network to host. So while you are running from VS you can point to your db using localhost (127.0.0.1:1433) just because you have exposed that port to host and your application is starting on host directly. When it is runing inside its own container localhost no longer refers to host but rather to that docker environment. To fix this you can run both containers in the same network (--network argument on run) and refer from one to another by host name (--name argument on run).
docker container run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=<YourStrong!Passw0rd>' \
-p 1433:1433 --name sql1 \
-d microsoft/mssql-server-linux:2017-latest \
--name sql_server
docker container run \
--name mytestapp
--publish 9090:9090
--detach my_.netapp_image_name
--network container:sql_server
and in your settings refer to your database as sql_server.
To make this process less painful you can research docker-compose.

Assigning Public IP to SQL Server Docker Image

I am using the latest Docker version (17 CE) on a Mac OSX and I have spun up an instance of SQL Server using the following tutorial: https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-setup-docker
The server was set up successfully and I managed to connect to it from outside the container via an SQL command line utility.
The next step is that I want to be able to connect to this instance from another PC within the same local network by assigning a public IP to the instance.
I have looked through a number of tutorials and it seems that with docker 10 this functionality is now possible, so I am looking to do it the 'right' way rather than the hacky way (pre-docker 10). I have looked through a number of tutorials namely How to assign static public IP to docker container and Assign static IP to Docker container.
I was testing using the ubuntu image to stay true to the example, but it still didn't work. Although the image ran, whenever I tried to ping the assigned IP from the same computer docker is installed on, I was not receiving a request timeout. Also on Kitematic the only host under IP AND PORTS is localhost. The image is being assigned to the custom network (docker network prune while instance is running does not prune my custom network) but I can't seem to discover my instance from the outside.
Commands I am using are
$ docker network create --subnet=172.18.0.0/16 mynet123
$ docker run --net mynet123 --ip 172.18.0.22 -it ubuntu bash
$ ping 172.18.0.22
and for my sql server
$ docker network create --driver=bridge --subnet=192.168.0.0/24 --gateway=192.168.0.1 mynet
$ docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=MyPassword123<>' -p 1433:1433 --ip=192.168.0.10 --net=mynet microsoft/mssql-server-linux
$ ping 192.168.0.10
What am I missing?
Any help would be appreciated.

Resources