I am trying to connect to SQL Server (on Azure VM) from Azure Databricks. I tried connecting using Python and Scala, but failed to connect.
I get the error:
The TCP/IP connection to the host HOSTNAME, port 1433 has failed.
I used the code in the following link:
https://docs.databricks.com/data/data-sources/sql-databases.html
Both the SQL Server and the Databricks are on the same VNET.
I tried connecting to the SQL Server using "username" and "pwd" and I am able to connect from Management Studio on a Windows laptop.
val jdbcUrl = s"jdbc:sqlserver://${jdbcHostname}:${jdbcPort};database=${jdbcDatabase}"
I have the following details:
Server Name - ABCD\EFGH (this is what I use to connect from Management Studio on Windows)
IP Address
I am using the default port 1433
Fully qualified name - ABCD.lalaland.lala.lala.la
Database name
Username / pwd
What am I missing? What should the hostname be in JDBC URL? Should the instance name be included?
According to the Subnets reqirement, the virtual network must include two subnets dedicated to Azure Databricks: a private subnet and public subnet. As #samuelsaumanchan said, you can try enabling service endpoints on the vnet hosting both your sql server as well as your databricks instance. Service Endpoints enables private IP addresses in the VNet to reach the endpoint of an Azure service without needing a public IP address on the VNet.
Related
Following the Azure VM cluster tutorial provided by Microsoft (https://learn.microsoft.com/en-us/azure/developer/terraform/create-vm-cluster-with-infrastructure),
I have a resource group created in Azure.
Under the Resource group, I have created VN, subnets, etc.
Now, I have two VMs created in the resources group. Both of them have a LoadBalancer on top. I have added NAT rule to LB to connect to individual instance, while they both share the same frontend IP.
I have created a new resource Azure MSSQL server, in the Firewall rule of the SQL server, added the public ip of the LB to the Azure Server
I have now logged in to the individual VMs, and tried to access the SQL server from the VM, but I am unable to connect to it.
I tried normal ping <hostname>
it fails 133 packets transmitted, 0 received, 100% packet loss, time 295ms
Then I try telnet from IP address (ip address was received from ping )
and I get
Trying <ip_address>...
Connected to <ip_address>.
Escape character is '^]'.
When I try to connect using an application, I get this error
com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset ClientConnectionId:8b2f0e3b
-fda0-41af-b8dc-ce80d0760b82
at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.jav
a:3806) ~[mssql-jdbc-11.2.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2109) ~[mssql-jdbc-11.2
.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.prelogin(SQLServerConnection.java
:3517) ~[mssql-jdbc-11.2.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection
.java:3325) ~[mssql-jdbc-11.2.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:29
50) ~[mssql-jdbc-11.2.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnecti
on.java:2790) ~[mssql-jdbc-11.2.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:
1663) ~[mssql-jdbc-11.2.0.jre8.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1064) ~[
mssql-jdbc-11.2.0.jre8.jar:?]
at jet.server.db.core.ConnectorManager$ConnectionPool.£(ConnectorManager.java:712) ~[
JRESServlets.jar:?]
at jet.server.db.core.ConnectorManager$ConnectionPool.connect(ConnectorManager.java:4
32) ~[JRESServlets.jar:?]
at jet.server.db.core.ConnectorManager$ConnectionPool.getConnection(ConnectorManager.
java:504) ~[JRESServlets.jar:?]
at jet.server.db.core.ConnectorManager$ConnectionPool.getConnection(ConnectorManager.
java:487) ~[JRESServlets.jar:?]
at jet.server.db.core.DatabaseImpl.open(DatabaseImpl.java:839) ~[JRESServlets.jar:?]
at jet.server.db.core.GlobalDataService.<init>(GlobalDataService.java:103) ~[JRESServ
lets.jar:?]
at jet.server.db.core.GlobalDataService.init(GlobalDataService.java:37) ~[JRESServlet
s.jar:?]
at jet.server.ServerEnv.K(ServerEnv.java:2322) ~[JRESServlets.jar:?]
How do I make this work?
Azure SQL is only available through TCP port 1433. Check whether firewall is allowed outgoing TCP communication on port 1433.
You can check the connection by using below command:
Test-NetConnection -Port 1433 -ComputerName ServerName.database.windows.net
To resolve the error, make sure to open firewall in the path from Server like Windows Firewall, Corporate Firewall or Azure NSG (Network security group to allow access.
In the Azure SQL Server, select Networking and enable the below option:
Connect to the Azure Virtual Machine and restart SQL Server like below:
Reference:
IP firewall rules - Azure SQL Database | Microsoft Learn
In my Azure environment I have private SQL Server. To access SQL Server and databases I use a private endpoint. When I connect to SQL from VM in the same Virtual Network I have no problem. I have AKS in the same Virtual Network I try to connect to the database from pod but Kubernetes DNS didn't resolve SQL Server FQDN correctly. DNS name resolved to external IP but private SQL didn't have external access.
This is example how SQL Server resolved from VM:
nslookup *****************.database.windows.net
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
*****************.database.windows.net canonical name = *****************.privatelink.database.windows.net.
Name: *****************.privatelink.database.windows.net
Address: 172.0.8.4
This is correct address to Private Endpoint
And how it resolve from pod in AKS cluster:
kubectl exec -it dnsutils -- nslookup *****************.database.windows.net
Server: 10.0.0.10
Address: 10.0.0.10#53
Non-authoritative answer:
*****************.database.windows.net canonical name = *****************.privatelink.database.windows.net.
*****************.privatelink.database.windows.net canonical name = dataslice6.******.database.windows.net.
dataslice6.******.database.windows.net canonical name = dataslice6*******.trafficmanager.net.
dataslice6*******.trafficmanager.net canonical name = cr5.******-a.control.database.windows.net.
Name: cr5.*******-a.control.database.windows.net
Address: 40.78.225.32
How I can set connection pods from AKS to SQL Private Endpoint?
Create a firewall rule on the Azure DB Server with a range of IP addresses of the AKS Cluster nodes.
Create a VNet Rule on the Azure DB Server that allows access from the subnet to the AKS nodes. This is used in Microsoft.Sql VNet Service Endpoint enabled on the cluster subnet.
If Azure SQL Database is restricted to allow only private access, we can use cross-region private endpoints via Azure Private Link for the Azure SQL database or in region private endpoint with Azure Global VNet-peering.
To connect to AKS from SQL Private Endpoint, we will use VNet Peering with Azure SQL Database Private Link.
For more in detail, please refer below links:
https://learn.microsoft.com/en-us/azure/aks/command-invoke
https://arsenvlad.medium.com/aks-workload-accessing-azure-sql-database-in-another-region-cb6fb30545e4
https://argonsys.com/microsoft-cloud/library/private-aks-and-acr-using-private-endpoint-part-1-2/
https://blog.crossplane.io/azure-secure-connectivity-for-aks-azure-db/
I guess your SQL server's private IP is falling in the docker bridge's CIDR range which is found in the Networking menu in your AKS cluster. If it falls in the range, then docker won't allow the request to go out of docker network.
I have SQL server 208 R2 hosted on AWS. Can I access that SQL server through my web application on local machine without VPN?
What connection string we need to give in web.config. So I no need to have SQL server on my local machine.
I can run and test my application without having SQL server on my local machine.
If your SQL Server is publicly accessible, then you can simply connect directly to it. It's a bad practice to expose database servers directly to the internet, however, so don't do this.
If your SQL Server is not publicly accessible, then you can connect to it via an SSH tunnel. You would launch a publicly-accessible EC2 instance, restrict its security groups to allowing SSH from your local IP address, and then use SSH on your local machine to port-forward to the SQL Server via the SSH tunnel. You can then connect your database client to the local port on your machine and that will be forwarded to the SQL server via the SSH tunnel. Here's an example.
The SQL Server will also have to allow inbound connections from your EC2 instance.
Assuming it's the default SQL Server instance listening on port 443, and your EC2 instance has a public IP address, you could do the following:
Enable access to your EC2 instance through port 443 from your local machine public IP.
Configure Windows firewall to allow access through port 443
Ensure TCP/IP protocol is enabled for your SQL Server Instance
Use data source=<the public ip address of your E2 instance> in the connection string.
To enable access to your EC2 instance through port 443 from your local machine public IP do the following:
Go to AWS mangement console.
Go to EC2 Service.
Click on Running Instances
Click on your EC2 instance running SQL Server
Click on one of the security groups listed on the Description tab.
Click on the Inbound tab.
Click on the Edit button
Click on Add Rule button
Select Custom TCP rule on the Type dropdown list.
Enter 443 on Port Range.
Select My IP on Source dropdown list.
Enter "SQL Server from My local IP" on Description text box.
Your public IP address might change over time, so you might need to update the source when this happens.
To configure Windows firewall to allow access through port 443 open an RDP session to your EC2 instance and use Windows Firewall with Advanced Security tool and add the the rule.
To ensure TCP/IP protocol is enabled for your SQL Server Instance do the following:
Open an Remote Desktop session to your EC2 instance.
Open SQL Server Configuration Manager.
Expand SQL Server Network Configuration
Click on Protocols for
Enable TCP/IP protocol if not enabled.
Hope it helps
Throwing error when trying to connect Azure SQL Server on Cisco AnyConnect VPN, error says " 'Run-time error '-2147467259 (80004005)': [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied. "
Adding to the above answer by arun thatham, you need to provide the NAT IP of the VPN (shouldn't be configuring your private IP).
You have to configure the firewall settings in Azure SQL Server to allow client within an ip range to connect to Azure SQL Server.
When ever you get into a VPN your ip address will be in the range between the ip range of your company/organisation network. In that case you need to specify the range of your latest ip address.
Also your VPN network must allow 1433 port in the outbound configuration. Then only the network can connect to your Azure SQL Server.
I'm sure this is out there somewhere but I can't find it and my brain hurts.
I have a VM on Azure. It has SQL Server running on it. I can access it locally on the VM no problem but I would like to manage it from my dev computer via SQL Server Management Studio.
Is this possible and what creds would I use? the VM Login?
Thanks,
R
You need to complete all of the following to achieve what you want:
Add a TCP endpoint for that VM on port 1433 - public & private (you can change the public port if you will).
Configure the SQL Server Instance to listen to TCP.
Enable the incomping TCP port 1433 in the local FireWall rules of the VM.
Enable mixed mode authentication on the SQL Server
Make sure your ISP does not block outgoing port 1433 (a common practice for most of ISVs since 2003' SQL Slammer) - this is overridable with custom Public port for your TCP endpoint.
Than you will be able to connect to the SQL Server in the Azure VM using your local SSMS.
Faced this issue with Azure VM, I can't connect to SQL server.
Completed all steps described in answer above but didn't get success..
It started to work when I set specific port for TCP/IP protocol.
So open SQL Server Configuration Manager, find TCP/IP protocol for your instance, open Properties and set port to 1433 in IPAll section.
I had the same problem, what I should change was:
click on the VM in AzurePortal --> SQL Server configuration --> Change SQL Connectivity to public
then it works for me!
Instead of SSMS you can also use Powershell to check connectivity with SQL server on Azure VM. The detailed powershell script is at the following link -
http://sanganakauthority.blogspot.in/2014/02/connecting-to-sql-server-virtual.html
This saves you from the installation of SSMS on the machine from which you wih to connect to SQL server virtual machine.