Robot Framework: Connect to PostgreSQL Database via SSH Tunnel - database

i'm facing the following problem.
I've got a PostgreSQL database on a remote server. In order to access it, i need to pass through a SSH Tunnel.
So, the steps of my test are:
create the tunneling
Connect to the db
perform query / queries
I'm able to create the tunnel by using SSHLibrary, and it works:
SSHLibrary.Open Connection 10.xxx.xxx.xx
SSHLibrary.Login MySSH_User MySSH_Password delay=1
SSHLibrary.Create Local SSH Tunnel 9191 ${IP_DATABASE} 22
So i go on by using DatabaseLibrary to perform the connection (and i've checked all the fields are right):
DatabaseLibrary.Connect To Database psycopg2 ${DB_NAME} ${USER_DB} ${PASSWORD_DB} ${IP_DATABASE} ${PORT_DB}
but i get the following error:
OperationalError: could not connect to server: Connection timed out (0x0000274C/10060)
Is the server running on host "10.xxx.xxx.xx" and accepting
TCP/IP connections on port 5432?
How can i solve it? I think i need to specify in some way that the database connection must be done by using the tunnel but i don't know how to do it.
Here's the complete test code:
*** Settings ***
Library DatabaseLibrary
Library SSHLibrary
*** Variables ***
${IP_DATABASE} 10.xxx.xxx.xx
${PORT_DB} 5432
${DB_NAME} MyNameDB
${SCHEMA_DATABASE} MySchemaDB
${USER_DB} MyUserDB
${PASSWORD_DB} MyPasswordDB
*** Keywords ***
*** Test Cases ***
Connect To DB Via SSH
SSHLibrary.Open Connection 10.xxx.xxx.xx
SSHLibrary.Login MySSH_User MySSH_Password delay=1
SSHLibrary.Create Local SSH Tunnel 9191 ${IP_DATABASE} 22
Sleep 2s
DatabaseLibrary.Connect To Database psycopg2 ${DB_NAME} ${USER_DB} ${PASSWORD_DB} ${IP_DATABASE} ${PORT_DB}
I would like to do it by using RF, Anyone can help?
Thank's a lot

Ssh tunnels should work in a way that that you establish SSH connection to another host and expose a PORT in localhost that will then tunnel a traffic from that port in localhost to a IP/PORT in the end point of the tunnel ..
So, lets say you have a host at 192.168.100.10 where your robotframework is running and it needs to connect to a database in 192.168.50.100:
If the postgres is accessible via ssh directly:
Connect To DB Via SSH
${IP_DATABASE}= 192.168.50.100
${DB_PORT}= 5432
SSHLibrary.Open Connection ${IP_DATABASE}
SSHLibrary.Login MySSH_User MySSH_Password delay=1
SSHLibrary.Create Local SSH Tunnel ${DB_PORT} ${IP_DATABASE} ${DB_PORT}
DatabaseLibrary.Connect To Database psycopg2 ${DB_NAME} ${USER_DB} ${PASSWORD_DB} 127.0.0.1 ${PORT_DB}
Now, if you have to use ssh jump host, eg, connect to a ssh server that will then connect to another server where postgres is running, all you need to do is change the tunnel ip address where the traffic will be forward to:
Connect To DB Via SSH
${IP_DATABASE}= 192.168.50.100
${IP_SSH_JUMPHOST}= 192.168.50.1
${DB_PORT}= 5432
SSHLibrary.Open Connection ${IP_JUMPHOST}
SSHLibrary.Login MySSH_User MySSH_Password delay=1
SSHLibrary.Create Local SSH Tunnel ${DB_PORT} ${IP_DATABASE} ${DB_PORT}
DatabaseLibrary.Connect To Database psycopg2 ${DB_NAME} ${USER_DB} ${PASSWORD_DB} 127.0.0.1 ${PORT_DB}

Related

Unable to make Remote Connection with Postgresql

I have PostgreSQL running on Ubuntu Server and I want to make remote connection with PostgreSQL running on port 5432.
I've checked if I can ping the public IP of ubuntu server from my machine and that works fine.
Next I've changed two files on ubuntu server first I've changed postgresql.conf which looks as below
#------------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#------------------------------------------------------------------------------
# - Connection Settings -
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5432 # (change requires restart)
Next I've added two lines in pg_hba.conf as below
host all all 0.0.0.0/0 trust
host all all ::/0 trust
Finally I checked if firewall is running by running sudo ufw verbose which outputted inactive.
As per my understanding I've allowed PostgreSQL to accept remote connection and firewall is also not present hence nothing is blocking. Still I get the following error.
psycopg2.OperationalError: connection to server at "XXX.XXX.XXX.XXX", port 5432 failed: Connection timed out (0x0000274C/10060)
Is the server running on that host and accepting TCP/IP connections?
How can I fix this error?
Edit
Although I can ping and ssh to the Ubuntu server using public IP but can not telnet.
I checked if port 5432 is open using this link but it turned out to be closed.

Symfony 2 SQLSTATE Connection refused

I have a problem with connecting my database online.
My site is hosted on one server, and connects to a remote database that is on another server.
But it sends me the following error:
SQLSTATE [HY000] [2002] Connection refused Error
I did, however, put the exact connection information .. I do not understand why the connection is refused ..
parameters.yml:
parameters:
database_host: **.**.**.**
database_port: 3306
database_name: zip
database_driver: pdo_mysql
database_user: root
database_password: null
mailer_transport: smtp
mailer_host:
mailer_user:
mailer_password:
secret: d87fdb6ecf4089df33360b187388e33c34e12f85
I thank you in advance
Make sure the user/password are the right ones. You can use the mysql command line cli to try to connect.

JEE application in Docker accessing local DB port

I am trying to deploy a JEE application in Docker container. The application requires embedded apache derby on port 1527. It works fine when run in regular dev environment. However, when I run inside a Docker container, the connection on port 1527 is refused.
Things to note, I am using 'default' db connection. That means the application is trying to access the database on localhost:1527. Following is the error message:
java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused (Connection refused).
I am wondering what does localhost mean within a Docker container, the host server or the container?
Any suggestions to fix it?

Issue using pg_dumpl

Ok, here's the issue: I want to upload a DB I've got locally and I'm using in localhost, but, when I use:
PGPASSWORD=mypassword pg_dump -Fc --no-acl --no-owner -h localhost -U root app_db > app_db.dump
I always get:
pg_dump: [archiver (db)] connection to database "app_db" failed: could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
I think the problem is that I'm writing localhost instead of my heroku app address, can anyone tell me how can I get the correct address?
You need to have postgresql installed and running

Sqoop connection to MS SQL timeout

I am attempting to connect to Microsoft SQL Server using Sqoop. I have installed the JDBC driver from Microsoft by following the instructions for the Sqoop Connector and the JDBC Driver. Next I attempt to list the databases on the server. I have tried the following commands:
sqoop list-databases --connect 'jdbc:sqlserver://<HOST>' --username <USER> --password <PASS>
sqoop list-databases --connect 'jdbc:sqlserver://<HOST>;username=<USER>;password=<PASS>'
sqoop list-databases --connect 'jdbc:sqlserver://<HOST>;username=<USER>;password=<PASS>' --username <USER> --password <PASS>
Each of these commands produces the same error messages.
13/01/02 10:44:52 ERROR sqoop.ConnFactory: Error loading ManagerFactory information from file <MY SQOOP DIRECTORY>/conf/managers.d/mssqoop-sqlserver: java.io.IOException: the content of connector file must be in form of key=value
at org.apache.sqoop.ConnFactory.addManagersFromFile(ConnFactory.java:219)
at org.apache.sqoop.ConnFactory.loadManagersFromConfDir(ConnFactory.java:294)
at org.apache.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:85)
at org.apache.sqoop.ConnFactory.<init>(ConnFactory.java:62)
at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:36)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:200)
at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:44)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
13/01/02 10:45:08 ERROR manager.CatalogQueryManager: Failed to list databases
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host <HOST>, port 1433 has failed. Error: "connect timed out. Verify the connection properties, check that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port, and that no firewall is blocking TCP connections to the port.".
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:171)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1033)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:817)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:700)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:842)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:221)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:665)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:56)
at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
I have connected to the database using Microsoft SQL Server Management Studio to ensure the database is operation and that that the host/username/password are all correct. Additionally I have ensured that the port is open and that MSSQL is on the other side with the following.
sudo nmap -sS -p 1433 <HOST>
Starting Nmap 5.21 ( http://nmap.org ) at 2013-01-02 11:04 PST
Nmap scan report for <HOST> (<HOST IP>)
Host is up (0.00070s latency).
rDNS record for <HOST IP: <HOST FQDN>
PORT STATE SERVICE
1433/tcp filtered ms-sql-s
Nmap done: 1 IP address (1 host up) scanned in 0.33 seconds
Any suggestions on where I should go from here? I have not been able to find any documentation on this error. Thanks
I am currently attempting to verify that the SQL server is reachable using OSQL from FreeTDS. Will update this post with my findings.
After much searching and talking to many people I have determined that this issue is being caused by the port being blocked. I am still not 100% certain why this error in particular occurred. If I try with an invalid username or password it will successfully tell me that they are invalid. It is just when it comes to making the actual query that the port is restricted. Most likely different ports are used.

Resources