I am trying to create a connection to an oracle db instance (oracle:thin) using Airflow.
According to their documentation I entered my hostname followed by port number and SID:
Host: example.com:1524/sid
filled other fields as:
Conn Type: Oracle
Schema: username ( documentation says: use your username for schema )
Login: username
Password: * * *
After connection is setup, it gives the save error code for every query that I tried to execute (ORA-12514).
It seems like oracle doesn't let airflow to connect:
ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Has someone experienced the same problem before. I mean connecting to a database shouldn't be a problem for a big platform like this. Or I am probably doing something wrong. Thanks
Version: Airflow v1.7.0, Oracle11g
EDIT:
I am using the same hostname which I use in Oracle SQLDeveloper client:
After digging into the source code, this is what finally how it worked for me:
Conn Type: Oracle
Host: example.com
schema: username
login: username
port: port number
extra: {"sid": "my sid", "dsn": "example.com"}
You have a problem in your connection settings, either your setting is not loading properly to the oracle hook or you are missing a python package that save/load your connection settings. You can test it by hard coding your credentials.
https://github.com/airbnb/airflow/blob/master/airflow/hooks/oracle_hook.py
conn = self.get_connection(self.oracle_conn_id)
dsn = conn.extra_dejson.get('dsn', None)
sid = conn.extra_dejson.get('sid', None)
service_name = conn.extra_dejson.get('service_name', None)
if dsn and sid and not service_name:
dsn = cx_Oracle.makedsn(dsn, conn.port, sid)
conn = cx_Oracle.connect(conn.login, conn.password, dsn=dsn)
elif dsn and service_name and not sid:
dsn = cx_Oracle.makedsn(dsn, conn.port, service_name=service_name)
conn = cx_Oracle.connect(conn.login, conn.password, dsn=dsn)
else:
conn = cx_Oracle.connect(conn.login, conn.password, conn.host)
for service name usage, if you leave (port, schema and extra) empty, you can put the full oracle connection descriptor under Host:
(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = mysidname)))
this worked for me in extra field
{ "dsn":"192.168.x.x" , "service_name":"some.service.name" }
I get from https://github.com/apache/airflow/blob/master/airflow/hooks/oracle_hook.py#L49
If anyone just does not see the connection in the Ad hoc query dropdown - you need to install the adapter: pip install cx_Oracle on the airflow server.
Related
How can I directly connect MS SQL Server to polars?
The documentation does not list any supported connections but recommends the use of pandas.
Update:
SQL Server Authentication works per answer, but Windows domain authentication is not working. see issue
Ahh, actually MsSQL is supported for loading directly into polars (via the underlying library that does the work, which is connectorx); the documentation is just slightly out of date - I'll take a look and refresh it accordingly.
Here you can connect to MS SQL Server with Polar (connectorx under the hood). Just use a connection string:
import polars as pl
# usually don't store sensitive info in plain text
username = 'my_username'
password = '1234'
server = 'SERVER1'
database = 'db1'
trusted_conn = 'no' # or yes
conn = f'mssql://{username}:{password}#{server}/{database}?driver=SQL+Server&trusted_connection={trusted_conn}'
query = "SELECT * FROM table1"
df = pl.read_sql(query, conn)
This is the error i get when i connect to snowflake via python?
OperationalError: 250003: Failed to execute request: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)
I connect using:
ctx = snowflake.connector.connect(
user='JoeBloggs',
password='pwd',
account='JoeBloggs',
database='DEV_DATA'
)
do i need to feed in other paramters such as port, host, etc how did i find what these are?
I think your value for 'account' needs to be modified. It looks like you're using your username there, but it should be the Snowflake account. This should be the portion of the URL that you connect directly to that precedes the snowflakecomputing.com portion. For example, 'xy12345.east-us-2.azure'.
My initial thoughts are that the error indicates a firewall or proxy issue. In particular, a proxy might intercept Snowflake's SSL certificate and replace it with their own. The best way to resolve this is to ensure the certificate is trusted in the proxy and the proxy is configured as per Snowflake's documentation so that the Snowflake certificate can pass through.
The documentation below has more information on using a proxy with SnowSQL. You can pass along the error with issuer details to your network engineer and can request to whitelist the required URLs (documentation also below outlining the whitelisting requirements). You can use the SYSTEM$WHITELIST function to get all the URLs to whitelist in a proxy or firewall for your account.
https://docs.snowflake.net/manuals/user-guide/snowsql-start.html#using-a-proxy-server
https://docs.snowflake.net/manuals/user-guide/hostname-whitelist.html
First, install Snowflake python connector .pip3 install snowflake-python-connector.
Can you try with code below:
------------------------------------------------------
import snowflake.connector
PASSWORD = '*****'
USER = '<UNAME>'
ACCOUNT = '<ACCNTNAME>'
WAREHOUSE = '<WHNAME>'
DATABASE = '<DBNAME>'
SCHEMA = 'PUBLIC'
print("Connecting...")
con = snowflake.connector.connect(
user=USER,
password=PASSWORD,
account=ACCOUNT,
warehouse=WAREHOUSE,
database=DATABASE,
schema=SCHEMA
)
con.cursor().execute("USE WAREHOUSE " + WAREHOUSE)
con.cursor().execute("USE DATABASE " + DATABASE)
try:
result = con.cursor().execute("Select * from <TABLENAME>")
result_list = result.fetchall()
print(result_list)
finally:
con.cursor().close()
con.cursor().close()
---------------------------------------------------
I am using sqlserver express , i want to do a connection from groovy and show the data on the console now i have an error and its not connecting. I decided to show what i have on here so we can debug together,
Code looks like this :
package com.connectgroovy
import groovy.sql.Sql
class GroovyConnectMssqlExample {
static main(args)
{
def conn = Sql.newInstance("jdbc:sqlserver://localhost:1433","DESKTOP-V2G35NU\\SQLEXPRESS","com.microsoft.sqlserver.jdbc.SQLServerDriver")
conn.eachRow("select * from [bonitasoft_jsonTest].[dbo].[people_data]"){
println "${it.fullname} | ${it.perdiem}"
}
}
}
I get this as an exception error :
Caught: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'DESKTOP-V2G35NU\SQLEXPRESS'.
com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'DESKTOP-V2G35NU\SQLEXPRESS'.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:197)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:246)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:83)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:2529)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1905)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:41)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:1893)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1045)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:817)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:700)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:842)
at com.connectgroovy.GroovyConnectMssqlExample.main(GroovyConnectMssqlExample.groovy:9)
What am i not doing right?
PS
I put the sqljdbc4.jar as a referenced Library. in eclipse
Connection URL - The url typically contains which server to connect to, what port and which database schema to work with. Example: "jdbc:mysql://localhost:3306/test"
Username and password - The credentials for authenticating the connection. Example: username = user and password = password.
JDBC Driver Class - The JDBC driver class to use to connect to the database. Example for MySQL case: "com.microsoft.sqlserver.jdbc.SQLServerDriver"
def sql = Sql.newInstance("jdbc:sqlserver://localhost:1433/test",
"user", "password", "com.microsoft.sqlserver.jdbc.SQLServerDriver")
For Window Authentication
def sql =Sql.newInstance("jdbc:sqlserver://localhost;Database=myDB;integratedSecurity=true","com.microsoft.sqlserver.jdbc.SQLServerDriver")
I'm using Weblogic and I set a JDBC driver to allow a service to get a connection to the db. This DB is also reachable by other clients for other purposes.What I want to do now, it's to make the JDBC connections "SHARED" with the other clients that connect to the database.
I read that the follow syntax let me achieve this:
jdbc:oracle:thin:<USER>/#(PROTOCOL = TCP)(HOST = <HOST>)(PORT = <PORT>)))(CONNECT_DATA = (SID = <SID>) (SERVER = SHARED)))
and would let me have the JDBC connections shared with the others regular connections.
Is it true ?
Where to set such parameters in weblogic ?
I'm currently using the JDBC driver ( SERVICES->JDBC->DATA SOURCES-> connection pool (tab) ) configured in the following manner:
URL:
jdbc:bea:oracle://[dbIP]:1521
Driver Class Name:
weblogic.jdbc.oracle.OracleDriver
Properties:
user=ETL
portNumber=1521
SID=LIVDEV1
serverName=[dbIP]
Thanks
The URL of Oracle should be jdbc:oracle:thin:#[host][:port]:SID.
No need to set these parameters in properties again, but you need to set the user
And for password, WebLogic already provides a standalone field for it. It will be encrypted by WebLogic.
You can reference http://download.oracle.com/docs/cd/E13222_01/wls/docs81/jdbc/programming.html#1053561
I am trying to use MS access to connect to a Oracle database.
I keep on getting the following error message:
ORA-12154: TSN- could not resolve the connect identifier secified
The Oracle Drivers OracleClient10g can verify that the database server exists.
I have a section in my tsnnames.ora file that looks like this:
UBASEP10G =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = bxxx-xxx.yyyy.com)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = UBASE)
)
)
per my attempts to get this error resolves I added this to the sqlnet.ora file:
NAMES.DIRECTORY_PATH= (HOSTNAME, ONAMES, TNSNAMES,LDAP,EZCONNECT)
When using the Windows ODBC driver configuration utility it asks for the following
following information
DATA SOURCE NAME : MYSOURCE NAME
TSN SERVICE NAME:UBASEP10G
USERID:MYUSERID
any suggestions ?????
I don't have Access, but using Excel 2007, I had to do the following:
Open ODBC Administrator (in the Administrator Control Panel)
For either User DSN or System DSN, click Add...
Select "Oracle in OraDb10g_home1" as the driver
In the Oracle ODBC Driver Configuration, I entered:
Data Source Name: myOracleDsn
Description: This is my DSN for my Oracle Database
TNS Service Name: oratns
User ID: scott
Click Test Connection, and enter "tiger" for the password. Obviously, I'm connecting to my scott/tiger sample Oracle database. Also, when I installed Oracle, I picked oratns as my TNS name. The other two values above are arbitrary -- you'll see them in dialogs in Excel.
If the test works, click OK.
Go to Excel and make a new connection to a DSN. The dialog will show "myOracleDsn" as an option. Select it, enter "tiger" for the password again, and you should get a working connection.
Try changing (CONNECT_DATA = (SERVICE_NAME = UBASE) ) to (CONNECT_DATA = (SID = UBASE) ) in your TNSNAMES.ora file.
ServiceName and SID aren't necessarily the same and consequently aren't always interchangeable.
The SERVICENAME parameter refers to a particular GLOBAL_DBNAME specified in the listener.ora file on the database server. It's an alias for an instance on the server. You can have multiple servicenames on a server referring to the same SID. The SID parameter refers to a particular instance on that server.
The advantage of using servicename on the client side is that the DBA can change the actual instance being referenced by a servicename transparently to the clients using that name. I can have this on the server listener.ora file:
(SID_DESC =
(GLOBAL_DBNAME = THESERVICE)
(ORACLE_HOME = d:\oracle\10.2.0_DB)
(SID_NAME = SID1)
Later, I can change the actual database being referenced by switching the listener.ora configuration:
(SID_DESC =
(GLOBAL_DBNAME = THESERVICE)
(ORACLE_HOME = d:\oracle\10.2.0_DB)
(SID_NAME = SID2)
and nobody's the wiser on the client side. No changes were necessary in the tnsnames.ora files on the clients.
Can you log in to the database in question via SQL*Plus? Doing this from another machine with a working connection (or the DB server itself) is fine also.
If so, run this:
select value from v$parameter where name='service_names';
In your TNSNAMES.ORA, use one of the values listed there for the SERVICE_NAME.
In you ODBC connection, all you'll need is to set the TNS Service Name to the name you used above, "UBASEP10G"
Let's back up to square one. Open a command window and connect to your database:
sqlplus myuserid/mypassword#UBASEP10G
Does this connect successfully?
Since the answer is no, is there a way you CAN connect successfully to this database? BQ is correct, your problem is with the servicename of UBASE. You need to determine what the listener on the server thinks the name of that database is. Do you have access to the server? Can you execute the command "lsnrctl status" on the server? This will tell you the services that are registered with the listener, and look something like this:
Services Summary...
Service "UBASE" has 1 instance(s).
Instance "UBASE", status READY, has 1 handler(s) for this service...
Try tnsping and report your results.
Bad:
C:\>tnsping notreal.world
TNS Ping Utility for 32-bit Windows: Version 9.2.0.5.0 - Production on 29-OCT-2008 15:56:47
Copyright (c) 1997 Oracle Corporation. All rights reserved.
Used parameter files:
C:\oracle\ora92\network\admin\sqlnet.ora
TNS-03505: Failed to resolve name
Good:
O:\>tnsping real.world
TNS Ping Utility for 32-bit Windows: Version 9.2.0.5.0 - Production on 29-OCT-2008 15:57:42
Copyright (c) 1997 Oracle Corporation. All rights reserved.
Used parameter files:
C:\oracle\ora92\network\admin\sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)
(HOST = DBSERVER.DOMAIN.COM)(PORT = 1521)) (LOAD_BALANCE = YES) (FAILOVER = YES))
(CONNECT_DATA = (SERVICE_NAME = REAL.WORLD)
(FAILOVER_MODE = (TYPE = SELECT) (METHOD = BASIC) (RETRIES = 10) (DELAY = 3))))
OK (40 msec)