Using pymssql to connect to SQL Server in Windows - sql-server

Pymssql -V:2.2.1
Python -V: 3.8
DB: SQL Server 2008 R2
This is my code:
class read_sql(object):
def __init__(self):
# 服务器名
self.server = HOST_24
# 用户名
self.user = SQL_SERVER_USER
# 密码
self.password = SQL_SERVER_PASSWORD
# 数据库名
self.database = PUBMED_DB
# 连接数据库
try:
self.conn = pymssql.connect(self.server, self.user, self.password, self.database)
# 创建cursor缓冲区,用来存放sql语句
self.cursor = self.conn.cursor()
except Exception as e:
print(e)
raise ValueError('数据库链接实例化出错')
def query(self, query_str):
# 输入query_str查询语句,内容返回到cursor缓冲区内
self.cursor.execute(query_str)
# 接收全部的返回结果行.
row = self.cursor.fetchall()
return row
if __name__ == "__main__":
sql_data = read_sql()
row = sql_data.query("SELECT TOP 1 * FROM JinMo_CheckTable order by id desc")
print(row)
This is my first time connecting to SQL Server!
I get this error:
pymssql._mssql.MSSQLDatabaseException: (18456, b"\xe7\x94\xa8\xe6\x88\xb7 'sa' \xe7\x99\xbb\xe5\xbd\x95\xe5\xa4\xb1\xe8\xb4\xa5\xe3\x80\x82DB-Lib error message 20018, severity 14:
General SQL Server error: Check messages from the SQL Server
DB-Lib error message 20002, severity 9:
Adaptive Server connection failed (192.168.0.24)
DB-Lib error message 20002, severity 9:
Adaptive Server connection failed (192.168.0.24)
Please give me some suggestions - thanks !

Related

Databricks and SQL server issue with token

I need your help to create a "permanently" connection from databricks to sql server database in Azure.
I have a code in pyspark to connect to database, using driver "com.microsoft.sqlserver.jdbc.spark" and JAR spark_mssql_connector_2_12_3_0_1_0_0_alpha.jar.
I have created a class to connect to DB is via token
class SQLSpark():
database_name: str = ""
sql_service_name: str = ""
service_principal_id: str = ""
service_principal_secret: str = ""
tenant_id: str = ""
authority: str = ""
state = None
except_error = None
def __init__(self, database_name, service_principal_id, service_principal_secret, tenant_id,
authority, spark, sql_service_name=None):
self.database_name = database_name
self.sql_service_name = sql_service_name
self.service_principal_id = service_principal_id
self.service_principal_secret = service_principal_secret
self.tenant_id = tenant_id
self.authority = authority
self.state = True
self.except_error = ""
self._spark_session = spark
context = adal.AuthenticationContext(self.authority)
token = context.acquire_token_with_client_credentials("https://database.windows.net", self.service_principal_id,
self.service_principal_secret)
self._access_token = token["accessToken"]
server_name = "jdbc:sqlserver://" + self.sql_service_name + ".database.windows.net"
self._url = server_name + ";" + "databaseName=" + self.database_name + ";"
def select_table(self, table, sql_query):
try:
logger.info(f"Reading table {table} in DB {self.database_name} ")
df = self._spark_session.read.format("com.microsoft.sqlserver.jdbc.spark") \
.options(
url=self._url,
databaseName=self.database_name,
accessToken=self._access_token,
hostNameInCertificate="*.database.windows.net",
query=sql_query) \
.load()
self.custom_logger.info(f"Table {table} in database {self.database_name} has been read")
return df
except Exception as ex:
logger.error(f"Failed to read table {table}")
logger.error(ex)
The problem is that I have to process huge data and processes took more that 1h to process and database token expired. Is there a way to refresh the token when I call to select_table method?
Error given is:
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user '<token-identified principal>'. Token is expired.
Full error:
Py4JJavaError: An error occurred while calling o9092.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 59.0 failed 4 times, most recent failure: Lost task 0.3 in stage 59.0 (TID 2611, 10.139.64.5, executor 0): com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user '<token-identified principal>'. Token is expired. ClientConnectionId:009909b8-d779-4df2-b077-59cf4c4b3c73
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:283)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:129)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:37)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:5173)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3810)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:94)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3754)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7225)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3053)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2562)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2216)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2067)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1204)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:825)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$createConnectionFactory$1(JdbcUtils.scala:64)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD.compute(JDBCRDD.scala:272)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:356)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:320)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:356)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:320)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:356)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:320)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
at org.apache.spark.scheduler.Task.doRunTask(Task.scala:144)
at org.apache.spark.scheduler.Task.run(Task.scala:117)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$9(Executor.scala:655)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1581)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:658)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2519)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2466)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2460)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2460)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1152)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1152)
at scala.Option.foreach(Option.scala:407)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1152)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2721)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2668)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2656)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user '<token-identified principal>'. Token is expired. ClientConnectionId:009909b8-d779-4df2-b077-59cf4c4b3c73
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:283)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:129)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:37)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:5173)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3810)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:94)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3754)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7225)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3053)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2562)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2216)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2067)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1204)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:825)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$createConnectionFactory$1(JdbcUtils.scala:64)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD.compute(JDBCRDD.scala:272)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:356)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:320)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:356)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:320)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:356)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:320)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
at org.apache.spark.scheduler.Task.doRunTask(Task.scala:144)
at org.apache.spark.scheduler.Task.run(Task.scala:117)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$9(Executor.scala:655)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1581)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:658)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Couple of things I can think of.
Check if there is an option to provide a refresh URL to Spark. So it can get new token. Similar to this but for your SQL Server instead of ADLS. You'll probably have to use some other API like acquire_token_with_refresh_token() to create the token.
I know some of the token generator implementations allow you to provide a requested expiry period while making the call to create a new token. If your does, then create a token valid for 2-3-6-whatever-you-need hours instead of letting it set expiry to default one hour.
Other option assuming your code is NOT correct. I.e. there is NOT a good reason to create token in __init__(). You should create token near where you use it. I.e.
class SQLSpark():
# ...
def __init__(self, database_name, service_principal_id, service_principal_secret, tenant_id,
authority, spark, sql_service_name=None):
# same as OP, except no token is created and stored in self.token
def select_table(self, table, sql_query):
# ...
# Generate the token closer to it's use.
token = adal.AuthenticationContext(self.authority).acquire_token_with_client_credentials("https://database.windows.net",
self.service_principal_id, self.service_principal_secret)
df = self._spark_session.read.format("com.microsoft.sqlserver.jdbc.spark") \
.options(
# ...
accessToken=token["accessToken"],
query=sql_query) \
.load()
# ...

Pyodbc error in Python How to fix connection

I am trying to Connect to sql server database but I get this error:
File "C:\Python37-32\SqlVersionpr_import.py", line 528, in proceed
with pyodbc.connect('DRIVER={ODBC Driver 17 for SQL
Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password) as myDbConn:
pyodbc.InterfaceError: ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found
and no default driver specified (0) (SQLDriverConnect)')
My code:
server = '***********'
database = '*****'
username = '***********'
password = '***********'
with pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password) as myDbConn:
with myDbConn.cursor(as_dict=True) as cursor:
cursor.execute(''' select * from empInfo where ssn = %s and storeid = %s ;''',(strLastSsn,stono))
emp_rows = cursor.fetchall() or []

SQL IM002 Errors while connecting to SQL Server from perl on heroku

I have a perl application hosted on heroku which needs to connect with some sql server. I am unable to establish connection. It fails with following error:
DBI connect('Driver={ODBC Driver 11 For SQL Server}; Server=****; UID=****; PWD=****','',...) failed: [unixODBC][Driver Manager]Data source name not found, and no default driver specified (SQL-IM002) at work.pl line 12.
This is the code.
work.pl
use strict;
use warnings;
use DBI;
my $DRIVER = '{ODBC Driver 11 for SQL Server}';
my $SERVER = '****';
my $UID = '****';
my $PWD = '****';
my $x = DBI->connect("dbi:ODBC:Driver=$DRIVER; Server=$SERVER; UID=$UID; PWD=$PWD");
Relevant Environment Vars:
LIBRARY_PATH=/app/.platform/vendor/usr/lib64:
LD_LIBRARY_PATH=/app/.platform/vendor/usr/lib64:
PATH=/app/.platform/vendor/usr/bin:/app/vendor/perl/bin:/usr/bin:/bin
LANG=en_US.UTF-8
ODBCSYSINI=/app/.platform/vendor/etc
HOME=/app
PWD=/app
ODBCINI=/app/.platform/vendor/etc/odbc.ini
ODBCHOME=/app/.platform/vendor/etc/
PERL5OPT=-Mlocal::lib=/app/vendor/perl-deps
odbcinst.ini:
Description = ODBC for PostgreSQL
Driver = /usr/lib/psqlodbc.so
Setup = /usr/lib/libodbcpsqlS.so
Driver64 = /usr/lib64/psqlodbc.so
Setup64 = /usr/lib64/libodbcpsqlS.so
FileUsage = 1
[MySQL]
Description = ODBC for MySQL
Driver = /usr/lib/libmyodbc5.so
Setup = /usr/lib/libodbcmyS.so
Driver64 = /usr/lib64/libmyodbc5.so
Setup64 = /usr/lib64/libodbcmyS.so
FileUsage = 1
[ODBC Driver 11 for SQL Server]
Description = Microsoft ODBC Driver 11 for SQL Server
Driver = .platform/vendor/opt/microsoft/msodbcsql/lib64/libmsodbcsql-11.0.so.2270.0
Threading = 1
UsageCount = 1
odbc.ini is empty.
Am I missing something?
Tried everything and found the reason. It should be dbd:ODBC:DRIVER instead of dbd:ODBC:Driver.

R connection to MS SQL

Using R I am trying to connect to MS SQL 2014 on an Azure VM (not windows authentication)
library(RODBC)
conn <- odbcDriverConnect(connection = "Driver=SQL Server;Server=someinternetmachine.cloudapp.net;Database=MyDatabase;Uid=MyUsername;Pwd=MyPassword;")
queryResult <- sqlQuery(conn, "SELECT top 10 * FROM sometable")
With RODBC is there anywhere to do this using just a connection string (no DSN)?
Using the above I get the below errors
Warning messages:
1: In odbcDriverConnect("driver={SQL Server};server=servername\\instancename,port;database=testing;uid=abc;pwd=123456") :
[RODBC] ERROR: state 08001, code 6, message [Microsoft][ODBC SQL Server Driver][DBNETLIB]Specified SQL server not found.
2: In odbcDriverConnect("driver={SQL Server};server=servername\\instancename,port;database=testing;uid=abc;pwd=123456") :
[RODBC] ERROR: state 01000, code 11001, message [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()).
3: In odbcDriverConnect("driver={SQL Server};server=servername\\instancename,port;database=testing;uid=abc;pwd=123456") :
ODBC connection failed
Did you choose public provisioning for the VM in Azure? Additionally, you need to open the ports for SQL Server in Windows Firewall (port number depends on default or named instance). Also, in case of named instance if you are using dynamic ports then SQL Browser also needs to be opened up. More information can be found in the link here.
Managed to get this working with
driver.name <- "SQL Server"
db.name <- "master"
host.name <- "someinternetmachine.cloudapp.net"
port <- ""
server.name <- "someinternetmachine.cloudapp.net"
user.name <- "MyUsername"
pwd <- "MyPassword"
# Use a full connection string to connect
con.text <- paste("DRIVER=", driver.name,
";Database=", db.name,
";Server=", server.name,
";Port=", port,
";PROTOCOL=TCPIP",
";UID=", user.name,
";PWD=", pwd, sep = "")
con1 <- odbcDriverConnect(con.text)
res <- sqlQuery(con1, 'select * from information_schema.tables')
odbcCloseAll()

SQL Server ODBC connection failed

I am trying to connect SQL server using ODBC.
Could some one help interpreting what this error is and how this can be rectified?
Kindly note that there are no password issues as I use the same credentials to connect to the SQL server using Aqua studio.
dbhandle <- odbcDriverConnect('driver={SQL Server};server=SQLBBAQA;database=bbadb;uid = "aaa_bbb_ccc", pwd = "aaabbbccc123&" ')
Warning messages:
1: In odbcDriverConnect("driver={SQL Server};server=SQLBBAQA;database=bbadb;uid = \"aaa_bbb_ccc\", pwd = \"aaabbbccc123&\" ") :
[RODBC] ERROR: state 08001, code 17, message [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied.
2: In odbcDriverConnect("driver={SQL Server};server=SQLBBAQA;database=bbadb;uid = \"aaa_bbb_ccc\", pwd = \"aaabbbccc123&\" ") :
[RODBC] ERROR: state 01000, code 2, message [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()).
3: In odbcDriverConnect("driver={SQL Server};server=SQLBBAQA;database=bbadb;uid = \"aaa_bbb_ccc\", pwd = \"aaabbbccc123&\" ") :
[RODBC] ERROR: state 01S00, code 0, message [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute
4: In odbcDriverConnect("driver={SQL Server};server=SQLBBAQA;database=bbadb;uid = \"aaa_bbb_ccc\", pwd = \"aaabbbccc123&\" ") :
ODBC connection failed
I see multiple mistakes in the connection string:
server=SQLRAPQA should be in the form server=MACHINE\INSTANCE. Use server=.\SQLRAPQA if the instance is located on the same machine.
Remove all whitespaces.
Use ; as separator, not ,.
As referenced by zx8754, in RODBC odbcDriverConnect() Connection Error it is shown that a connection string should look like:
'driver={SQL Server};server=servername\\instancename,port;database=testing;uid=abc;pwd=123456' . Note the double occurences of \, this seems to be rodbc specific.

Resources