I have just installed MS SQL server on my system and I am having trouble using it from python. It works fine from the Management Studio, and I can see all my tables. I also have an MySQL server installed and it works fine from python using sqlalchemy. I would like now to use MS SQL as well. Here is what I tried:
from sqlalchemy import create_engine
import pyodbc
server = 'NEW-OFFICE\\NEWOFFICE'
database = 'testdb'
username = 'NEW-OFFICE\user'
password = 'password'
DRIVER = "ODBC Driver 17 for SQL Server"
SERVERNAME = "NEWOFFICE"
INSTANCENAME = "\MSSQLSERVER_ZVI"
First I tried:
SQLALCHEMY_DATABASE_URI = "mssql+pyodbc://{username}:{password}#{hostname}/{database}".format(
username=username,
password=password,
hostname=server,
database=database,
)
Second I tried:
engine = create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver={DRIVER}')
Third I tried:
engine = create_engine(
f"mssql+pyodbc://{username}:{password}#{SERVERNAME}{INSTANCENAME}/{database}?driver={DRIVER}", fast_executemany=True
)
Then the next code for all tries is:
db = SQLAlchemy(app)
db.app = app
db.init_app(app)
engine_container = db.get_engine(app)
engine_container.dispose() # to close db connection
db.engine.connect()
hallDB.query.filter_by(client_id = 1).first()
At the last line I get an exception:
(sqlite3.OperationalError) no such table: hall
Of course table "hall" exists and I can query it in the Management Studio.
I am stuck so any help will be mush appreciated.
An Update
The following code did work, but I want the sqlalchemy model approach, which I am not able to use, as listed above.
u = 'DRIVER=ODBC Driver 17 for SQL Server;SERVER=NEW-OFFICE\user;DATABASE=testdb;Trusted_Connection=yes;'
cursor = pyodbc.connect(u).cursor()
cursor.execute("SELECT * FROM hall;")
row = cursor.fetchone()
while row:
print(row[0])
row = cursor.fetchone()
Maybe this will show what needs to be done in order to connect using sqlalchemy.
An Update 2
The error related to sqlite3 was resolved after I saw the following warning
UserWarning: Neither SQLALCHEMY_DATABASE_URI nor SQLALCHEMY_BINDS is set. Defaulting SQLALCHEMY_DATABASE_URI to "sqlite:
I realized that although I set SQLALCHEMY_DATABASE_URI, in the test code, I skipped the line app.config["SQLALCHEMY_DATABASE_URI"] = SQLALCHEMY_DATABASE_URI
After fixing it, and using:
SQLALCHEMY_DATABASE_URI = r'mssql+pyodbc://NEW-OFFICE\user/testdb;driver=ODBC Driver 17 for SQL Server;Trusted_Connection=yes'
or
SQLALCHEMY_DATABASE_URI = r"mssql+pyodbc://NEW-OFFICE\user/testdb?driver=ODBC Driver 17 for SQL Server?trusted_connection=yes"
the error now is:
Message=(pyodbc.InterfaceError) ('IM002', '[IM002] [Microsoft][ODBC
Driver Manager] Data source name not found and no default driver
specified (0) (SQLDriverConnect)')
(Background on this error at: https://sqlalche.me/e/14/rvf5)
And also a warning:
A Warning: No driver name specified; this is expected by PyODBC when using DSN-less connections
Finally after many frustrating hours I found the problem -> the connection staring requires # at the beginning. The proper one is:
mssql+pyodbc://#NEW-OFFICE\user/testdb?driver=ODBC Driver 17 for SQL Server
Related
How can I directly connect MS SQL Server to polars?
The documentation does not list any supported connections but recommends the use of pandas.
Update:
SQL Server Authentication works per answer, but Windows domain authentication is not working. see issue
Ahh, actually MsSQL is supported for loading directly into polars (via the underlying library that does the work, which is connectorx); the documentation is just slightly out of date - I'll take a look and refresh it accordingly.
Here you can connect to MS SQL Server with Polar (connectorx under the hood). Just use a connection string:
import polars as pl
# usually don't store sensitive info in plain text
username = 'my_username'
password = '1234'
server = 'SERVER1'
database = 'db1'
trusted_conn = 'no' # or yes
conn = f'mssql://{username}:{password}#{server}/{database}?driver=SQL+Server&trusted_connection={trusted_conn}'
query = "SELECT * FROM table1"
df = pl.read_sql(query, conn)
I'm new at pyodbc and I'm using the following code to connect my database:
(IDE is VS Code) I'm testing the connection to ensure that an error message is displayed if I fail to connect to the server:
All is fine if the server name is correct, but if I deliberately misspell the server name in order to prove that the connection is not successful, then the code appears to hang. (VSC shows code still running and I need to manually click on Stop), so then I don't appear to reach the try/except statement.
import pyodbc
try:
conn = pyodbc.connect(Trusted_Connection='yes', driver = '{SQL Server}',server = 'xxx-xxxx\SQLEXPRESS' , database = 'master')
except pyodbc.Error as ex:
print(ex)
else:
print('Connected')
print('Done')
Thanks all
Russ
Im trying to create alembic migrations for Azure Synapse DW. Im constantly getting following error :
[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]111214;An attempt to complete a transaction has failed. No corresponding transaction found.
My connection string in alembic.ini file is :
sqlalchemy.url = mssql+pyodbc:///?odbc_connect=Driver={ODBC Driver 17 for SQL Server};Server=tcp:{host},1433;Database={database};Uid=sqladminuser;Pwd={Password};Encrypt=yes;TrustServerCertificate=no;Connection+Timeout=30;
Version im trying to migrate :
def upgrade():
op.create_table(
'test',
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('description', sa.String, nullable=False),
sa.Column('source_type', sa.String, nullable=False),
sa.Column('source_schema', sa.String, nullable=False),
sa.Column('source_entity', sa.String, nullable=False),
)
def downgrade():
op.drop_table('test')
I resolved this by following the github link provided by Gord in the comments. I updated by sqlalchemy.url to
mssql+pyodbc://{user}:{password}#{host/server}:1433/{db}?autocommit=True&driver=ODBC+Driver+17+for+SQL+Server
I'm trying to connect to the SQL Sever database using R but not sure on the details for the query string. I normally use SQL server management studio on SQL Server 2008 and connnect using single sign on. I found the below example
myconn <- odbcDriverConnect(connection="Driver={SQL Server
Native Client 11.0};server=hostname;database=TPCH;
trusted_connection=yes;")
I get the below warning message
Warning messages:
1: In odbcDriverConnect(connection = "Driver={SQL Server \nNative Client 11.0};server=hostname;database=TPCH;\ntrusted_connection=yes;") :
[RODBC] ERROR: state IM002, code 0, message [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
2: In odbcDriverConnect(connection = "Driver={SQL Server \nNative Client 11.0};server=hostname;database=TPCH;\ntrusted_connection=yes;") :
ODBC connection failed
How do I go about finding the specifics i need?
I have done this in the past with an odbc named connection that I've already had in place. In case you don't know, you can create one in windows by typing into the search prompt 'odbc' and selecting "set up data sources". For example - if you named an odbc connection 'con1' you can connect the following way:
con<-odbcConnect('con1') #opening odbc connection
df<-sqlQuery(con, "select *
from ssiD.dbo.HOURLY_SALES
") #querying table
close(con)
This works for me.
library(RODBC)
dbconnection <- odbcDriverConnect("Driver=ODBC Driver 11 for SQL Server;Server=server_name; Database=table_name;Uid=; Pwd=; trusted_connection=yes")
initdata <- sqlQuery(dbconnection,paste("select * from MyTable;"))
odbcClose(channel)
Also, see these links.
RODBC odbcDriverConnect() Connection Error
https://www.simple-talk.com/sql/reporting-services/making-data-analytics-simpler-sql-server-and-r/
The problem is simpler than this. The big clue is the \n in the error message. Something has re-flowed your connection string such that there is now a new-line character in the driver name. That won't match any registered driver name. Pain and suffering then ensues. Make sure your whole connection string is on a single line!
I often use:
driver={SQL Server Native Client 11.0}; ...
and it works really well. Much better than having to rely on pre-defined connection names.
Try another ODBC driver.
In windows press the "windows" button and then type "odbc".
Click the "Data sources (ODBC)" link.
Go to the "Drivers" tab to see the available drivers for SQL Server.
Also - remove the " " spaces after the semicolons in your connection string.
Note - the database property should point to a database name rather than a table name.
This worked for me:
odbcDriverConnect("Driver=SQL Server Native Client 11.0;Server=<IP of server>;Database=<Database Name>;Uid=<SQL username>;Pwd=<SQL password>")
First, you need to install the package 'RSQLServer', and all its dependencies.
Then execute the following command in RStudio, with relevant parameters:
conn <- DBI::dbConnect(RSQLServer::SQLServer(),
server = '<server>',
port = '<port>',
properties = list(
user = '<user>',
password = '<password>'
))
Finally, db_list_tables(conn) gives you the list of tables in the corresponding database.
How can pypyodbc connect to linked tables in the .accdb database? Is this possible at all, or is this a limitation of pyodbc?
I need to get data from an MS Acess .accdb database into Python. This works perfectly and I can use pypyodbc to access tables and queries defined inside the .accdb Database. However, the database also has tables linked to an external SQL Server. When accessing such linked tables pypyodbc complains that it cannot connect to the SQL server.
test.accdb contains two tables: Test (local table) and cidb_ain (linked SQL table)
The following Python 3 code is my attempt to access the data:
import pypyodbc as pyodbc
cnxn = pyodbc.connect(driver='Microsoft Access Driver (*.mdb, *.accdb)',
dbq='test.accdb',
readonly=True)
cursor = cnxn.cursor()
# access to the local table works
for row in cursor.execute("select * from Test"):
print(row)
print('----')
# access to the linked table fails
for row in cursor.execute("select * from cidb_ain"):
print(row)
Output:
(1, 'eins', 1)
(2, 'zwei', 2)
(3, 'drei', 3)
----
Traceback (most recent call last):
File "test_02_accdb.py", line 14, in <module>
for row in cursor.execute("select * from cidb_ain"):
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 1605, in execute
self.execdirect(query_string)
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 1631, in execdirect
check_success(self, ret)
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 986, in check_success
ctrl_err(SQL_HANDLE_STMT, ODBC_obj.stmt_h, ret, ODBC_obj.ansi)
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 964, in ctrl_err
raise Error(state,err_text)
pypyodbc.Error: ('HY000', "[HY000] [Microsoft][ODBC-Treiber für Microsoft Access] ODBC-Verbindung zu 'SQL Server Native Client 11.0SQLHOST' fehlgeschlagen.")
The error message roughly translates to "ODBC connection to 'SQL Server Native Client 11.0SQLHOST' failed".
I cannot access the SQL Server through the .accdb database with pypyodbc, but querying the cidb_ain table from within MS Access is possible. Furthermore, I can connect to the SQL Server directly:
cnxn = pyodbc.connect(driver='SQL Server Native Client 11.0',
server='SQLHOST',
trusted_connection='yes',
database='stuffdb')
Considering that (1) MS Access (and Matlab too) can use the information contained in the .accdb file to query the linked tables, and (2) the SQL Server is accessible, I assume the problem is related to pypyodbc. (The way driver name and host name are mangled into 'SQL Server Native Client 11.0SQLHOST' in the error message seems somewhat suspicious, too.)
I have no previous experience with Access, so please be patient and let me know if I omitted important information that seemed unnecessary to me...
First, MS Access is a unique type of database application that is somewhat different than other RDMS's (e.g., SQLite, MySQL, PostgreSQL, Oracle, DB2) as it ships with both a default back-end Jet/ACE SQL Relational Engine (which by the way is not an Access-restricted component but a general Microsoft technology) and a front-end GUI interface and report generator. In essence, Access is a collection of objects.
Linked tables are somewhat a feature of the front-end side of MS Access used to replace the default Jet/ACE database (i.e., local tables) for another backend database, specifically for you SQL Server. Moreover, linked tables are ODBC/OLEDB connections themselves! You had to have used a DSN, Driver, or Provider to even establish and create linked tables in the MS Access file.
Hence, any external client, here being your Python script, that connects to the MS Access database [driver='Microsoft Access Driver (*.mdb, *.accdb)] is actually connecting to the backend Jet/ACE database. Client/script never interacts with frontend objects. In your error Python reads the ODBC connection of the linked table and since the SQL Server Driver/Provider [SQL Server Native Client 11.0SQLHOST] is never called in script, the script fails.
Altogether, to resolve your situation you must connect Python directly to the SQL Server database (and not use MS Access as a medium) to connect to any local tables there, here being cidb_ain. Simply use the connection string of the Access linked table:
#(USING DSN)
db = pypyodbc.connect('DSN=dsn name;')
cur = db.cursor()
cur.execute("SELECT * FROM dbo.cidb_ain")
for row in cur.fetchall()
print(row)
cur.close()
db.close()
# (USING DRIVER)
constr = 'Trusted_Connection=yes;DRIVER={SQL Server};SERVER=servername;' \
'DATABASE=database name;UID=username;PWD=password'
db = pypyodbc.connect(constr)
cur = db.cursor()
cur.execute("SELECT * FROM dbo.cidb_ain")
for row in cur.fetchall()
print(row)
cur.close()
db.close()
Update:
It turns out that the solution to this problem is as simple as setting pyodbc.pooling = False before establishing the connection to the Access database:
import pyodbc
# ... also works with `import pypyodbc as pyodbc`, too
pyodbc.pooling = False # this prevents the error
cnxn = pyodbc.connect(r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ= ... ")
(previous answer)
It appears that neither pypyodbc nor pyodbc can read a SQL Server linked table from an Access database. However, System.Data.Odbc in .NET can do it so IronPython can, too.
To verify, I created a table named [Foods] in SQL Server
id guestId food
-- ------- ----
1 1 pie
2 2 soup
I created an ODBC linked table named [dbo_Foods] in Access which pointed to that table on SQL Server.
I also created a local Access table named [Guests] ...
id firstName
-- ---------
1 Gord
2 Jenn
... and a saved Access query named [qryGuestPreferences] ...
SELECT Guests.firstName, dbo_Foods.food
FROM Guests INNER JOIN dbo_Foods ON Guests.id = dbo_Foods.guestId;
Running the following script in IronPython ...
import clr
import System
clr.AddReference("System.Data")
from System.Data.Odbc import OdbcConnection, OdbcCommand
connectString = (
r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=C:\Users\Public\Database1.accdb;"
)
conn = OdbcConnection(connectString)
conn.Open()
query = """\
SELECT firstName, food
FROM qryGuestPreferences
"""
cmd = OdbcCommand(query, conn)
rdr = cmd.ExecuteReader()
while rdr.Read():
print("{0} likes {1}.".format(rdr["firstName"], rdr["food"]))
conn.Close()
... returns
Gord likes pie.
Jenn likes soup.