So ... I have 2 SQL Server 2019 instances (CTP2.2) and I have one installed with Polybase in single node config (reference this as SS-A). I have created MASTER KEY in the master of SS-A, and created a DATABASE SCOPED CREDENTIAL in a database on SS-A. When I try to do the following:
CREATE EXTERNAL DATA SOURCE acmeAzureDB WITH
(TYPE = RDBMS,
LOCATION = 'ss2019azure.database.windows.net',
DATABASE_NAME = 'dbAcmeAzure',
CREDENTIAL = acmeAzureCred
);
I get an error
Msg 102, Level 15, State 1, Line 6
Incorrect syntax near 'RDBMS'
I have tried to work with MS SQL Server SMEs without any luck (been working on this for many weeks to no avail).
Any ideas here -- plus a message to Microsoft -- your docs on this are AWFUL!!
You have 2 SQL Server 2019 instances (CTP2.2).
But they are not Azure SQL Database instance.
RDBMS External Data Sources are currently only supported on Azure SQL Database.
-- Elastic Database query only: a remote database on Azure SQL Database as data source
-- (only on Azure SQL Database)
CREATE EXTERNAL DATA SOURCE data_source_name
WITH (
TYPE = RDBMS,
LOCATION = '<server_name>.database.windows.net',
DATABASE_NAME = '<Remote_Database_Name>',
CREDENTIAL = <SQL_Credential>
)
Another way, you can create a linked server for your SQL Server 2019 instance to Azure SQL Database. Then you can query data from the Azure SQL DB as EXTERNAL DATA SOURCE.
To see this official tutorial: How to Create a Linked Server.
Reference blob:Incorrect syntax near 'RDBMS'. When I try to create external data source, Anyone having the same issue?
Hope this helps.
SO - worked with MS today - and success -- you can do a CREATE EXTERNAL DATA SOURCE in SS2019 and point to AZURE SQL -- here is the TSQL I used:
(MASTER KEY ALREADY CREATED)
CREATE DATABASE SCOPED CREDENTIAL acmeCred WITH IDENTITY = 'remoteAdmin', SECRET ='XXXXXXXXX';
go
CREATE EXTERNAL DATA SOURCE AzureDB
WITH (
LOCATION = 'sqlserver://ss2019azure.database.windows.net',
CREDENTIAL = acmeCred
);
go
CREATE EXTERNAL TABLE [dbo].[tblAcmeDataAzure]
(
ID varchar(10)
)
WITH (
LOCATION='dbAcmeAzure.dbo.tblAcmeDataAzure',
DATA_SOURCE=AzureDB
);
go
Related
Trying to create an External Source using Polybase in SQL Server and got this error:
Msg 46505, Level 16, State 1, Line 1 Missing required external DDL option 'type
However, base online examples there was no type when creating the external source.
CREATE EXTERNAL DATA SOURCE test_external_data_source_name with (
LOCATION = 'oracle://10.170.35.249:1522',
credential = credential_name
);
I am creating a Database project for a SQL Database in Azure.
The Target Platform is Microsoft Azure SQL Database V12 which I have set in the project settings.
I then have a post deployment script containing the following code to add the database to an elastic pool
ALTER DATABASE [$(DatabaseName)] MODIFY ( SERVICE_OBJECTIVE = ELASTIC_POOL ( name = mypoolname ) )
Ref: https://learn.microsoft.com/en-us/sql/t-sql/statements/alter-database-azure-sql-database?view=azuresqldb-current#b-moving-a-database-to-a-different-elastic-pool
Because my Post Deployment script is set to build (because it has to as part of being a post deployment script) I get the following error:
Error: SQL72007: The syntax check failed 'Incorrect syntax near
ELASTIC_POOL.' in the batch near:
But I know the syntax is ok, it is obviously not recognising the Azure TSQL.
Is it possible to do this as part of my Visual Studio Database Project?
I'm not sure how you're actually sending the query over to the database server, but in order to bypass a preliminary clientside syntax check you could simply use
exec sp_executesql N'ALTER DATABASE [$(DatabaseName)] MODIFY ( SERVICE_OBJECTIVE = ELASTIC_POOL ( name = mypoolname ) )'
As per https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-query-getting-started-vertical/, it is now possible for one database in Azure SQL to query other Azure SQL databases. For my use case, I plan to have one database serving reference data for other databases, which fits nicely to Topology 1 (vertical sharding).
This is great for a deployed environment, but for local development I typically develop using SQL Server Express. As of SQL Server 2012 Express, the CREATE EXTERNAL DATA SOURCE is not a valid syntax.
Is it possible to also reap the benefit of external data source for local development?
After weighing the feature sets, I decided to differentiate the setup of my local database and Azure SQL.
When local SQL Server database wants to reference an Azure SQL database, it can do so using Linked Server
When a fellow Azure SQL database wants to reference another Azure SQL database, only then it uses external data source
i.e. locally
-- Make a link to the cloud
EXEC sp_addlinkedserver
#server=N'MyExternalServer',
#srvproduct=N'Azure SQL Db',
#provider=N'SQLNCLI',
#datasrc=N'<server address>',
#catalog='<database name>';
GO
EXEC sp_addlinkedsrvlogin
#rmtsrvname = '<server address>',
#useself = 'FALSE',
#locallogin=NULL,
#rmtuser = '<username>',
#rmtpassword = '<password>'
GO
select * from [MyExternalServer].[<database name>].[<schema>].[<table name>]
Whereas for Azure SQL:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<password>';
CREATE DATABASE SCOPED CREDENTIAL ElasticDBQueryCred
WITH IDENTITY = '<username>',
SECRET = '<password>';
CREATE EXTERNAL DATA SOURCE MyElasticDBQueryDataSrc WITH
(TYPE = RDBMS,
LOCATION = '<server>',
DATABASE_NAME = '<database name>',
CREDENTIAL = ElasticDBQueryCred,
) ;
create schema <internalschema>
CREATE EXTERNAL TABLE <internalschema>.<internaltablename>
(
... // list of columns
WITH
( DATA_SOURCE = MyElasticDBQueryDataSrc,
SCHEMA_NAME = <schema>,
OBJECT_NAME = <table name>
)
select * from <internalschema>.<internaltablename>
The challenge now is to make the database scripts be common using both approaches. To reference a table using Linked Server, it has to be addressed using four-part identifier [server].[database].[schema].[tablename]. Contrast this with external data source where it can be addressed just by using [schema].[tablename].
Using inspiration from this question: https://dba.stackexchange.com/questions/74566/sql-server-using-4-part-identifiers-when-database-may-be-on-the-same-server, my approach is to create a synonym on my local database that redirects [schema].[tablename] to [externalserver].[externaldatabase].[externalschema].[tablename].
i.e. locally:
create schema <internalschema>
CREATE SYNONYM <internalschema>.<internaltablename> FOR [MyExternalServer].[<database name>].[<schema>].[<table name>]
After which, the same statement would work for both cases:
select * from <internalschema>.<internaltablename>
EDIT:
One big problem with this approach is that you cannot use wrap your script under distributed transaction, because Azure SQL does not allow DTC.
if we are talking about SQL Server, external data source/table/file format are supported starting from SQL Server 2016 - https://msdn.microsoft.com/en-us/library/dn935022.aspx .
How can pypyodbc connect to linked tables in the .accdb database? Is this possible at all, or is this a limitation of pyodbc?
I need to get data from an MS Acess .accdb database into Python. This works perfectly and I can use pypyodbc to access tables and queries defined inside the .accdb Database. However, the database also has tables linked to an external SQL Server. When accessing such linked tables pypyodbc complains that it cannot connect to the SQL server.
test.accdb contains two tables: Test (local table) and cidb_ain (linked SQL table)
The following Python 3 code is my attempt to access the data:
import pypyodbc as pyodbc
cnxn = pyodbc.connect(driver='Microsoft Access Driver (*.mdb, *.accdb)',
dbq='test.accdb',
readonly=True)
cursor = cnxn.cursor()
# access to the local table works
for row in cursor.execute("select * from Test"):
print(row)
print('----')
# access to the linked table fails
for row in cursor.execute("select * from cidb_ain"):
print(row)
Output:
(1, 'eins', 1)
(2, 'zwei', 2)
(3, 'drei', 3)
----
Traceback (most recent call last):
File "test_02_accdb.py", line 14, in <module>
for row in cursor.execute("select * from cidb_ain"):
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 1605, in execute
self.execdirect(query_string)
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 1631, in execdirect
check_success(self, ret)
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 986, in check_success
ctrl_err(SQL_HANDLE_STMT, ODBC_obj.stmt_h, ret, ODBC_obj.ansi)
File "C:\software\installed\miniconda3\lib\site-packages\pypyodbc.py", line 964, in ctrl_err
raise Error(state,err_text)
pypyodbc.Error: ('HY000', "[HY000] [Microsoft][ODBC-Treiber für Microsoft Access] ODBC-Verbindung zu 'SQL Server Native Client 11.0SQLHOST' fehlgeschlagen.")
The error message roughly translates to "ODBC connection to 'SQL Server Native Client 11.0SQLHOST' failed".
I cannot access the SQL Server through the .accdb database with pypyodbc, but querying the cidb_ain table from within MS Access is possible. Furthermore, I can connect to the SQL Server directly:
cnxn = pyodbc.connect(driver='SQL Server Native Client 11.0',
server='SQLHOST',
trusted_connection='yes',
database='stuffdb')
Considering that (1) MS Access (and Matlab too) can use the information contained in the .accdb file to query the linked tables, and (2) the SQL Server is accessible, I assume the problem is related to pypyodbc. (The way driver name and host name are mangled into 'SQL Server Native Client 11.0SQLHOST' in the error message seems somewhat suspicious, too.)
I have no previous experience with Access, so please be patient and let me know if I omitted important information that seemed unnecessary to me...
First, MS Access is a unique type of database application that is somewhat different than other RDMS's (e.g., SQLite, MySQL, PostgreSQL, Oracle, DB2) as it ships with both a default back-end Jet/ACE SQL Relational Engine (which by the way is not an Access-restricted component but a general Microsoft technology) and a front-end GUI interface and report generator. In essence, Access is a collection of objects.
Linked tables are somewhat a feature of the front-end side of MS Access used to replace the default Jet/ACE database (i.e., local tables) for another backend database, specifically for you SQL Server. Moreover, linked tables are ODBC/OLEDB connections themselves! You had to have used a DSN, Driver, or Provider to even establish and create linked tables in the MS Access file.
Hence, any external client, here being your Python script, that connects to the MS Access database [driver='Microsoft Access Driver (*.mdb, *.accdb)] is actually connecting to the backend Jet/ACE database. Client/script never interacts with frontend objects. In your error Python reads the ODBC connection of the linked table and since the SQL Server Driver/Provider [SQL Server Native Client 11.0SQLHOST] is never called in script, the script fails.
Altogether, to resolve your situation you must connect Python directly to the SQL Server database (and not use MS Access as a medium) to connect to any local tables there, here being cidb_ain. Simply use the connection string of the Access linked table:
#(USING DSN)
db = pypyodbc.connect('DSN=dsn name;')
cur = db.cursor()
cur.execute("SELECT * FROM dbo.cidb_ain")
for row in cur.fetchall()
print(row)
cur.close()
db.close()
# (USING DRIVER)
constr = 'Trusted_Connection=yes;DRIVER={SQL Server};SERVER=servername;' \
'DATABASE=database name;UID=username;PWD=password'
db = pypyodbc.connect(constr)
cur = db.cursor()
cur.execute("SELECT * FROM dbo.cidb_ain")
for row in cur.fetchall()
print(row)
cur.close()
db.close()
Update:
It turns out that the solution to this problem is as simple as setting pyodbc.pooling = False before establishing the connection to the Access database:
import pyodbc
# ... also works with `import pypyodbc as pyodbc`, too
pyodbc.pooling = False # this prevents the error
cnxn = pyodbc.connect(r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ= ... ")
(previous answer)
It appears that neither pypyodbc nor pyodbc can read a SQL Server linked table from an Access database. However, System.Data.Odbc in .NET can do it so IronPython can, too.
To verify, I created a table named [Foods] in SQL Server
id guestId food
-- ------- ----
1 1 pie
2 2 soup
I created an ODBC linked table named [dbo_Foods] in Access which pointed to that table on SQL Server.
I also created a local Access table named [Guests] ...
id firstName
-- ---------
1 Gord
2 Jenn
... and a saved Access query named [qryGuestPreferences] ...
SELECT Guests.firstName, dbo_Foods.food
FROM Guests INNER JOIN dbo_Foods ON Guests.id = dbo_Foods.guestId;
Running the following script in IronPython ...
import clr
import System
clr.AddReference("System.Data")
from System.Data.Odbc import OdbcConnection, OdbcCommand
connectString = (
r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=C:\Users\Public\Database1.accdb;"
)
conn = OdbcConnection(connectString)
conn.Open()
query = """\
SELECT firstName, food
FROM qryGuestPreferences
"""
cmd = OdbcCommand(query, conn)
rdr = cmd.ExecuteReader()
while rdr.Read():
print("{0} likes {1}.".format(rdr["firstName"], rdr["food"]))
conn.Close()
... returns
Gord likes pie.
Jenn likes soup.
We are using a trial version of Azure. We are trying to perform cross server queries from our SQL Server 2012 in-house.
We seem to have our local 2012 linked with Azure. When I go into Server Object -> Linked Servers in management studio, I see our Azure database.
But if I try to open the catalog and tables, I get an error message saying
Reference to database and/or server name in 'Perseus.sys.sp_tables_rowset2' is not supported in this version of SQL Server
** Perseus is the name of our catalog in Azure Sql.
Running a query from local connection:
SELECT * FROM [azureDBServer].[Perseus].[dbo].[accounts]
Tesult is:
OLE DB provider "SQLNCLI11" for linked server "azureDBServer" returned message
"Unspecified error". Msg 40515, Level 16, State 2, Line 1 Reference to database and/or
server name in 'Perseus.sys.sp_tables_info_90_rowset' is not supported in this version of
SQL Server.
This same in house SQL 2012 Server is able to connect to our in-house 2008 by cross server queries and by viewing its structure through Linked Servers.
I know from this article Azure supports Linked Servers.
So I'm lost about what is wrong. Our Admin thinks it may be that we have a Web-Sql account vs a business SQL account. This Azure Web vs Business SQL outdated Stack link implies that SQL version is NOT the problem, but pre-dates when Azure offered Linked Servers.
So, I'm trying to understand if
a) we didn't set up something right to provide SQL Linking?
b) we are limited by trial?
c) are we limited by Web SQL version?
d) anything else?
Need to execute below mentioned three stored procedures to add SQL Azure. Using below these stored procedure I was able to query SQL azure.
EXEC sp_addlinkedserver
#server='PROD',
#srvproduct='',
#provider='sqlncli',
#datasrc='azureserver.database.windows.net',
#location='',
#provstr='',
#catalog='database name'
EXEC sp_addlinkedsrvlogin
#rmtsrvname = 'PROD',
#useself = 'false',
#rmtuser = 'Azure login',
#rmtpassword = 'password'
EXEC sp_serveroption 'PROD', 'rpc out', true
While adding linked server from SQL Management, you are not given option to set default database. So use something like below
EXEC sp_addlinkedserver
#server='name for referring locally', -- here you can specify the name of the linked server
#srvproduct='',
#provider='sqlncli', -- using SQL Server native client
#datasrc='AzureMachineName.database.windows.net', -- add here your server name
#location='',
#provstr='',
#catalog='yourdatabasename'
I figured this works.
Did you actually setup connection to perseus database? By looking at the error message your are sending a query with 3 part or 4 part name to Azure which doesn't work as is in Azure. Please check your query and set it to use 2 part name and only three part name if it is connecting to the same database
This works for me:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'mypassword';
CREATE DATABASE SCOPED CREDENTIAL MySecurity
WITH IDENTITY = 'mylogin',
SECRET = 'mypassword';
CREATE EXTERNAL DATA SOURCE MyDbAccess
WITH (
TYPE=RDBMS,
LOCATION='server name',
DATABASE_NAME='db_name',
CREDENTIAL= MySecurity);
CREATE EXTERNAL TABLE MyExtTable (
[Id] [int] NOT NULL,
[Name] [varchar(20)] NULL)
WITH
(DATA_SOURCE = MyDbAccess);
After that you can just use it:
SELECT * FROM MyExtTable
I got this error and my issue ended up being a type in the database name SERVER.Database.dbo.tableName.