When obtaining an image grabde with sql server linked servers from PostgreSQL, I get the following error: OLE DB provider 'MSDASQL' for linked server 'bd_acceso_ruisegip' returned data that does not match expected data length for column '[MSDASQL] . fot_imagen '. The data length (maximum) expected is 255 and the data returned is 38471.
Don't know if you were dealing with a bytea column but I was having the same problem. Found the answer in the configuring of the postrgres ODBC system dsn. Under the Options/Datasource-page 2 there is a option for bytea as LO. Clicked that and now it works like a champ.
I found a similar issue when replicating some Forum data from PostgreSQL to MSSQL, using the PostgreSQL 64-bit driver and a Linked Server (.
When I coded like this:...
select * into Post from OpenQuery(PostgreSQL_Test1, 'select * From public.post')
... the MSSQL table defaulted to a column size of nvarchar(4000).
My fix: First, run it once with a small limit on the number of rows copied:
select * into Post from OpenQuery(PostgreSQL_Test1, 'select * From public.post limit 10')
Next, right-click on the local Post table. Choose "Script table as drop and create"
In the create script, replace the size of the offending column with VARCHAR(MAX)
Next, create the table.
Then use:
Insert Post select * from OpenQuery(PostgreSQL_Test1, 'select * From public.post')
Hope that helps.
Your mileage may vary.
Related
I have created a view that includes a view from a linked server and I am inserting into a table. when I query the table it returns no data when the where statement is referencing data from the linkserver table. It does however pull all the data when not using the where statement. why is this for I do not have much experience working with LinkedServers.
Here is code I used to build my view i am using to populate my table.
ALTER VIEW [dbo].[weightsDashboard]
AS
SELECT *
FROM [dbo].[weightsYak]
UNION ALL
SELECT *
FROM OPENQUERY([10.3.50.62\AGJET], 'SELECT * FROM [CheckWeigher].
[dbo].[weightsSs]')
Then I ran this to create the dataset I want to report off of.
select *
into Dataset_SaleSummary
from [dbo].[weightsDashboard]
Thanks in advance.
The root cause Would be driver. Use ODBC instead of OLE DB.If you are trying to connect oracle database from sql server
Delete the linked server from SQL Server
Right click on the "Linked Servers" folder and select "New Linked Server..."
Linked Server: enter anything..this will be the name of your new linked server
Provider: Select "Oracle Provider for OLE DB"
Product Name: enter "Oracle" (without the double quotes)
Data Source: enter the alias from your TNSNAMES.ORA file.
Provider String: leave it blank
Location: leave it blank
Catalog: leave it blank
Now go to the "Security" tab, and click the last radio button that says "Be made using this security context:" and enter the username & password for your connection.
I had exact same problem with an SQL 2014 getting data from SQL 2000 through OPENQUERY. Because ODBC compatibility problem, I had to keep generic OLE DB for ODBC driver. Moreover, the problem was only with SQL non-admin account. So finally, the solution I found was to add SET ROWCOUNT 0:
SELECT * FROM OPENQUERY(DWH_LINK, 'SET ROWCOUNT 0 SELECT * FROM TABLEA ')
It seems the rowcount might been change somewhere through the SQL procedure (or for this user session), so setting it to 0 force it to return "all rows".
There are plenty of examples around showing how to use OPENROWSET and OPENDATASOURCE to open queries against tables on remote SQL servers, and to get data from files on other servers. Unfortunately, none of them tell me how to do the specific thing that I need to do, which is use one of these commands to get either a .csv or .xlsx Excel file from a remote server using the server IP address and windows login where the server has no SQL Server instance installed - only either IIS7 or 8.
It would be great if Microsoft documentation didn't omit basic examples of how to use their tools in the most likely of ways. It takes no real intelligence to try 40 odd different combinations of parameters list: it's just an inefficient waste of time. Developers have more important things to do than waste hours trying to discover some 'secret knowledge' recipe which is really just a not-properly-documented variant of the command parameters, and pretending that this is something to do with being clever.
So to assuage my philosophical angst, could someone please provide an example of how to use either OPENROWSET or OPENDATASOURCE to get/select content from either a .csv file or an excel spreadsheet using the remote server IP address XXX.XXX.XXX.XXX, a port number if it should be included, the file system path correctly appended to that or included in the command, and the correct arrangement of the username and password parameters. There is no SQL server instance on the remote server - just IIS and a website with windows auth. If you have an example that works with SQL server instance on the remote server, that will do (although I think some of the REMOTEDATASOURCE examples cover this already) but I REALLY want an example where I don't have to have an SQL server instance on the remote web server.
I know that you can use linked servers for this, but there is some extra baggage to put on the remote server containing the file to access, and in my case the server containing the excel or .csv text file will not even have a SQL Server instance on it - only IIS and a website.
(Also - can someone confirm whether you can use linked servers with NO SQL Server or other database server instance on the remote server with the desired text data file? Is it linked database servers only, or linked servers where the remote server can just be a windows and web server with no SQL server installed on it?)
This example is handy, but does not tell me if SERVERPROPERTY('MachineName') AS nvarchar(128))
can contain an IP address of a remote windows server with no SQL server instance on it, and it is not about accessing text files.
This example comes blisteringly close, but there does not seem to be a positive outcome in the post, and there is no text file involved - only a DB query so the remote system has SQL server installed?:
SELECT *
FROM OPENROWSET('PACISOleDb', '192.168.200.172';'admin';'admin', 'SELECT * FROM DB')
Existing examples that are good but don't help much:
SELECT *
FROM OPENROWSET('SQLNCLI',
'DRIVER={SQL Server};SERVER=YourServer;UID=UserID;PWD=Password',
'select * from sys.sysobjects')
(Source)
SELECT ContactName, CompanyName, ContactTitle
FROM OPENDATASOURCE('SQLOLEDB',
'Data Source=ServerName;User ID=MyUID;Password=MyPass' ).Northwind.dbo.Customers
(source)
-- SQL Server 2012
SELECT
*
FROM
OPENDATASOURCE ('SQLNCLI11',
'Data Source=SQLInstanceName;Catalog=DBName;User ID=SQLLogin;Password=Password;').DBName.SchemaName.TableName
SELECT *
FROM OPENROWSET('SQLNCLI11',
'DRIVER={SQL Server};SERVER=SQLInstanceName;UID=SQLLogin;PWD=Password',
'select * from DBName..TableName')
--Access DB
SELECT * FROM OPENDATASOURCE ('Microsoft.ACE.OLEDB.12.0',
'Data Source=D:\MyDB\MyAccessDB.accdb')...TableName
(source)
SELECT *
FROM OPENROWSET('SQLNCLI',
'DRIVER={SQL Server};SERVER=MyServer;UID=MyUserID;PWD=MyCleverPassword',
'select ##ServerName')
(source. I take it that {SQL Server} here has to be a SQL server name - not a remote IP Address? It is not clear.)
SELECT *
FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=C:\test.xls;HDR=No',
'SELECT * FROM [Sheet1$]') b
(source)
--Excel 2007-2010
SELECT * --INTO #productlist
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml;HDR=YES;Database=C:\temp\Products.xlsx',
'SELECT * FROM [ProductList$]');
--Excel 97-2003
SELECT * --INTO #productlist
FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;HDR=YES;Database=C:\temp\Products.xls',
'select * from [ProductList$]');
(source. Again - no remote server, no IP server address, no credentials)
Lots to chose from here. GREAT blog post, but not useful for my specific ends:
SELECT *
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml;
Database=C:\DataFiles\EmployeeData1.xlsx',
[vEmployee$]);
SELECT *
FROM OPENDATASOURCE('Microsoft.ACE.OLEDB.12.0',
'Data Source=C:\DataFiles\EmployeeData1.xlsx;
Extended Properties=Excel 12.0 Xml')...[vEmployee$];
SELECT *
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml; HDR=YES;
Database=C:\DataFiles\EmployeeData1.xlsx',
[vEmployee$]);
SELECT *
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml; HDR=NO;
Database=C:\DataFiles\EmployeeData1.xlsx',
[vEmployee$]);
SELECT * INTO EmployeeInfo3
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0 Xml; HDR=YES; IMEX=1;
Database=C:\DataFiles\EmployeeData1.xlsx',
[vEmployee$]);
(source)
The MS openrowset documentation has an '|' in the command schema suggesting this cannot be done remotely:
SELECT *
FROM OPENROWSET(BULK N'D:\XChange\test-csv.csv',
FORMATFILE = N'D:\XChange\test-csv.fmt',
FIRSTROW=2,
FORMAT='CSV') AS cars;
H. Accessing data from a CSV file without a format file:
tsql
Copy
SELECT * FROM OPENROWSET(
BULK 'C:\Program Files\Microsoft SQL Server\MSSQL14.CTP1_1\MSSQL\DATA\inv-2017-01-19.csv',
SINGLE_CLOB) AS DATA;
So I am thinking that the correct OR ONLY approach involves OPENDATASOURCE with the jet driver present and accessible on the remote server. However, where in the following example from MS does the IP address of the remote server and the login password and username go? If it cannot then that seems contradictory to the claims of the capability of the command in the documentation (based on the words they use), and it would be great if they could say 'you cannot do this', as it is pretty clear that most people will try...
SELECT * FROM OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0',
'Data Source=C:\DataFolder\Documents\TestExcel.xls;Extended Properties=EXCEL 5.0')...[Sheet1$] ;
So,
Something like:
SELECT * FROM OPENDATASOURCE('Microsoft.ACE.OLEDB.12.0'|'Microsoft.Jet.OLEDB.4.0'|'PACISOleDb', 'Data Source=XXX.XXX.XXX.XXX\DataFolder\Documents\TestExcel.xls';User ID=MyUID;Password=MyPass;Extended Properties=EXCEL 5.0')...[Sheet1$] ;
or maybe
(TWIDDLING THUMBS - BORING BORING BORING)
I should give up and use a linked server:
EXEC sp_addlinkedserver
#server = 'ExcelLinkSrv1',
#srvproduct = 'Excel',
#provider = 'Microsoft.ACE.OLEDB.12.0',
#datasrc = 'C:\DataFiles\EmployeeData1.xlsx',
#provstr = 'Excel 12.0 Xml; HDR=YES; IMEX=1';
GO
But again - can I have:
#server = '202.217.XXX.XXX'
?
Yes - I know that you would normally not hard code it, but let's start simple in SSMS. I wanted to avoid linked server for different reasons. How do you do it with OPENROWSET or else OPENDATASOURCE against excel spreadsheet or else .csv file?
You can have all of my reputation points or whatever they are if your solution or information works (and someone else did not get them first), because I don't care about that stuff.
Simply use windows sharepaths for that.
EXEC sp_addlinkedserver
#server = 'ExcelLinkSrv1',
#srvproduct = 'Excel',
#provider = 'Microsoft.ACE.OLEDB.12.0',
#datasrc = '\\SERVEROTHER\c$\DataFiles\EmployeeData1.xlsx',
#provstr = 'Excel 12.0 Xml; HDR=YES; IMEX=1';
An Excel connection string has specific requirements, which are based on connection DLL string mappings based on the provider you have selected (ACE/JET). This then is translated to behavior defined in the registry, which is difficult to parse.
The underlying technology used to interact with your provided string is OLE Data Access which is of course COM based.
As others pointed out, you may use a sharepath, but I wanted to provide more detail.
Information about the internals of these providers is difficult to come by, and official MSFT documentation on the matter is no longer directly accessible online (I would imagine there is an archive somewhere on Excel Data Sources in one giant PDF, but I was unable to locate it). COM-based reading material is generally all been deprecated or removed, despite its integral role in the Windows OS.
I share your sentiment about these commands. If it helps at all, here are the details of the "secret" commands within OPENDATASOURCE, taken from https://www.codeproject.com/Articles/37055/Working-with-MS-Excel-xls-xlsx-Using-MDAC-and-Oled (at risk of being lost as well):
The connectionstring has some parts:
Provider: It is the main oledb provider that is used to open the Excel
sheet. This will be Microsoft.Jet.OLEDB.4.0 for Excel 97 onwards Excel
file format and Microsoft.ACE.OLEDB.12.0 for Excel 2007 or higher
Excel file format (One with xlsx extension)
Data Source: It is the entire path of the Excel workbook. You need to mention a dospath that corresponds to an Excel file. Thus, it will
look like: Data Source=C:\testApp.xls".
Extended Properties (Optional): Extended properties can be applied to Excel workbooks which may change the overall activity of the Excel
workbook from your program. The most common ones are the following:
HDR: It represents Header of the fields in the Excel table. Default is YES. If you don't have fieldnames in the header of your
worksheet, you can specify HDR=NO which will take the columns of the
tables that it finds as f1,f2 etc.
ReadOnly: You can also open Excel workbook in readonly mode by specifying ReadOnly=true; By default, Readonly attribute is false, so
you can modify data within your workbook.
FirstRowHasNames: It is the same as HDR, it is always set to 1 ( which means true) you can specify it as false if you don't have your
header row. If HDR is YES, provider disregards this property. You can
change the default behaviour of your environment by changing the
Registry Value
[HKLM\Software\Microsoft\Jet\4.0\Engines\Excel\FirstRowHasNames] to 00
(which is false)
MaxScanRows: Excel does not provide the detailed schema defination of the tables it finds. It need to scan the rows before
deciding the data types of the fields. MaxScanRows specifies the
number of cells to be scanned before deciding the data type of the
column. By default, the value of this is 8. You can specify any value
from 1 - 16 for 1 to 16 rows. You can also make the value to 0 so that
it searches all existing rows before deciding the data type. You can
change the default behaviour of this property by changing the value of
[HKLM\Software\Microsoft\Jet\4.0\Engines\Excel\TypeGuessRows] which is
8 by default. Currently, MaxScanRows is ignored, so you need only to
depend on TypeGuessRows Registry value. Hope Microsoft fixes this
issue to its later versions.
IMEX: (A Caution) As mentioned above, Excel will have to guess a number or rows to select the most appropriate data type of the
column, a serious problem may occur if you have mixed data in one
column. Say you have data of both integer and text on a single column,
in that case, Excel will choose its data type based on majority of the
data. Thus it selects the data for the majority data type that is
selected, and returns NULL for the minority data type. If the two
types are equally mixed in the column, the provider chooses numeric
over text.
For example, in your eight (8) scanned rows, if the column contains five (5) numeric values and three (3) text values, the
provider returns five (5) numbers and three (3) null values.
To work around this problem for data, set "IMEX=1" in the Extended Properties section of the connection string. This enforces
the ImportMixedTypes=Text registry setting. You can change the
enforcement of type by changing
[HKLM\Software\Microsoft\Jet\4.0\Engines\Excel\ImportMixedTypes] to
numeric as well.
Thus if you look into the simple connectionstring with all of them, it will look like:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\testexcel.xls;
Extended Properties=\"Excel 8.0;HDR=YES;IMEX=1;MAXSCANROWS=15;READONLY=FALSE\""
or:
Copy Code
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\\testexcel.xlsx;
Extended Properties=\"Excel 12.0;HDR=YES;IMEX=1;MAXSCANROWS=15;READONLY=FALSE\""
We need to place extended properties into Quotes(") as there are multiple number of values.
If you want to link to csv files, Erland Sommarskog has a very good article: here, which discusses the practical use of OPENROWSET (BULK) , as well as BCP and BULK IMPORT . As he points out in detail, there are number of gotchas, especially if you are stuck with a version of SQL Server before 2017.
The layout of the csv file may not allow you to use OPENROWSET (BULK) without some prior processing. The article gives full details. If it is possible, however, it is by far the fastest method for gulping large volumes of text into a SQL Server database.
Here is a simple SQL query for CSV import using opendatasource and the access text driver 64 bit
SELECT * FROM OPENDATASOURCE('Microsoft.ACE.OLEDB.12.0',
'Text;Database=D:\TEMP;HDR=YES')..[pdf_attachment#csv]
In SQL Server 2016 I have a stored procedure. In this SP I want to get all data from a view from a linked server (PostgreSQL) by using the following statement:
INSERT INTO myTable
SELECT Field1,
Field2,
Field3,
...
FROM OPENQUERY(myServer, 'SELECT * FROM myDatabase.mySchema.myView')
When I use it like this, I'm getting the following error message after a few minutes:
Out of memory while reading tuples.
I changed the SELECT statement in OPENQUERY to get only the first 1000000 rows which worked fine:
SELECT * FROM myDatabase.mySchema.myView ORDER BY Field1 LIMIT 1000000
Now I am unsure what the most practical way to get all data would be. I could insert the first 1000000 rows and then insert the next 1000000 using OFFSET. But I don't think this would be a nice solution as I don't know what the total number of rows is. A loop would be another way but I really don't know if this would be the easiest way to achieve what I want.
Any help would be appreciated.
i think you are using odbc driver for creating linked sever .It is issue with psqlODBC driver memory configuration .
You change odbc driver setting opening your data sources
Press "Configure", then in the opened data source details
"Options" section select "Datasource"
and in the opened window check the "use declare/fetch".
In my project I am rebuilding my Access database to an SQL Database. So to do this I am transferring the Access DATA to the SQL Database. I made sure they both have the same structure and the Access fields are modified correctly in the SQL database.
For most of the data this works.
Except for 1 table. This table gives me the following weird error message:
OLE DB provider 'Microsoft.ACE.OLEDB.12.0' for linked server 'OPS_JMD_UPDATE' returned data that does not match expected data length for column '[OPS_JMD_UPDATE]...[OrderStatus].Omschrijving'. The (maximum) expected data length is 100, while the returned data length is 21.
So here some more information about both the Access and SQL field/column:
Access type: Short text
SQL type: nvarchar(MAX)
Access column data in it: Normal letters and & - % é + €
. , : being the 'not normal ones'.
A few empty Access records (which is allowed)
A total of 135314 record in the Access table
Iv'e set the SQL datatype to nvarchar(MAX) so that the field can never be to small, this didn't seem to help though..
*The OPS_JMD_UPDATE is the linked Access database
What causes this problem? Is it because some characters aren't allowed or..?
There was 1 record that generated the error. I pinned out the exact record with a TOP select and DESC Select and then used the select ascii replaceto remove the error! Solved thanks to xQbert, once more thank you!
While in Management Studio, I am trying to run a query/do a join between two linked servers.
Is this a correct syntax using linked db servers:
select foo.id
from databaseserver1.db1.table1 foo,
databaseserver2.db1.table1 bar
where foo.name=bar.name
Basically, do you just preface the db server name to the db.table ?
The format should probably be:
<server>.<database>.<schema>.<table>
For example:
DatabaseServer1.db1.dbo.table1
Update: I know this is an old question and the answer I have is correct; however, I think any one else stumbling upon this should know a few things.
Namely, when querying against a linked server in a join situation the ENTIRE table from the linked server will likely be downloaded to the server the query is executing from in order to do the join operation. In the OP's case, both table1 from DB1 and table1 from DB2 will be transferred in their entirety to the server executing the query, presumably named DB3.
If you have large tables, this may result in an operation that takes a long time to execute. After all it is now constrained by network traffic speeds which is orders of magnitude slower than memory or even disk transfer speeds.
If possible, perform a single query against the remote server, without joining to a local table, to pull the data you need into a temp table. Then query off of that.
If that's not possible then you need to look at the various things that would cause SQL server to have to load the entire table locally. For example using GETDATE() or even certain joins. Others performance killers include not giving appropriate rights.
See http://thomaslarock.com/2013/05/top-3-performance-killers-for-linked-server-queries/ for some more info.
SELECT * FROM OPENQUERY([SERVER_NAME], 'SELECT * FROM DATABASE_NAME..TABLENAME')
This may help you.
For those having trouble with these other answers , try OPENQUERY
Example:
SELECT * FROM OPENQUERY([LinkedServer], 'select * from [DBName].[schema].[tablename]')
If you still find issue with <server>.<database>.<schema>.<table>
Enclose server name in []
You need to specify the schema/owner (dbo by default) as part of the reference. Also, it would be preferable to use the newer (ANSI-92) join style.
select foo.id
from databaseserver1.db1.dbo.table1 foo
inner join databaseserver2.db1.dbo.table1 bar
on foo.name = bar.name
select * from [Server].[database].[schema].[tablename]
This is the correct way to call.
Be sure to verify that the servers are linked before executing the query!
To check for linked servers call:
EXEC sys.sp_linkedservers
right click on a table and click script table as select
select name from drsql01.test.dbo.employee
drslq01 is servernmae --linked serer
test is database name
dbo is schema -default schema
employee is table name
I hope it helps to understand, how to execute query for linked server
Usually direct queries should not be used in case of linked server because it heavily use temp database of SQL server. At first step data is retrieved into temp DB then filtering occur. There are many threads about this. It is better to use open OPENQUERY because it passes SQL to the source linked server and then it return filtered results e.g.
SELECT *
FROM OPENQUERY(Linked_Server_Name , 'select * from TableName where ID = 500')
For what it's worth, I found the following syntax to work the best:
SELECT * FROM [LINKED_SERVER]...[TABLE]
I couldn't get the recommendations of others to work, using the database name. Additionally, this data source has no schema.
In sql-server(local) there are two ways to query data from a linked server(remote).
Distributed query (four part notation):
Might not work with all remote servers. If your remote server is MySQL then distributed query will not work.
Filters and joins might not work efficiently. If you have a simple query with WHERE clause, sql-server(local) might first fetch entire table from the remote server and then apply the WHERE clause locally. In case of large tables this is very inefficient since a lot of data will be moved from remote to local. However this is not always the case. If the local server has access to remote server's table statistics then it might be as efficient as using openquery More details
On the positive side T-SQL syntax will work.
SELECT * FROM [SERVER_NAME].[DATABASE_NAME].[SCHEMA_NAME].[TABLE_NAME]
OPENQUERY
This is basically a pass-through. The query is fully processed on the remote server thus will make use of index or any optimization on the remote server. Effectively reducing the amount of data transferred from the remote to local sql-server.
Minor drawback of this approach is that T-SQL syntax will not work if the remote server is anything other than sql-server.
SELECT * FROM OPENQUERY([SERVER_NAME], 'SELECT * FROM DATABASE_NAME.SCHEMA_NAME.TABLENAME')
Overall OPENQUERY seems like a much better option to use in majority of the cases.
I have done to find out the data type in the table at link_server using openquery and the results were successful.
SELECT * FROM OPENQUERY (LINKSERVERNAME, '
SELECT DATA_TYPE, COLUMN_NAME
FROM [DATABASENAME].INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME =''TABLENAME''
')
Its work for me
Following Query is work best.
Try this Query:
SELECT * FROM OPENQUERY([LINKED_SERVER_NAME], 'SELECT * FROM [DATABASE_NAME].[SCHEMA].[TABLE_NAME]')
It Very helps to link MySQL to MS SQL
PostgreSQL:
You must provide a database name in the Data Source DSN.
Run Management Studio as Administrator
You must omit the DBName from the query:
SELECT * FROM OPENQUERY([LinkedServer], 'select * from schema."tablename"')
For MariaDB (and so probably MySQL), attempting to specify the schema using the three-dot syntax did not work, resulting in the error "invalid use of schema or catalog". The following solution worked:
In SSMS, go to Server Objects > Linked Servers > Providers > MSDASQL
Ensure that "Dynamic parameter", "Level zero only", and "Allow inprocess" are all checked
You can then query any schema and table using the following syntax:
SELECT TOP 10 *
FROM LinkedServerName...[SchemaName.TableName]
Source: SELECT * FROM MySQL Linked Server using SQL Server without OpenQuery
Have you tried adding " around the first name?
like:
select foo.id
from "databaseserver1".db1.table1 foo,
"databaseserver2".db1.table1 bar
where foo.name=bar.name