I have set up a SQL link server between an SQLserver 2005 on a W2003 R2 and a SYBASE 12.5.0.2 server on a IBM AIX H70 system.
I use the Sybase ODBC driver 04.20.00.67
The problem is than most of the times (there is no pattern ) when I select rows from a Sybase table I get ONLY ONE ROW without any error. Please note that there is no problem when I insert rows from SQL server to the SYBASE server
I appreciate any possible solutions...
How are you selecting the rows? Is it via
select a,b,c FROM LINKED_SERVERNAME.foo.bar.bletch
or are you doing 'pass-thru' sql like this
EXEC('SELECT * FROM foo..bletch') AT LINKED_SERVERNAME
I have had way more success with the pass-thru style query: SQL Server does not appear to get what it needs metadata-wise from the Sybase ODBC driver. I was forced to write replacements for the sp_tables_ex and friends family of stored procedures as they did not work for me with linked Sybase dataservers.
pjjH
Related
Here's the basic idea of what I want to do in SSIS:
I have a large query against a production Oracle database, and I need the following where clause that brings in a long list of ids from SQL Server. From there, the results are sent elsewhere.
select ...
from Oracle_table(s) --multi-join
where id in ([select distinct id from SQL_SERVER_table])
Alternatively, I could write the query this way:
select ...
from Oracle_table(s) --multi-join
...
join SQL_SERVER_table sst on sst.ID = Oracle_table.ID
Here are my limitations:
The Oracle query is large and cannot be run without the where id in (... clause
This means I cannot run the Oracle query, then join it against the ids in another step. I tried this, and the DBA's killed the temp table after it became 3 TB in size.
I have 160k id's
This means it is not practical to iterate through the id's one by one. In the past, I have run against ~1000 IDs, using a comma-separated list. It runs relatively fast - a few minutes.
The main query is in Oracle, but the ids are in SQL Server
I do not have the ability to write to Oracle
I've found many questions like this.
None of the answers I have found have a solution to my limitations.
Similar question:
Query a database based on result of query from another database
To prevent loading all rows from the Oracle table. The only way is to apply the filter in the Oracle database engine. I don't think this can be achieved using SSIS since you have more than 160000 ids in the SQL Server table, which cannot be efficiently loaded and passed to the Oracle SQL command:
Using Lookups and Merge Join will require loading all data from the Oracle database
Retrieving data from SQL Server, building a comma-separated string, and passing it to the Oracle SQL command cannot be done with too many IDs (160K).
The same issue using a Script Task.
Creating a Linked Server in SQL Server and Joining both tables will load all data from the Oracle database.
To solve your problem, you should search for a way to create a link to the SQL Server database from the Oracle engine.
Oracle Heterogenous Services
I don't have much experience in Oracle databases. Still, after a small research, I found something in Oracle equivalent to "Linked Servers" in SQL Server called "heterogeneous connectivity".
The query syntax should look like this:
select *
from Oracle_table
where id in (select distinct id from SQL_SERVER_table#sqlserverdsn)
You can refer to the following step-by-step guides to read more on how to connect to SQL Server tables from Oracle:
What is Oracle equivalent for Linked Server and can you join with SQL Server?
Making a Connection from Oracle to SQL Server - 1
Making a Connection from Oracle to SQL Server - 2
Heterogeneous Database connections - Oracle to SQL Server
Importing Data from SQL Server to a staging table in Oracle
Another approach is to use a Data Flow Task that imports IDs from SQL Server to a staging table in Oracle. Then use the staging table in your Oracle query. It would be better to create an index on the staging table. (If you do not have permission to write to the Oracle database, try to get permission to a separate staging database.)
Example of exporting data from SQL Server to Oracle:
Export SQL Server Data to Oracle using SSIS
Minimizing the data load from the Oracle table
If none of the solutions above solves your issue. You can try minimizing the data loaded from the Oracle database as much as possible.
As an example, you can try to get the Minimum and Maximum IDs from the SQL Server table, store both values within two variables. Then, you can use both variables in the SQL Command that loads the data from the Oracle table, like the following:
SELECT * FROM Oracle_Table WHERE ID > #MinID and ID < #MaxID
This will remove a bunch of useless data in your operation. In case your ID column is a string, you can use other measures to filter data, such as the string length, the first character.
This may as well be named "How to check if PolyBase is installed on SQL Server using just SQL Query" (as those two are basically synonyms).
When PolyBase is not installed, you're not able to call CREATE EXTERNAL TABLE, DATA SOURCE or FILE FORMAT, etc.
How can you check beforehand if those server features are available, using just SQL Query?
Is there anything like SELECT SERVERPROPERTY('IsPolyBaseInstalled') for PolyBase component?
You will find some information with :
SELECT *
FROM sys.configurations
WHERE name LIKE '%polybase%' COLLATE French_CI_AI;
To enable PolyBase use sp_configure
I'm trying to query my Hortonworks cluster Hive tables from SQL Server. My scenario below:
HDP 2.6, Ambari, HiveServer2
SQL Server 2016 Enterprise
Kerberos configuration for secure logins in HDP
I was reading about the PolyBase service in SQL Server 2016 and I suppose later versions. However, I realize that according to the documentation what this service is going to perform in SQL Server is a bridge to reach out my HDFS and recreate external tables based in this data source.
Otherwise what I'm expecting is query Hive objects like these would be SQL Server objects as well, such as a linked server.
Someone has an example or knows if this is possible within SQL Server and Hive?
Thanks so much
Hive acts more as a job compiler than a database. This means every SQL statement you are writing will be translated into a job for Hadoop, sent over to the cluster and become executed there. From the user perspective it looks like querying a table.
The already mentioned approach by reading the HDFS data source and re-create it in SQL Server is the correct one. Since both, Hive and database server are different technologies, something like a linked server seems to technically not possible for me.
Hive provides nowadays a JDBC interface which could be used to connect to it. But even with Hive JDBC, every query will end up as cluster job for distributed computing, running over the files in HDFS, create a result set and present that to you.
If you want to querying Hive from SQL server, you can download ODBC driver (Microsoft or Hortonsworks) and create a Data Source Name (DSN) for Hive. In Advanced option check Use Native Query. Then just create new linked server in the SQL server with the same name of datasource as Data Source Name in ODBC driver.
Write openquery something like:
select top 100 * from
openquery(HadoopLinkedServer,
'column1, column2 from databaseInHadoop.tableInHadoop')
I'm running into a problem when accessing a SQL Server table from an Oracle setup via ODBC.
I can access 90% of the tables absolutely fine, but there's a few tables that have a name that's longer than 30 characters. Whenever I try to interact with the table (describes, selects, etc) Oracle throws an "identifier too long" error and gives up.
Is there a way to coax Oracle into playing nice with the SQL Server tables?
Assuming that we are talking about an Oracle database that has a database link created to a SQL Server database via Heterogeneous Services, you would need to write code using the DBMS_HS_PASSTHROUGH package to interact with the tables in question. You'd also need to use this package if you have tables where there are column names that are not valid Oracle identifiers.
I am attempting to insert data across a DB link from an MS SQL Server 2008 into and Oracle 11g server using a pass through insert:
insert into openquery(ORACLE,'select varchar2_1,varchar2_2, number1 from table1')
select varchar_1, varchar_2, integer_1 from mssql_table;
When I try and execute this I get and error saying that the data violates the schema.
Error:
The OLE DB provider "MSDAORA" for linked server "ORACLE" could not INSERT INTO table "[MSDAORA]" because of column "VARCHAR2_1". The data value violated the schema for the column.
I searched around google and stackoverflow but could not seem to find a specific solution to this problem.
fyi: I cannot change the schema in the oracle DB.
Any help would be greatly appreciated.
Check the lengths of the data in varchar_2. Can the data fit into the oracle column varchar2_1?