Problem Conditions
I have a very simple Oracle (11g) Stored Procedure that is declared like so:
CREATE OR REPLACE PROCEDURE pr_myproc(L_CURSOR out SYS_REFCURSOR)
is
BEGIN
OPEN L_CURSOR FOR
SELECT * FROM MyTable;
END;
This compiles correctly. The cursor contains col1, col2 and col3.
In SSRS, i have a Shared Data Source that uses Oracle OLEDB Provider for Oracle 11g:
Provider=OraOLEDB.Oracle.1;Data Source=LIFEDEV
(Plus the user credentials).
What Works OK:
The stored procedure executes
correctly in PL/SQL Developer
The 'test connect' in works fine in SSRS
A query string of SELECT * FROM MyTable; with Command Type of 'text' produces the correct fields in the SSRS report.
.NET Oracle Provider instead of Oracle OLE DB Provider
What Fails:
If i change the Command Type to 'Stored Procedure' and enter 'pr_myproc', when I click 'OK' Visual Studio 2005 (service pack 2) simply hangs/crashes.
Does anyone have any knowledge/experience of this?
Any help would be most appreciated. Thanks.
FURTHER INFORMATION
I've modified the provider from the Oracle OLE DB Provider to the .NET Oracle Provider, and, magically, it works.
This would seem to indicate an issue with the Oracle provider.
Any more thoughts?
We got to the bottom of this.
On the environment where the procedure resided, we have a substantial data dictionary. The two providers when looking up information use two different queries.
Here is the one the Oracle Provider used, taking 10+ minutes:
select * from (select null PROCEDURE_CATALOG
, owner PROCEDURE_SCHEMA
, object_name PROCEDURE_NAME
, decode (object_type, 'PROCEDURE', 2, 'FUNCTION', 3, 1) PROCEDURE_TYPE
, null PROCEDURE_DEFINITION
, null DESCRIPTION
, created DATE_CREATED
, last_ddl_time DATE_MODIFIED
from all_objects where object_type in ('PROCEDURE','FUNCTION')
union all
select null PROCEDURE_CATALOG
, arg.owner PROCEDURE_SCHEMA
, arg.package_name||'.'||arg.object_name PROCEDURE_NAME
, decode(min(arg.position), 0, 3, 2) PROCEDURE_TYPE
, null PROCEDURE_DEFINITION
, decode(arg.overload, '', '', 'OVERLOAD') DESCRIPTION
, min(obj.created) DATE_CREATED
, max(obj.last_ddl_time) DATE_MODIFIED
from all_objects obj, all_arguments arg
where arg.package_name is not null
and arg.owner = obj.owner
and arg.object_id = obj.object_id
group by arg.owner, arg.package_name, arg.object_name, arg.overload ) PROCEDURES
WHERE PROCEDURE_NAME = '[MY_PROCEDURE_NAME]' order by 2, 3
More info can be found here
Related
I have an SSIS Package, which contains multiple flows.
Each flow is responsible for creating a "staging" table, which gets filled up after creation.
These tables are global temporary tables.
I added 1 extra flow (I did not make the package) which does exactly as stated above, for another table. However, for some reason, the package fails intermittently on this flow, while it is exactly the same as others, besides some table names.
The error that keeps popping up:
Update - Insert Data Flow:Error: SSIS Error Code DTS_E_OLEDBERROR. An
OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is
available. Source: "Microsoft SQL Server Native Client 11.0"
Hresult: 0x80004005 Description: "Unspecified error". An OLE DB
record is available. Source: "Microsoft SQL Server Native Client
11.0" Hresult: 0x80004005 Description: "The metadata could not be determined because statement 'select * from
'##TmpMcsConfigurationDeviceHistory86B34BFD041A430E84CCACE78DA336A1'' uses a temp table.".
Creation expression:
"CREATE TABLE " + #[User::TmpMcsConfigurationDeviceHistory] + " ([RecId] [bigint] NULL,[DataAreaID] [nvarchar](4) COLLATE database_default NULL,[Asset] [bigint] NULL,[Code] [nvarchar](255) COLLATE database_default NULL,[Configuration] [bigint],[StartdateTime] [datetime] NULL,[EndDateTime] [datetime] NULL)
"
Parsed expression (=evaluated):
CREATE TABLE ##TmpMcsConfigurationDeviceHistory764E56F088DC475C9CC747CC82B9E388 ([RecId] [bigint] NULL,[DataAreaID] [nvarchar](4) COLLATE database_default NULL,[Asset] [bigint] NULL,[Code] [nvarchar](255) COLLATE database_default NULL,[Configuration] [bigint],[StartdateTime] [datetime] NULL,[EndDateTime] [datetime] NULL)
Using WITH RESULT SETS to explicitly define the metadata will allow SSIS to skip the sp_describe_first_result_set step and use the metadata that you define. The upside is that you can use this to get SSIS to execute SQL that contains a temporary table (for me, that performance helped a lot); the downside is, you have to manually maintain and update this if anything changes.
Query sample (stored procedure:)
EXEC ('dbo.MyStoredProcedure')
WITH RESULT SETS
(
(
MyIntegerColumn INT NOT NULL,
MyTextColumn VARCHAR(50) NULL,
MyOtherColumn BIT NULL
)
)
Query sample (simple SQL:)
EXEC ('
CREATE TABLE #a
(
MyIntegerColumn INT NOT NULL,
MyTextColumn VARCHAR(50) NULL,
MyOtherColumn BIT NULL
)
INSERT INTO #a
(
MyIntegerColumn,
MyTextColumn,
MyOtherColumn
)
SELECT
1 AS MyIntegerColumn,
''x'' AS MyTextColumn,
0 AS MyOtherColumn
SELECT MyIntegerColumn, MyTextColumn, MyOtherColumn
FROM #a')
WITH RESULT SETS
(
(
MyIntegerColumn INT NOT NULL
,MyTextColumn VARCHAR(50) NULL
,MyOtherColumn BIT NULL
)
)
Another option (kind of a hack, but it works and doesn't require you to change your use of global temp tables) is to use a SET FMTONLY ON command in front of your actual query to send a fake "First result set" to SSIS with your correct column structure. So you can do something like
SET FMTONLY ON
select 0 as a, 1 as b, 'test' as C, GETDATE() as D
SET FMTONLY OFF
select a, b, c, d from ##TempTable
When SSIS runs sp_describe_first_result_set, it will return the metadata and column names of your FMTONLY command, and won't complain about not being able to determine the metadata of your temp table because it won't even try.
If you are working on SSIS 2012, then it uses system stored procedure sp_describe_first_result_set to fetch the metadata of the tables and it does not support temporary tables. But you can go for other options like table variables and CTEs which are going to work fine. https://connect.microsoft.com/SQLServer/feedback/details/629077/denali-engine-metadata-discovery-shuns-temp-tables
I had faced a similar issue when SSSI packages were migrated from 2008 to 2016. The latest version uses sp_describe_first_result_set to fetch metadata and it does not work with temporary tables. As a workaround, I used the below query in the OLEDB source editor. I did not change the SQL stored procedure, and it still uses a temporary table. Do be sure to use the Parse Query and Preview option to ensure it works fine. See the image below.
Query:
EXEC [dbo].[spGetNames]
WITH RESULT SETS((
FirstName varchar(50),
LastName varchar(50)
));
Had the same issue as we use temp table for staging. After spending some time, found a work around.
In the OLE DB/ADO Destination of Data flow task where you specify the name of the staging table .
Change the AccessMode property to SQL command instead of OpenRowSet and specify SQL Command property to "select * from #temp".
Hurray, Its working as expected.
Catch here is when you specify Access mode other than SQL Command, SSIS expects that to be a table / view and it changed the SSIS to call sp_describe_first_result_set to get the meta data. but when you specify SQL Command, it's expecting a query or SP command etc. so luckily it still uses the old way of getting the meta data .
http://social.msdn.microsoft.com/Forums/sqlserver/en-US/cfe1c7c1-910a-4f52-9718-c3406263b177/usage-of-temp-tables-in-ssis-2012?forum=sqlintegrationservices#cfe1c7c1-910a-4f52-9718-c3406263b177
I found that the problem lied in a GUID duplicate issue, I copied elements (like the one to create temp tables) and they all received the same guid upon copying. I used a tool to reset all these guids in my package and this solved my problem.
Thanks!
I need to compare data across two different DB2 database instances. We're not permitted to set up federation. I've found references saying how to specify data loads from remote databases, and also references on how to specify a database connection including database name, username etc. Ideally I would be able to execute a query against one database, then compare that to the second database either one-by-one (using SQL PL loops etc.), or as a single large join. I've gotten to the point where the SQL PL script can connect to each in turn (and it prompts me for the password to both), but it only recognizes the second one when I attempt to query the table.
What we've tried:
Adding two different CONNECT statements at the beginning.
Declaring a cursor and specifying the database name (this seems to only work when doing loads from one database to another, which we're trying to avoid).
set serveroutput on#
set sqlcompat DB2#
connect to first user myname#
connect to second user myname#
-- run command: db2 -td# -vf test3.sql
begin
declare loop_counter int;
call dbms_output.enable(100000);
set loop_counter = 0;
FIRSTLOOP: for o as ord1 cursor for
select field1, field2 from first.firstschema.firsttable fetch first 10 rows only with ur
do
set loop_counter = loop_counter + 1;
call dbms_output.put_line('Field: '||field1||', other '||field2);
end for;
call dbms_output.put_line('End first program: ');
SECONDLOOP: for p as ord2 cursor for
select field1, field2 from second.secondschema.secondtable fetch first 10 rows only with ur
do
set loop_counter = loop_counter + 1;
call dbms_output.put_line('Field: '||field1||', other '||field2);
end for;
call dbms_output.put_line('After second call');
end#
Ideally, each of the two cursor loops would print 10 rows. In reality, whichever CONNECT was done second is the one that works. For example, if I have the connect to SECOND followed by the connect to FIRST, the first loop works and the second says "..... is an undefined name". If I do the connect to FIRST then the connect to SECOND, the first loop throws the error and I get no output.
SQL PL can connnect only to one database at a time - that is the design.
In your script example, the second connect will close any current connection first.
Federation lets you access remote tables as if they were local.
If you are prevented from using federation , your options include these:
materialising the remote table locally and copying the data
(this can be done via load from remote cursor).
You can then use SQL to compare rows, as both tables are then in the same database.
This is only feasible if you have sufficient capacity to fit both tables in same database, although compression will help here.
not using SQL but instead using another tool
For example: depending on data volumes , and data-types, you could export source/target tables
to flat files and compare the files (diff etc). You could also export to pipe and use in memory comparisons.
Or you could use python or perl or any scripting language and do the comparison in memory in chunks (in all cases
each thread can only connect to a single database at one time).
use third party tools for data comparison.
if you use embedded-SQL , type-2 connect offers another possibility.
On Db2 for IBM i, federation is only available via Db2 LUW box...
However, the following works in Db2 for IBM i...
create or replace function myschema.myudtf ()
returns table (SERVER VARCHAR(18)
, as_of timestamp
, ORDINAL_POSITION INTEGER
, JOB_NAME VARCHAR(28)
, SUBSYSTEM VARCHAR(10)
, AUTHORIZATION_NAME VARCHAR(10)
, JOB_TYPE VARCHAR(3)
)
modifies SQL data
external action
not deterministic
language SQL
specific CHKAWSJOBS
begin
declare insertStmt varchar(1500);
declare global temporary table
GLOBAL_TEMP_MY_JOBS (
SERVER VARCHAR(18)
, as_of timestamp
, ORDINAL_POSITION INTEGER
, JOB_NAME VARCHAR(28)
, SUBSYSTEM VARCHAR(10)
, AUTHORIZATION_NAME VARCHAR(10)
, JOB_TYPE VARCHAR(3)
) with replace;
for systemLoop as systemsCursor cursor for
select * from table( values ('mysys1'),('mysys2'),('mysys3'))
as systems (server_Name)
do
set insertStmt =
' insert into GLOBAL_TEMP_MY_JOBS
select
current_server as server, current_timestamp as as_of
, ordinal_position, job_name, subsystem, authorization_name, job_type
from table(QSYS2.ACTIVE_JOB_INFO(
SUBSYSTEM_LIST_FILTER => ''MYSBS'')) X
where exists (select 1 from ' concat server_name concat '.sysibm.sysdummy1)';
execute immediate InsertStmt;
end for;
return select * from GLOBAL_TEMP_MY_JOBS;
end;
The example above is more complex than your use case, I'm pulling data from a UDTF on the remote system, the trick is the use of a 3 part name in the where clause, which forces the DB to run the entire select statement on the remote machine; with the insert being into the table on the local machine.
You should be able to build a dynamic insert that's just
set insertStmt = 'insert into lcltable
select field1, field2
from ' concat server_name concat table_name
concat ' fetch first 10 rows only with ur';
Don't know for sure that this will work on Db2 LUW, but there's a good chance.
Visual studio hangs on me when using the Slowly Changing Dimension Wizard.
I select the correct connection.
Then I try to open the dropdown 'Table or view' to select a destination table.
At this moment visual studio hangs on me..
I have this on all client machines and on different visual studio versions and only on this specific database. In activity monitor I noticed that the wizard does a select * on all tables in the database... I have one table that has +4billion rows (+300GB). It is the select * on this table that takes so long.
Does anybody have any idea what causes the select * on my database, or why they are doing this? And even better, how to fix this?
Don't use the slowly changing dimension wizard in SSIS at all. The data flow it creates performs really badly compared to what you can write with TSQL.
A couple of assumption; you need a type 2 SCD and you are using at least SQL Server 2008 with MERGE statements available.
Instead of SSIS use the OUTPUT clause of the MERGE statement within TSQL to perform the dimension update/insert. For example:
INSERT INTO Customer_Master
SELECT
Source_Cust_ID,
First_Name,
Last_Name,
Eff_Date,
End_Date,
Current_Flag
FROM
(
MERGE
Customer_Master CM
USING
Customer_Source CS
ON
CM.Source_Cust_ID = CS.Source_Cust_ID
WHEN NOT MATCHED
THEN
INSERT VALUES
(
CS.Source_Cust_ID,
CS.First_Name,
CS.Last_Name,
CONVERT(char(10), GETDATE()-1, 101),
'12/31/2199',
'y'
)
WHEN MATCHED
AND CM.Current_Flag = 'y'
AND (CM.Last_Name <> CS.Last_Name )
THEN
UPDATE
SET
CM.Current_Flag = 'n',
CM.End_date = convert(char(10), getdate()- 2, 101)
OUTPUT
$Action Action_Out,
CS.Source_Cust_ID,
CS.First_Name,
CS.Last_Name,
convert(char(10), getdate()-1, 101) Eff_Date,
'12/31/2199' End_Date,
'y' Current_Flag
) AS MERGE_OUT
WHERE
MERGE_OUT.Action_Out = 'UPDATE';
Source: http://www.kimballgroup.com/2008/11/design-tip-107-using-the-sql-merge-statement-for-slowly-changing-dimension-processing/
I'm trying to load incremental data from ODBC server to SQL server using common table expression.
When running the query in the Dbeabver application, is executed correctly:
with test as
(
SELECT userid,sum(goldbalance)
FROM Server.events_live
where eventTimestamp>=DATE '2016-01-01' + INTERVAL '-100 day'
group by userid
order by sum(goldbalance) desc)
)
select * from test
when running it from an sql command expression of the ODBC source, it fails due to wrong syntax. It looks as follow:
with test as
(
SELECT userid,sum(goldbalance)
FROM deltadna.events_live
where eventTimestamp>=DATE '"+#[User::datestring]+"' + INTERVAL '-100 day'
group by userid
order by sum(goldbalance) desc)
)
select * from test"
the datestring variable is getting the server date and convert it to string in the format yyyy-mm-dd. I'm usually use this method to pull data from ADO.NET and it works properly.
Is there any other way to pull incremental data from ODBC server using ssis variables?
With OLE DB
Try this code, it works for me with my own tables with SQL Server :
SELECT userid,sum(goldbalance) AS SUMGOLD
FROM deltadna.events_live
WHERE eventTimestamp >= DATEADD(DAY, -100,CONVERT(DATE,?))
GROUP BY userid
ORDER BY SUMGOLD desc
You have to click on Parameters in the OLEDB Source Editor to configure what you need. Use the '?' to represent a variable in your query.
If you query if too complicated, stored it in a stored procedure and call it like this:
EXEC shema.storedProcedureName ?
And map the '?' to your variable #user::DateString
With ODBC
The expressions are outside the data flow in Data Flow Properties.
Select the expression property and add your dynamic query.
And your expression will be
"SELECT userid,sum(goldbalance) AS SumGold
FROM deltadna.events_live
where eventTimestamp>=DATE "+#[User::datestring]+" +INTERVAL '-100 day'
group by userid
order by SumGold desc"
We have one database existing on server A. Server A also hosts our program code which will be calling the SQL statement.
We have another database VMIntranetTest existing on server B VMC-MMS
Server A and server B have different logon user credentials. Server A and server B both exist on our internal network.
Using PHP, I have the following SQL statement defined.
$strSql = 'INSERT INTO VMC-MMS.VMIntranetTest.dbo.TestTable (FirstName, LastName, Age) ' .
'SELECT FNAME, LNAME, AGE ' .
'FROM BSLIB.SQLTSTF ';
FROM -> BSLIB.SQLTSTF <- is on our local server (A), so my connection string used to execute the statement will have the user credentials to connect to server A.
INSERT INTO -> VMC-MMS.VMIntranetTest.dbo.TestTable <- is the different server.database.dbo.table (Server B).
How do I specify the user credentials to be used for the INSERT INTO portion of the statement? The secondary portion containing the SELECT FROM statement should already be covered by my initial connection string.
Thank you,
Edit 1 in regards to Paul's answer.
I've attempted to use the OPENROWSET as mentioned, and have the following SQL statement.
INSERT INTO VMIntranetTest.TestTable (FirstName, LastName, Age)
OPENROWSET('vmas400',
'Server=192.168.1.2;Trusted_Connection=yes;user_id=INTRAIS;password=****',
'SELECT FNAME, LNAME, AGE FROM BSLIB.SQLTSTF' ) as a
As you can see, I changed things around a little bit. My connection string through the code opens the connection to Server B "VMC-MMS". My SQL statement "select" portion, uses the OPENROWSET to open a connection to Server A "192.168.1.2".
However, I am getting this error message:
SQLSTATE[HY000]: General error: 1 near "OPENROWSET": syntax error
Edit 2 i needed to put the entire OPENROWSET portion inside a VALUES ( ) clause. Now I'm getting a message:
SQLSTATE[HY000]: General error: 1 no such table: VMIntranetTest.TestTable
Edit 3
I've now got the following SQL
INSERT INTO VMIntranetTest.TestTable (FirstName, LastName, Age)
select a.FNAME, a.LNAME, a.AGE FROM
OPENROWSET('vmas400',
'Server=192.168.1.2;Trusted_Connection=yes;user_id=INTRAIS;password=****',
'SELECT FNAME, LNAME, AGE FROM BSLIB.SQLTSTF' ) as a
And am getting this error:
SQLSTATE[HY000]: General error: 1 near "(": syntax error
You just need the SELECT statement in there. Try this:
INSERT INTO VMIntranetTest.dbo.TestTable (a.FirstName, a.LastName, a.Age)
SELECT a.FirstName, a.LastName, a.Age FROM
OPENROWSET('vmas400',
'Server=192.168.1.2;Trusted_Connection=yes;user_id=INTRAIS;password=****',
'SELECT FNAME, LNAME, AGE FROM BSLIB.dbo.SQLTSTF' ) as a
EDIT: Try the query now. You had the table specified from the database without specifying the schema. If you have appropriate permissions, the above query will now work. Otherwise, you will need to specify the schema. Sorry for not catching that!
You could use OPENROWSET to select data from the different database
http://msdn.microsoft.com/en-us/library/ms190312.aspx