I am trying do the following using SSIS. I am using SDT latest version and querying SQL Server 2008 R2.
Could somebody suggest if I am doing it wrong and if there is a better way in doing it
I basically need to execute a stored procedures from the source and store the results it in the destination. There are around 20 stored procedures.
Each of the stored procedures takes XML as input. I have created a Control task within that a data flow task. Within the data flow i am creating the oledb source.
I have selected Data access mode as SQL command. I am created my query below but when i preview , I get error saying that declare statement is not supported
DECLARE #xmldata XML
SET #xmldata = (SELECT DISTINCT TOP 1000 (mc.companyId) AS id
FROM ciqMarketCap mc
JOIN ciqCompany c ON c.companyid = mc.companyid
WHERE c.companyStatusTypeId NOT IN (5, 6)
AND c.companyTypeId IN (1, 4)
FOR XML PATH(''), TYPE, ROOT('partyIds'))
EXEC getPartiesMarketData #xmldata
How do I execute the rest of the stored procedures? Do I need to create several oledb sources?
I recommend doing in steps:
Create variables in SSIS:
xmldata string set = ""
SQLquery string set formula of "Exec getPartiesMarketData " + #xmldata
In execute sql task enter sql statement into Direct Input:
SELECT DISTINCT top 1000 (mc.companyId) as id
FROM ciqMarketCap mc
JOIN ciqCompany c
ON c.companyid = mc.companyid
WHERE c.companyStatusTypeId NOT IN (5,6) AND c.companyTypeId IN (1,4) for xml path(''), type, root('partyIds')
Change the result set to XML
Map result xmldata
Then in another execute sql connected you will do your real query:
SQL Source type: variable
Source Variable: SQLquery
I haven't tested this, but I am pretty sure it will work.
Related
I'm looking to replace connection strings across multiple stored procedures.The development database I'm working with is restored from production and stored procedures within contain connection strings to a production linked server. My aim is replace the linked server connection strings to point at a development linked server for testing purposes. I'm looking to automated this as a step in the SQL Agent restore job to run immediately after the restore.
The issue I'm having is setting the stored proc definition as a variable while keeping the formatting. When selecting the definition text, the stored proc is all on one line so after replacing the CREATE for ALTER along with the connection stings I cannot execute the SQL due to the formatting. I've tried playing around with STRING_SPLIT as but to no avail. Is there a way to do this or is it not possible?
Here is an example of part of one of the procedures
SELECT
a.AgreementNumber,
a.AgreementProposalID,
c.CustomerNumber,
pc.Id CustomerID,
a.AgreementCreateDate
INTO
#a
FROM
SENTINEL.DotDot_S3DB01_Replica.dbo.AgreementTable a
INNER JOIN
SENTINEL.DotDot_S3CUSTDB_Replica.dbo.CustomerTable c ON a.AgreementCustomerNumber = c.CustomerNumber
INNER JOIN
SENTINEL.DotDot_Proposal_Replica.dbo.Customer pc ON pc.Code = c.CustomerNumber
WHERE
a.AgreementCreateDate > #LastEnteredDate;
After my changes it needs to look like this to reference the test database names
SELECT
a.AgreementNumber,
a.AgreementProposalID,
c.CustomerNumber,
pc.Id CustomerID,
a.AgreementCreateDate
INTO
#a
FROM
SENTINEL.DotDotUAT_S3DB01.dbo.AgreementTable a
INNER JOIN
SENTINEL.DotDotUAT_S3CUSTDB.dbo.CustomerTable c ON a.AgreementCustomerNumber = c.CustomerNumber
INNER JOIN
SENTINEL.DotDotUAT_Proposal .dbo.Customer pc ON pc.Code = c.CustomerNumber
WHERE
a.AgreementCreateDate > #LastEnteredDate;
I've tried using this code to select the stored procedure code as a variable so I can execute the alter procedure command with the changes however the object definition selected from sys.procedures is displayed all on one line.
DECLARE #SQL VARCHAR(MAX)
SET #SQL = ( SELECT OBJECT_DEFINITION(object_id)
FROM sys.procedures
WHERE name = 'Usp_Proc')
SET #SQL = REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#SQL,'CREATE','ALTER'),'OriginalString','ReplacementString'),'OriginalString','ReplacementString'),'OriginalString','ReplacementString'),'OriginalString','ReplacementString')
EXEC (#SQL)
Does the command FOR XML in MS SQL Server save the file in disk?
I'm creating a trigger to log operations in a table and part of this trigger is create a XML with the affected row. I'm thinking of using the FOR XML to generate the XML.
SELECT *
FROM TBL_Test
WHERE ID=3040
FOR XML RAW
My worry is that I will be using it in a trigger and I don´t want to save files in the server every time I call the FOR XML function.
In addition: would you guys know how to parse it to varchar?
Thanks in advance for any help!
Like any SELECT query, a query with the FOR XML clause will return the result to the client and not save the result to disk.
You can use a scalar subquery to assign the result XML to a varchar variable instead of returning to the client:
DECLARE #xml varchar(MAX) =
(
SELECT *
FROM dbo.TBL_Test
WHERE ID=3040
FOR XML RAW
);
I am trying to push binary data from SQL Server to an Oracle LONG RAW column. I have a linked server created on SQL Server that connects to the Oracle server. I have a stored procedure on the Oracle side that I am trying to call from SQL Server. I can't seem to get the binary to pass into the stored procedure. I've tried changing the from and to types; however, the data ultimately needs to end up in a LONG RAW column. I have control of the Oracle stored procedure and the SQL Server code, but I do not have control of the predefined Oracle table structure.
varbinary(max) -> long raw
ORA-01460: unimplemented or unreasonable conversion requested
varbinary(max) -> blob
PLS-00306: wrong number or types of arguments in call to 'ADDDOC'
varbinary -> long raw
No errors, but get data truncation or corruption
The varbinary(max) does work if I set the #doc = null.
Below is the Oracle procedure and the SQL Server.
Oracle:
CREATE OR REPLACE
PROCEDURE ADDDOC (param1 IN LONG RAW)
AS
BEGIN
-- insert param1 into a LONG RAW column
DBMS_OUTPUT.PUT_LINE('TEST');
END ADDDOC;
SQL Server:
declare #doc varbinary(max)
select top 1 #doc = Document from Attachments
execute ('begin ADDDOC(?); end;', #doc) at ORACLE_DEV
-- tried this too, same error
--execute ('begin ADDDOC(utl_raw.cast_to_raw(?)); end;', #doc) at ORACLE_DEV
I've also tried creating the record in the Oracle Documents table then updating the LONG RAW field from SQL Server without invoking a stored procedure, but the query just seems to run and run and run and run...
--already created record and got the Id of the record I want to put the data in
--hard coding for this example
declare #attachmentId, #documentId
set #attachmentId = 1
set #documentId = 1
update ORACLE_DEV..MYDB.Documents
set Document = (select Document from Attachments where Id = #attachmentId)
where DocumentId=#documentId
As noted in the comments, LONG RAW is very difficult to work with; unfortunately, our vendor is using the datatype in their product and I have no choice but to work with it. I found that I could not pass binary data from SQL Server to an Oracle stored procedure parameter. I ended up having to create a new record with a NULL value for the LONG RAW field then using an OPENQUERY update to set the field to the VARBINARY(MAX) field; I did try using an update with the four part identifier, as noted in my code sample, but it took over 11 minutes for a single update, this new approach completes in less than 3 seconds. I am using an Oracle stored procedure here because in my real world scenario I am creating multiple records in multiple tables and coded business logic that is not relevant then tying them together with the docId.
This feels more like a workaround than a solution, but it actually works with acceptable performance.
Oracle:
create or replace procedure ADDDOC(docId OUT Number)
as
begin
select docseq.nextval into docId from dual;
-- insert new row, but leave Document LONG RAW field null for now
insert into DOC (Id) values(docId);
end ADDDOC;
SQL Server:
declare #DocId float, #AttachmentID int, #Qry nvarchar(max)
set #AttachmentID = 123 -- hardcoded for example
execute('begin ADDDOC(?); end;', #DocId output) at ORACLE_DEV
-- write openquery sql that will update Oracle LONG RAW field from a SQL Server varbinary(max) field
set #Qry = '
update openquery(ORACLE_DEV, ''select Document from Documents where Id=' + cast(#DocId as varchar) + ''')
set Document = (select Document from Attachments where Id = ' + cast(#AttachmentID as varchar) + ')
'
execute sp_executesql #Qry
I need to export data with SQL procedure from MSSQL (2012) to XML and run the script periodically with SQL Server Agent.
I would like to do it directly from SQL Server, because now I am doing it with console application which is generating XML structure from raw data from SQL and it takes over 10 hours! An output XML now has 300M.
Does somebody now how to do it? I already have stored procedure generating XML output.
Thanks much.
The procedure was executed successfully. I only had to set up rights for a destination folder for user "SQLSERVERAGENT", because Agent finishes successfully but wrote nothing.
But I have a small problem with final XML. At the beginning and at the end of XML file are included some information from Agent.
Do somebody know how to remove Header and Footer?
Header:
Job 'Xml Dump' : Step 1, 'Xml Dump Procedure' : Began Executing 2013-07-23 12:24:52
XML_F52E2B61-18A1-11d1-B105-00805F49916B
---------------------------------------------------------------------------------
Footer:
(146270 rows(s) affected)
Stored procedure contains this:
CREATE PROCEDURE GetXmlDumpCompleteData
(#Xml XML)
as
SET #Xml = (SELECT * FROM [MyTable]
FOR XML AUTO, ELEMENTS, ROOT('data'))
SELECT #Xml
Command in job agent:
DECLARE #Xml XML
EXEC [dbo].[GetXmlDumpCompleteData]
#Xml = #Xml OUTPUT
SELECT #Xml as N'#Xml'
GO
Is there a way to cause the result set of a SQL Server stored procedure (or any result set, after the fact) to be encoded in XML format?
I want the result set to be encoded in XML as if the FOR XML RAW clause was used during selection.
However the complex stored procedure logic and its internal SELECT statements should not be modified to return XML because the procedure is used for its standard/non-XML result set most of the time.
Update: Emphasis on the fact I'm looking for an answer in the SQL Server environment - the
results should be returned as if SQL Server has directly encoded them itself, as XML, just like it does when using the built-in XML features like the FOR XML clause.
You would insert the data from the SP into a temp table, then select from that FOR XML
This won't work if the SP itself already does a INSERT .. EXEC SPROC because you cannot nest them
Working examples
use tempdb;
create proc giveme
as
select a = 1, b = GETDATE()
union all
select 2, b = '20100101'
Using INSERT.. EXEC
declare #t table (a int, b datetime)
insert #t
exec giveme
select * from #t for xml raw
Using OPENQUERY
exec sp_addlinkedserver 'localhost'
exec sp_serveroption #server = 'localhost'
,#optname = 'DATA ACCESS'
,#optvalue = 'TRUE'
select *
from openquery(localhost, 'exec tempdb..giveme')
for xml raw
You could try using OPENROWSET in cooperation with FOR XML to do the transformation.
By 'after the fact', do you mean still within the SQL Server environment? Or are you talking about a client program?
Within SQL, you could probably write a sproc that acts as a wrapper for your other sprocs, along these lines. The wrapper sproc would handle the FOR XML work.
In .NET, there are a number of ways to do this.
You can try inserting the result set from the stored procedure into a table variable( or temporary table) and selecting the table rows with the FOR XML clause.
Here is an example:
DECLARE #MyDataTable AS TABLE ( col1 int,...., colN int)
Make sure that the #MyDataTable has the same columns as the stored procedure result set(s).
INSERT INTO #MyDataTable
EXECUTE mysp_GetData #param1=value,....,#paramN;
SELECT * FROM #MyDataTable
FOR XML AUTO