I got a stored proc to retrieve XML data from web service It works as I expected. After test for a while, I want to use this stored proc as SCHEDULED JOB then I created new one with T-SQL 'EXEC sp_GetData;' and got failed results (error 9400).
I reproduced that my stored proc can work fine if I execute it via Query Editor Window (EXEC sp_GetData;) but can't work if retrieving data are more than 3 objects of following XML data via scheduled job.
Question :
How to increase SQL Server Agent to retrieve all data (about 3MB file size) then insert into specific table.
<ArrayOfEmployeeMobilesInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://tempuri.org/">
<EmployeeMobilesInfo>
<EmployeeCode>00001</EmployeeCode>
<TitleEN>Mr.</TitleEN>
<EmpNameEN>John</EmpNameEN>
<EmpSurnameEN>Doe</EmpSurnameEN>
<PositionNameEN>Police</PositionNameEN>
<Title>Mr.</Title>
<EmpName>John</EmpName>
<EmpSurname>Doe</EmpSurname>
<PositionName>Police</PositionName>
<CenterCode>PP777</CenterCode>
<Location>PP</Location>
<Email>john#person.com</Email>
<DivisionCode>PPD77</DivisionCode>
<DivisionName>PPD77</DivisionName>
<DepartmentCode>PDD25</DepartmentCode>
<DepartmentName>PDD25</DepartmentName>
<PositionLevel>POC</PositionLevel>
<PositionWeight>H</PositionWeight>
<JoinDate>2013-10-01T00:00:00</JoinDate>
<Mobiles>
<Mobile>
<MobileNo>1500111232</MobileNo>
</Mobile>
</Mobiles>
<TerminateDate xsi:nil="true" />
</EmployeeMobilesInfo>
... x 2
</ArrayOfEmployeeMobilesInfo>
my stored proc.
SET NOCOUNT ON;
DECLARE #URL NVARCHAR(500);
DECLARE #obj INT;
IF OBJECT_ID('xml_employees', 'U') IS NOT NULL DROP TABLE xml_employees;
IF OBJECT_ID('Employees', 'U') IS NOT NULL DROP TABLE Employees;
CREATE TABLE xml_employees ( resp XML );
SET #URL = 'http://localhost/DataService/Employee.asmx/GetEmployees';
EXEC sp_OACreate 'MSXML2.ServerXMLHttp', #obj OUT
EXEC sp_OAMethod #obj, 'Open', NULL, 'GET', #URL , false
EXEC sp_OAMethod #obj, 'send'
INSERT xml_employees ( resp )
EXEC sp_OAGetProperty #obj, 'responseXML.xml'
Error message
XML parsing: line 12, character 18, unexpected end of input [SQLSTATE 42000] (Error 9400). The step failed.
Please give me some advice.
Thank you
EDIT 1: Try to set TEXTSIZE as this thread mention
SET TEXTSIZE 2147483647; save my ass !
Default TEXTSIZE of SQL Server Agent is 4KB, BUT my instance got 1KB.
Related
I have a database on AWS RSD, I have a basic account of AWS. I found that I have to store the XML file on internet, then I have to read it from the URL.
So, I store the XML file into a Github repository and then I execute the query below. On my local server I have to execute a RECONFIG to activate some permissions.
It worked well on my local server, but in my database in AWS it didn't work. It prints a error of permissions of sp_OACreate, sp_OAMethod, sp_OAGetProperty, sp_OADestroy. Also I don't have permissions to execute sp_configure and RECONFIGURE.
I tried to change setting in the server page but I didn't found anything.
DECLARE
#url VARCHAR(1000),
#url2 VARCHAR(300),
#win INT,
#hr INT,
#Text VARCHAR(4000),
#xml XML
SET #url = 'https://raw.githubusercontent.com/KevinFallas03/FacturacionMunicipal_BD/master/Base%20de%20Datos/XML/Administradores.xml'--current file
SET #url2 = 'http://www.bnr.ro/nbrfxrates.xml'--test file
EXEC #hr = sp_OACreate 'WinHttp.WinHttpRequest.5.1', #win OUT
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OAMethod #win, 'Open', NULL, 'GET', #url, 'false'
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OAMethod #win, 'Send'
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OAGetProperty #win, 'ResponseText', #Text OUT
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OADestroy #win
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
SELECT #Text
SET #xml=CAST(#Text AS XML)
SELECT #xml
Query to activate permissions, it works on local database:
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ole Automation Procedures', 1;
GO
RECONFIGURE;
GO
My intention is read a XML file from my database in AWS. So, if there is another way to do it, please tell me.
RDS offers many different types of database. It looks like you are using Microsoft SQL Server.
RDS is a managed database service. That means that you do not have full admin permissions on the database. You seem to be hoping to use OLE Automation, which requires the use of certain Stored Procedures like sp_OACreate.
OLE Automation is not supported in RDS Microsoft SQL Server. One alternative is to install and manage your own MS SQL Server running on EC2.
One option to solve the problem is to create a bucket on Amazon Storage S3, then upload the XML file on the bucket. Next you have to integrate AWS S3 buckets with AWS RDS SQL Server.
For complete guide to integrate AWS S3 buckets with AWS RDS SQL Server consulte here: https://www.sqlshack.com/integrating-aws-s3-buckets-with-aws-rds-sql-server/
With all set you can download the XML file with:
EXEC msdb.dbo.rds_download_from_s3
#s3_arn_of_file = 'arn:aws:s3:::<bucket-name>/<xml-name>.xml', --string with the ARN
#rds_file_path = 'D:\S3\<storage-name>\<xml-name>.xml', --where xml is saved
#overwrite_file = 1;
To check the status of the previuos task:
EXEC msdb..rds_task_status #task_id = <task_number>;
To parse the XML file:
SELECT CONVERT(XML, BulkColumn) AS BulkColumn, GETDATE()
FROM OPENROWSET(BULK 'D:\S3\<storage-name>\<xml-name>.xml', SINGLE_BLOB) AS x;
I have the following issue. running the sql below on our server it returns the expected results. Running the same on another server it returns no values.
Did the following:
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ole Automation Procedures', 1;
GO
RECONFIGURE;
GO
USE tempdb
GO
IF OBJECT_ID('tempdb..#xml') IS NOT NULL DROP TABLE #xml
CREATE TABLE #xml ( yourXML XML )
GO
DECLARE #URL VARCHAR(8000)
--DECLARE #QS varchar(50)
-- & or ? depending if there are other query strings
-- Use this for when there is other query strings:
--SELECT #QS = '&date='+convert(varchar(25),getdate(),126)
-- Use this for when there is NO other query strings:
-- SELECT #QS = '?date='+convert(varchar(25),getdate(),126)
SELECT #URL = 'http://exampleURL' -- + #QS
DECLARE #Response varchar(8000)
DECLARE #XML xml
DECLARE #Obj int
DECLARE #Result int
DECLARE #HTTPStatus int
DECLARE #ErrorMsg varchar(MAX)
EXEC #Result = sp_OACreate 'MSXML2.XMLHttp', #Obj OUT
EXEC #Result = sp_OAMethod #Obj, 'open', NULL, 'GET', #URL, false
EXEC #Result = sp_OAMethod #Obj, 'setRequestHeader', NULL, 'Content-Type', 'application/x-www-form-urlencoded'
EXEC #Result = sp_OAMethod #Obj, send, NULL, ''
EXEC #Result = sp_OAGetProperty #Obj, 'status', #HTTPStatus OUT
INSERT #xml ( yourXML )
EXEC #Result = sp_OAGetProperty #Obj, 'responseXML.xml'--, #Response OUT
declare #input XML=(
SELECT
yourXML
from
#xml)
SELECT
Item.value('(Code)[1]', 'nvarchar(max)') as Code,
Item.value('(Description)[1]', 'varchar(max)') as Description,
Item.value('(ImageUrl)[1]', 'nvarchar(max)') as ImageUrl
from
#input.nodes('//product') AS T(Item)
Within the second server the #input returns null. There is a proxy to access the site on the server and it operates with sql server 2008.
Any ideas why the null values?
I just had a similar issue, a query using Ole Automation suddenly returns null without any error. After changing 'MSXML2.XMLHttp' to 'MSXML2.ServerXMLHTTP', it started to work again.
To know more about the difference between these two, see this article and Microsoft documentation. I copied some from Microsoft site, in case both sites are down in the future.
The ServerXMLHTTP object offers functionality similar to that of the XMLHTTP object. Unlike XMLHTTP, however, the ServerXMLHTTP object does not rely on the WinInet control for HTTP access to remote XML documents. ServerXMLHTTP uses a new HTTP client stack. Designed for server applications, this server-safe subset of WinInet offers the following advantages:
Reliability — The HTTP client stack offers longer uptimes. WinInet features that are not critical for server applications, such as URL caching, auto-discovery of proxy servers, HTTP/1.1 chunking, offline support, and support for Gopher and FTP protocols are not included in the new HTTP subset.
Security — The HTTP client stack does not allow a user-specific state to be shared with another user's session. ServerXMLHTTP provides support for client certificates.
So, it seems that my issue was that I had no access over the internet from my SQL server, so I had to use the below line to set the proxy.
exec #Result = sp_OAMethod #Obj, 'setProxy', NULL, '2', 'http://myProxy'
Once this was sorted, I managed to get my results.
I'm not familiar with Microsoft SQL Server and the varbinary(max) data type but the database which I need to use stores images as that.
My question is what do I need to do to read that data back and convert it back to readable image type?
So far I have done was
SELECT CONVERT(VARCHAR(max), [IMAGE], 2)
FROM [demo].[dbo].[DOCIMAGES]
WHERE [TITLE] = 'test.jpg'
but what I have got it was almost the same just removed 0x from the beginning of that data when
SELECT CONVERT(VARCHAR(max), [IMAGE], 0)
FROM [demo].[dbo].[DOCIMAGES]
WHERE [TITLE] = 'test.jpg'
returns value as Lead.
Then I have tried to do stored procedure which will save it to a file like:
DECLARE #ImageData VARBINARY (max);
DECLARE #Path2OutFile NVARCHAR (2000);
DECLARE #Obj INT
SET NOCOUNT ON
SELECT #ImageData = (
SELECT CONVERT(VARBINARY(max), [IMAGE], 1)
FROM [demo].[dbo].[DOCIMAGES]
WHERE [TITLE] = 'test.jpg'
);
SET #Path2OutFile = CONCAT (
'C:\Users\MyPC\Downloads'
,'\'
,'test.jpg'
);
BEGIN TRY
EXEC sp_OACreate 'ADODB.Stream' ,#Obj OUTPUT;
EXEC sp_OASetProperty #Obj ,'Type',1;
EXEC sp_OAMethod #Obj,'Open';
EXEC sp_OAMethod #Obj,'Write', NULL, #ImageData;
EXEC sp_OAMethod #Obj,'SaveToFile', NULL, #Path2OutFile, 2;
EXEC sp_OAMethod #Obj,'Close';
EXEC sp_OADestroy #Obj;
END TRY
BEGIN CATCH
EXEC sp_OADestroy #Obj;
END CATCH
SET NOCOUNT OFF
but that fails with an error:
SQL Server blocked access to procedure 'sys.sp_OADestroy' of component
'Ole Automation Procedures' because this component is turned off as
part of the security configuration for this server. A system
administrator can enable the use of 'Ole Automation Procedures' by
using sp_configure.
Problem is that I'm not the admin of that server.
In that case can I dump content of the [IMAGE] to a file and then use Java or PHP to stream that content and save it as an image file?
I am calling a SSIS package inside procedure using xm_cmdshell. Below is the part of code from stored procedure.
DECLARE #ReturnCode INT
EXEC #ReturnCode=xp_cmdshell #cmd
IF #ReturnCode <>0
BEGIN
END
#cmd has the DTEXEC command to execute SSIS package.
If SSIS package fails I want to access SSIS error message inside IF clause. How can I achieve this?
To achieve what you want, use SSIS logging from within an SSIS package. For example you can log to a table. In your SQL script, you can read that table after calling xp_cmdshell to get errors.
Note also that MS is moving away from DTExec, look into SSIS catalog
Partially, you can achieve this following way:
DECLARE #output TABLE (lines varchar(2000))
DECLARE #ReturnCode INT
INSERT INTO #output
EXEC #ReturnCode=xp_cmdshell #cmd
IF #ReturnCode <>0
BEGIN
--do something with output
SELECT lines FROM #output WHERE lines IS NOT NULL
END
However, as #under mentioned, consider to use SSIS Catalog. in that case accomplishment of your task could be much simpler:
Start of SSIS package:
Declare #execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] #package_name=N'Package.dtsx',
#execution_id=#execution_id OUTPUT,
#folder_name=N'Deployed Projects',
#project_name=N'Integration Services Project1',
#use32bitruntime=False,
#reference_id=Null
Select #execution_id
EXEC [SSISDB].[catalog].[start_execution] #execution_id
Querying for errors:
SELECT * FROM SSISDB.catalog.event_messages
WHERE operation_id = #execution_id AND event_name = 'OnError'
I want to call a Webservice when a row inserted into a table.
So I enable Ole Automation Procedures, call Webservice like below and everything goes fine.
--enabling 'Ole Automation Procedures'
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ole Automation Procedures', 1;
GO
RECONFIGURE;
GO
--calling Webservice
create PROCEDURE callwebservice
#url varchar(128)=null
AS
Declare #Object as Int;
Declare #ResponseText as varchar(8000);
Exec sp_OACreate 'MSXML2.ServerXMLHttp.3.0', #Object OUT;
Exec sp_OAMethod #Object, 'open', NULL, 'get', #url ,'false'
Exec sp_OAMethod #Object, 'send'
Exec sp_OAGetProperty #Object, 'ResponseText', #ResponseText OUTPUT
Exec sp_OADestroy #Object
return
But, Since Webservice call is extremely lengthy, I need to do it with Service Broker. so I create a new stored-procedure and pass it as an activation parameter of my Queue like this:
--declare procedure
create procedure handleNewMessageInQueue
as
begin
DECLARE #TargetDlgHandle UNIQUEIDENTIFIER
DECLARE #ReplyMessage xml
DECLARE #ReplyMessageName
BEGIN TRAN;
receive top(1)
#TargetDlgHandle =Conversation_handle
,#ReplyMessage = Message_Body
,#ReplyMessageName = Message_Type_Name
from newUserQueue;
if #ReplyMessageName = '//goodType'
begin
exec callwebservice 'http://path/to/my/webservice'
end
commit tran;
end
--declare queue
create queue myQueue
with ACTIVATION (
PROCEDURE_NAME = handleNewMessageInQueue,
MAX_QUEUE_READERS =100,
EXECUTE AS owner
);
But this time, no request sent. I checked and I realize that the sp_OACreate method does not create the Object since the value of #Object is null after execution of it. Also when I call sp_OAGetErrorInfo to get error info, everything is null
I'm using SQL Server 2016
I appreciate if you can find the problem here.
EDIT
I found that if I enable TRUSTWORTHY on that database everything goes fine. but enabling it can open security hole.
is there any better way to do that?