I have a database on AWS RSD, I have a basic account of AWS. I found that I have to store the XML file on internet, then I have to read it from the URL.
So, I store the XML file into a Github repository and then I execute the query below. On my local server I have to execute a RECONFIG to activate some permissions.
It worked well on my local server, but in my database in AWS it didn't work. It prints a error of permissions of sp_OACreate, sp_OAMethod, sp_OAGetProperty, sp_OADestroy. Also I don't have permissions to execute sp_configure and RECONFIGURE.
I tried to change setting in the server page but I didn't found anything.
DECLARE
#url VARCHAR(1000),
#url2 VARCHAR(300),
#win INT,
#hr INT,
#Text VARCHAR(4000),
#xml XML
SET #url = 'https://raw.githubusercontent.com/KevinFallas03/FacturacionMunicipal_BD/master/Base%20de%20Datos/XML/Administradores.xml'--current file
SET #url2 = 'http://www.bnr.ro/nbrfxrates.xml'--test file
EXEC #hr = sp_OACreate 'WinHttp.WinHttpRequest.5.1', #win OUT
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OAMethod #win, 'Open', NULL, 'GET', #url, 'false'
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OAMethod #win, 'Send'
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OAGetProperty #win, 'ResponseText', #Text OUT
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
EXEC #hr = sp_OADestroy #win
IF #hr <> 0 EXEC sp_OAGetErrorInfo #win
SELECT #Text
SET #xml=CAST(#Text AS XML)
SELECT #xml
Query to activate permissions, it works on local database:
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ole Automation Procedures', 1;
GO
RECONFIGURE;
GO
My intention is read a XML file from my database in AWS. So, if there is another way to do it, please tell me.
RDS offers many different types of database. It looks like you are using Microsoft SQL Server.
RDS is a managed database service. That means that you do not have full admin permissions on the database. You seem to be hoping to use OLE Automation, which requires the use of certain Stored Procedures like sp_OACreate.
OLE Automation is not supported in RDS Microsoft SQL Server. One alternative is to install and manage your own MS SQL Server running on EC2.
One option to solve the problem is to create a bucket on Amazon Storage S3, then upload the XML file on the bucket. Next you have to integrate AWS S3 buckets with AWS RDS SQL Server.
For complete guide to integrate AWS S3 buckets with AWS RDS SQL Server consulte here: https://www.sqlshack.com/integrating-aws-s3-buckets-with-aws-rds-sql-server/
With all set you can download the XML file with:
EXEC msdb.dbo.rds_download_from_s3
#s3_arn_of_file = 'arn:aws:s3:::<bucket-name>/<xml-name>.xml', --string with the ARN
#rds_file_path = 'D:\S3\<storage-name>\<xml-name>.xml', --where xml is saved
#overwrite_file = 1;
To check the status of the previuos task:
EXEC msdb..rds_task_status #task_id = <task_number>;
To parse the XML file:
SELECT CONVERT(XML, BulkColumn) AS BulkColumn, GETDATE()
FROM OPENROWSET(BULK 'D:\S3\<storage-name>\<xml-name>.xml', SINGLE_BLOB) AS x;
How do I find the ssl certificate path? That one installed by VisualStudio, to be more expecific when we use .NET Core and run "dotnet dev-certs https --trust" at the console...
Could you guys help me pls?
Everything I've tried drop me in the certificate name only. I could not see the path to set on that pattern 'LOCAL_MACHINE\My\'Certificate Name'.
I want to pass it ike a parameter on the SQL code below:
EXEC #hResult = sp_OAMethod
#Object,
'setOption',
NULL,
3,
'LOCAL_MACHINE\My\'Certificate Name';
Just because I'm trying to call a API using SQL Server and I need to use that SSL certificate to run my tests.
Could you guys help me pls?
The return message I've got untill now says that it's not a valid certificate but I think it's bacause I'm not finding the full path.
I don't know if I made this clear... Just summarazing... I need to do this by sql and I need to know how to use SSL certificate path as a parameter.
There follows the code I'm using:
begin
declare #test as int;
declare #Object as int;
declare #hResult as nvarchar(256)
declare #Response as nvarchar(256)
exec #test = sp_OACreate 'MSXML2.XMLHTTP', #Object out;
select #test, #Object;
EXEC #hResult = sp_OAMethod
#Object,
'setOption',
NULL,
3,
'LOCAL_MACHINE\My\'Certificate Name';
exec #test = sp_OAMethod #Object, 'open', null, 'get',
'https://localhost:44351/api/v1/account/3215', 'false'
select #test
exec #test = sp_OAMethod #Object, 'send'
IF #test <> 0
BEGIN
EXEC sp_OAGetErrorInfo #object
RETURN
END
exec sp_OAMethod #Object, 'responseText', #Response OUTPUT;
end
print #Response
I had the same problem using the same VS-generated cert. I could find the cert called "localhost" in the certmgr ("Personal"->"Certificates"->localhost), installed when I began the project. I added this line:
EXEC #hr = sp_OAMethod #Object,'setOption',NULL,3,'LOCAL_MACHINE\My\localhost';
and it worked.
However, i found I could change the line to anything and it still worked, ie:
EXEC #hr = sp_OAMethod #Object,'setOption',NULL,3,'LOCAL_MACHINE\My\bogusfake';
So I cannot promise that will work for you, nor explain how/why, but I am able to proceed with the project of posting from a stored procedure to a local api. The only difference I see between my code and yours is I didn't surround the cert name in quotes.
I'm not familiar with Microsoft SQL Server and the varbinary(max) data type but the database which I need to use stores images as that.
My question is what do I need to do to read that data back and convert it back to readable image type?
So far I have done was
SELECT CONVERT(VARCHAR(max), [IMAGE], 2)
FROM [demo].[dbo].[DOCIMAGES]
WHERE [TITLE] = 'test.jpg'
but what I have got it was almost the same just removed 0x from the beginning of that data when
SELECT CONVERT(VARCHAR(max), [IMAGE], 0)
FROM [demo].[dbo].[DOCIMAGES]
WHERE [TITLE] = 'test.jpg'
returns value as Lead.
Then I have tried to do stored procedure which will save it to a file like:
DECLARE #ImageData VARBINARY (max);
DECLARE #Path2OutFile NVARCHAR (2000);
DECLARE #Obj INT
SET NOCOUNT ON
SELECT #ImageData = (
SELECT CONVERT(VARBINARY(max), [IMAGE], 1)
FROM [demo].[dbo].[DOCIMAGES]
WHERE [TITLE] = 'test.jpg'
);
SET #Path2OutFile = CONCAT (
'C:\Users\MyPC\Downloads'
,'\'
,'test.jpg'
);
BEGIN TRY
EXEC sp_OACreate 'ADODB.Stream' ,#Obj OUTPUT;
EXEC sp_OASetProperty #Obj ,'Type',1;
EXEC sp_OAMethod #Obj,'Open';
EXEC sp_OAMethod #Obj,'Write', NULL, #ImageData;
EXEC sp_OAMethod #Obj,'SaveToFile', NULL, #Path2OutFile, 2;
EXEC sp_OAMethod #Obj,'Close';
EXEC sp_OADestroy #Obj;
END TRY
BEGIN CATCH
EXEC sp_OADestroy #Obj;
END CATCH
SET NOCOUNT OFF
but that fails with an error:
SQL Server blocked access to procedure 'sys.sp_OADestroy' of component
'Ole Automation Procedures' because this component is turned off as
part of the security configuration for this server. A system
administrator can enable the use of 'Ole Automation Procedures' by
using sp_configure.
Problem is that I'm not the admin of that server.
In that case can I dump content of the [IMAGE] to a file and then use Java or PHP to stream that content and save it as an image file?
I want to call a Webservice when a row inserted into a table.
So I enable Ole Automation Procedures, call Webservice like below and everything goes fine.
--enabling 'Ole Automation Procedures'
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ole Automation Procedures', 1;
GO
RECONFIGURE;
GO
--calling Webservice
create PROCEDURE callwebservice
#url varchar(128)=null
AS
Declare #Object as Int;
Declare #ResponseText as varchar(8000);
Exec sp_OACreate 'MSXML2.ServerXMLHttp.3.0', #Object OUT;
Exec sp_OAMethod #Object, 'open', NULL, 'get', #url ,'false'
Exec sp_OAMethod #Object, 'send'
Exec sp_OAGetProperty #Object, 'ResponseText', #ResponseText OUTPUT
Exec sp_OADestroy #Object
return
But, Since Webservice call is extremely lengthy, I need to do it with Service Broker. so I create a new stored-procedure and pass it as an activation parameter of my Queue like this:
--declare procedure
create procedure handleNewMessageInQueue
as
begin
DECLARE #TargetDlgHandle UNIQUEIDENTIFIER
DECLARE #ReplyMessage xml
DECLARE #ReplyMessageName
BEGIN TRAN;
receive top(1)
#TargetDlgHandle =Conversation_handle
,#ReplyMessage = Message_Body
,#ReplyMessageName = Message_Type_Name
from newUserQueue;
if #ReplyMessageName = '//goodType'
begin
exec callwebservice 'http://path/to/my/webservice'
end
commit tran;
end
--declare queue
create queue myQueue
with ACTIVATION (
PROCEDURE_NAME = handleNewMessageInQueue,
MAX_QUEUE_READERS =100,
EXECUTE AS owner
);
But this time, no request sent. I checked and I realize that the sp_OACreate method does not create the Object since the value of #Object is null after execution of it. Also when I call sp_OAGetErrorInfo to get error info, everything is null
I'm using SQL Server 2016
I appreciate if you can find the problem here.
EDIT
I found that if I enable TRUSTWORTHY on that database everything goes fine. but enabling it can open security hole.
is there any better way to do that?
Is there a way to call out from a TSQL stored procedure or function to a webservice?
Yes , you can create like this
CREATE PROCEDURE CALLWEBSERVICE(#Para1 ,#Para2)
AS
BEGIN
Declare #Object as Int;
Declare #ResponseText as Varchar(8000);
Exec sp_OACreate 'MSXML2.XMLHTTP', #Object OUT;
Exec sp_OAMethod #Object, 'open', NULL, 'get', 'http://www.webservicex.com/stockquote.asmx/GetQuote?symbol=MSFT','false'
Exec sp_OAMethod #Object, 'send'
Exec sp_OAMethod #Object, 'responseText', #ResponseText OUTPUT
Select #ResponseText
Exec sp_OADestroy #Object
END
Sure you can, but this is a terrible idea.
As web-service calls may take arbitrary amounts of time, and randomly fail, depending on how many games of counterstrike are being played on your network at the time, you can't tell how long this is going to take.
At the bare minimum you're looking at probably half a second by the time it builds the XML, sends the HTTP request to the remote server, which then has to parse the XML and send a response back.
Whichever application did the INSERT INTO BLAH query which caused the web-service to fire is going to have to wait for it to finish. Unless this is something that only happens in the background like a daily scheduled task, your app's performance is going to bomb
The web service-invoking code runs inside SQL server, and uses up it's resources. As it's going to take a long time to wait for the HTTP request, you'll end up using up a lot of resources, which will again hurt the performance of your server.
Not in T-SQL code itself, but with SQL Server 2005 and above, they've enabled the ability to write CLR stored procedures, which are essentially functions in .NET code and then expose them as stored procedures for consumption. You have most of the .NET framework at your fingertips for this, so I can see consumption of a web service possible through this.
It is a little lengthy to discuss in detail here, but here's a link to an MSDN article on the topic.
I would not do this for heavy traffic or mission critical stuff, HOWEVER, if you do NOT need to receive feedback from a service, then it is actually a great thing to do.
Here is an example of what I have done.
Triggers Insert and Update on a Table
Trigger called Stored Proc that is passes the JSON data of the transaction to a Web Api Endpoint that then Inserts into a MongoDB in AWS.
Don't do old XML
JSON
EXEC sp_OACreate 'WinHttp.WinHttpRequest.5.1', #Object OUT;
EXEC sp_OAMethod #Object, 'Open', NULL, 'POST', 'http://server/api/method', 'false'
EXEC sp_OAMethod #Object, 'setRequestHeader', null, 'Content-Type', 'application/json'
DECLARE #len INT = len(#requestBody)
Full example:
Alter Procedure yoursprocname
#WavName varchar(50),
#Dnis char(4)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #Object INT;
DECLARE #Status INT;
DECLARE #requestBody NVARCHAR(MAX) = '{
"WavName": "{WavName}",
"Dnis": "{Dnis}"
}'
SET #requestBody = REPLACE(#requestBody, '{WavName}', #WavName)
SET #requestBody = REPLACE(#requestBody, '{Dnis}', #Dnis)
EXEC sp_OACreate 'WinHttp.WinHttpRequest.5.1', #Object OUT;
EXEC sp_OAMethod #Object, 'Open', NULL, 'POST', 'http://server/api/method', 'false'
EXEC sp_OAMethod #Object, 'setRequestHeader', null, 'Content-Type', 'application/json'
DECLARE #len INT = len(#requestBody)
EXEC sp_OAMethod #Object, 'setRequestHeader', null, 'Content-Length', #len
EXEC sp_OAMethod #Object, 'send', null, #requestBody
EXEC sp_OAGetProperty #Object, 'Status', #Status OUT
EXEC sp_OADestroy #Object
In earlier versions of Sql, you could use either an extended stored proc or xp_cmdshell to shell out and call a webservice.
Not that either of these sound like a decent architecture - but sometimes you have to do crazy stuff.
You can do it with the embedded VB objects.
First you create one VB object of type 'MSXML2.XMLHttp', and you use this one object for all of your queries (if you recreate it each time expect a heavy performance penalty).
Then you feed that object, some parameters, into a stored procedure that invokes sp_OAMethod on the object.
Sorry for the inprecise example, but a quick google search should reveal how the vb-script method is done.
--
But the CLR version is much....MUCH easier.
The problem with invoking webservices is that they cannot keep pace with the DB engine. You'll get lots of errors where it just cannot keep up.
And remember, web SERVICES require a new connection each time. Multiplicity comes into play. You don't want to open 5000 socket connections to service a function call on a table. Thats looney!
In that case you'd have to create a custom aggregate function, and use THAT as an argument to pass to your webservice, which would return a result set...then you'd have to collate that. Its really an awkward way of getting data.
Here'a an example to get some data from a webservice. In this case parse a user agent string to JSON.
--first configure MSSQL to enable calling out to a webservice (1=true, 0=false)
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'Ole Automation Procedures', 1;
GO
RECONFIGURE;
GO
CREATE PROCEDURE CallWebAPI_ParseUserAgent #UserAgent VARCHAR(512)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #Object INT;
DECLARE #ResponseText AS VARCHAR(8000);
DECLARE #url VARCHAR(512)
SET #url = 'http://www.useragentstring.com/?getJSON=all&uas=' + #UserAgent;
EXEC sp_OACreate 'WinHttp.WinHttpRequest.5.1', #Object OUT;
EXEC sp_OAMethod #Object, 'Open', NULL, 'GET', #url, 'false'
EXEC sp_OAMethod #Object, 'setRequestHeader', NULL, 'Content-Type', 'application/json'
EXEC sp_OAMethod #Object, 'send'
EXEC sp_OAMethod #Object, 'responseText', #ResponseText OUTPUT
SELECT #ResponseText
EXEC sp_OADestroy #Object
END
--example how to call the API
CallWebAPI_ParseUserAgent 'Mozilla/5.0 (Windows NT 6.2; rv:53.0) Gecko/20100101 Firefox/53.0'
If you're working with sql 2000 compatibility levels and cannot do clr integration, see http://www.vishalseth.com/post/2009/12/22/Call-a-webservice-from-TSQL-(Stored-Procedure)-using-MSXML.aspx
I been working for big/global companies around the world, using Oracle databases. We are consuming web services all time thru DB with store procedures and no issues, even those ones with heavy traffic. All of them for internal use, I mean with no access to internet, only inside the plant. I would recommend to use it but being really careful about how you design it