I want to execute the OpenRowSet command to pull the contents of a .txt file into 1 column. In my script, I need to loop through a collection of existing records and build a dynamic file path for each .txt file I want to import into the database, and therefore, I am using the sp_executesql procedure to execute the OpenRowSet command and output the contents of the .txt file to an output parameter.
I have tested OpenRowSet with a hard-coded file path and without passing it to the sp_executesql procedure, and I am able to retrieve the .txt file contents and insert it into my desired table in SQL. All of that is working. The issue I run into with the sp_executesql procedure is the output parameter is coming back empty. Here is a snip of the code I am running in MSSQL SMS version 15.0.18206.0 on Windows Server 2019.
DECLARE #rootDirectory VARCHAR(100)
DECLARE #filePathWithName VARCHAR(255)
DECLARE #txtFileContents VARCHAR(MAX)
DECLARE #commandText NVARCHAR(MAX)
Set #commandText = N'(Select BulkColumn FROM OPENROWSET (BULK '''+ #rootDirectory + #filePathWithName + ''', SINGLE_CLOB) Myfile)'
-- Print the command text, should contain full text file path.
print 'Command Text: ' + #commandText
-- Execute command and output text file contents.
EXEC sp_executesql #commandText,
N'#fileContentsOut VARCHAR(MAX) OUTPUT',
#fileContentsOut = #txtFileContents OUTPUT;
-- Select the file contents output.
SELECT #txtFileContents; -- <-- comes back empty???
When I execute the command above, I get a query result window that shows the contents of the .txt file in a column called "BulkColumn", but #txtFileContents parameter is empty.
Here is what the #commandText looks like before executed:
Command Text: (Select BulkColumn FROM OPENROWSET (BULK 'F:\Assets\mypath\myfile12345.TXT', SINGLE_CLOB) Myfile)
I don't understand why #txtfileContents comes back empty when I select it.
Try this instead:
DECLARE #txtFileContents VARCHAR(MAX);
SELECT #txtFileContents = BulkColumn
FROM OPENROWSET ( BULK 'F:\Assets\mypath\myfile12345.TXT', SINGLE_CLOB ) AS x;
SELECT #txtFileContents AS TxtFileContents;
UPDATE:
I am iterating through a list of records and building a dynamic file path each time through...
This works for me:
DECLARE
#cmd nvarchar(1000),
#file nvarchar(255) = 'F:\Assets\mypath\myfile12345.TXT',
#txtFileContents varchar(MAX);
SET #cmd = FORMATMESSAGE ( 'SELECT #txtFileContents = BulkColumn FROM OPENROWSET ( BULK ''%s'', SINGLE_CLOB ) AS x;', #file );
EXEC sp_executesql #cmd, N'#txtFileContents varchar(MAX) OUT', #txtFileContents = #txtFileContents OUT;
SELECT #txtFileContents AS TxtFileContents;
It turns out you're missing SELECT #fileContentsOut = BulkColumn in your #commandText.
This:
SET #commandText = N'SELECT BulkColumn FROM OPENROWSET (BULK '''+ #rootDirectory + #filePathWithName + ''', SINGLE_CLOB) Myfile;'
Needs to be:
SET #commandText = N'SELECT #fileContentsOut = BulkColumn FROM OPENROWSET (BULK '''+ #rootDirectory + #filePathWithName + ''', SINGLE_CLOB) Myfile;'
Could use SSIS with a for each loop container.
https://www.red-gate.com/simple-talk/sql/ssis/ssis-basics-introducing-the-foreach-loop-container/
Related
I'm using T-SQL. The goal is to insert multiples files into a database.
If I'm using without a loop, it's working fine.
In the loop, I always get this error:
#InputXML should be declared
My code:
IF OBJECT_ID('TEMPDB..#TEMP_FILES') IS NOT NULL
DROP TABLE #TEMP_FILES
CREATE TABLE #TEMP_FILES
(
FileName VARCHAR(MAX),
DEPTH VARCHAR(MAX),
[FILE] VARCHAR(MAX)
)
INSERT INTO #TEMP_FILES
EXEC master.dbo.xp_DirTree '\\MyServer\MyFolder\',1,1
DELETE FROM #TEMP_FILES WHERE RIGHT(FileName,4) != '.XML'
--
SET QUOTED_IDENTIFIER ON
GO
TRUNCATE Table MyTable2
DECLARE #InputXML XML
DECLARE #FILENAME VARCHAR(MAX),#SQL VARCHAR(MAX)
WHILE EXISTS(SELECT * FROM #TEMP_FILES)
BEGIN
SET #FILENAME = (SELECT TOP 1 FileName FROM #TEMP_FILES)
SET #sql = 'SELECT #InputXML = CAST(x AS XML) FROM OPENROWSET(BULK \\MyServer\MyFolder\'''+ #FILENAME +''', SINGLE_BLOB) AS T(x)
INSERT INTO MyTable2 ([id],[version], [name], [listId], [listCode])
SELECT
product.value(''(#id)[1]'', ''NVARCHAR(10)''),
product.value(''(#version)[1]'', ''NVARCHAR(14)''),
product.value(''(name[1])'', ''NVARCHAR(255)''),
product.value(''(listId[1])'', ''NVARCHAR(9)''),
product.value(''(listCode[1])'', ''NVARCHAR(10)'')
FROM #InputXML.nodes(''xxx/values/value'') AS X(product)'
EXEC(#SQL)
DELETE FROM #TEMP_FILES
WHERE FileName = #FILENAME
END
You need to declare the variable inside the dynamic SQL (which should be nvarchar not varchar). You should also use QUOTENAME to ensure no issues with the filename:
DECLARE #sql nvarchar(max) = N'
DECLARE #InputXML XML;
SELECT #InputXML = CAST(x AS XML) FROM OPENROWSET(BULK ' + QUOTENAME(N'\\MyServer\MyFolder\' + #FILENAME, '''') + N', SINGLE_BLOB) AS T(x)
INSERT INTO MyTable2 ([id],[version], [name], [listId], [listCode])
SELECT
product.value(''(#id)[1]'', ''NVARCHAR(10)''),
product.value(''(#version)[1]'', ''NVARCHAR(14)''),
product.value(''(name[1])'', ''NVARCHAR(255)''),
product.value(''(listId[1])'', ''NVARCHAR(9)''),
product.value(''(listCode[1])'', ''NVARCHAR(10)'')
FROM #InputXML.nodes(''xxx/values/value'') AS X(product)'
I will say though, that I urge you to find another method to load files into SQL Server. Dynamic OPENROWSET, especially from user input, is not advisable. Bulk Insert or BCP may be an option.
So I'm trying to read a bunch of PDFs from a folder into an SQL table, saving them in a varbinary(max) field. This is what I thought would work at first:
CREATE TABLE tempFileName(filnavn VARCHAR(100));
INSERT INTO tempFileName
EXEC xp_cmdshell 'dir /B "C:\temp\Test Folder\"';
--------
DECLARE #path VARCHAR(100) SET #path = 'C:\temp\Test Folder\'
DECLARE #pdf VARBINARY(MAX)
DECLARE #navn varchar(50)
DECLARE #fullpath nvarchar(max)
DECLARE #sql nvarchar(max)
DECLARE c CURSOR FOR
SELECT filnavn
FROM tempFileName
OPEN c
FETCH NEXT FROM c INTO #navn
WHILE(##FETCH_STATUS = 0)
BEGIN
SET #fullpath = #path + #navn
SELECT #pdf = BulkColumn
FROM OPENROWSET(BULK #fullpath, SINGLE_BLOB) AS Document;
--print #sql
INSERT INTO pdftest VALUES(#navn, #pdf)
FETCH NEXT FROM c INTO #navn
END
CLOSE c
DEALLOCATE c
But this doesn't work as it won't allow me to use a variable in this line:
FROM OPENROWSET(BULK #fullpath, SINGLE_BLOB) AS Document;
So I'm pretty sure the trick is to make the whole "select #pdf.." line into a string and then execute it, but I'm not sure how to get the output into a table. I've tried something like this:
SET #fullpath = #path + #navn
SET #sql = 'DECLARE #pdf VARBINARY(MAX) SELECT #pdf = BulkColumn
FROM OPENROWSET(BULK ''' + #fullpath + ''' , SINGLE_BLOB) AS Document;'
--print #sql
--SELECT #pdf, DATALENGTH(#pdf)
--INSERT INTO pdftest VALUES(#navn, #pdf)
EXEC sp_executesql #sql, N'#fil varbinary(max) out', #fil out
But the #fil variable is just empty after this. How do I best go about getting these files into a table?
Why not just skip the variable assignment of the SELECT in your loop and use the OPENROWSET function inside your INSERT? The general idea:
INSERT INTO pdftest SELECT #navn, * FROM OPENROWSET(BULK, 'C:\thefile.txt', SINGLE_BLOB) AS document
And of course turn above into dynamic SQL. I'll probably get a few single-quotes wrong here, but again the general idea:
SET #sql =
'INSERT INTO pdftest
SELECT '' + #navn + '', *
FROM OPENROWSET(BULK, ''' + #fullpath + ''', SINGLE_BLOB) AS document
'
I am trying to insert file through SQL. I use following query.
INSERT INTO [dbo].[Attachments] (FileName, FileBinary)
SELECT
'non-date-in-sql-server-column',
BulkColumn
FROM
OPENROWSET(Bulk 'C:\Users\Pictures\Picture.JPG', SINGLE_BLOB) AS BLOB
It's working fine.
I want to write the procedure that take dynamic path. Its giving me error that I cannot take Filebinary in addin. Which is datatype varbinary. What is the best way to do ?
I have done following but its not taking properly binary value.
DECLARE #SQLString NVARCHAR(MAX)
SET #SQLString = 'SELECT ' + '''' +#Filename +'''' + ' AS Name,' + 'FileBinary
FROM OPENROWSET(BULK N''' + #ImagePath + ''',SINGLE_BLOB) AS FileBinary(FileBinary);'
Insert Into Attachments (ApplicantID, FileName, FileBinary)
Values (#ApplicantID, #FileName, Convert(varbinary(max), #SQLString))
Put the Insert statement inside a dynamic query and execute it.
Now your #SQLString will not have the FileBinary value it will have the dynamically framed string . You need to execute it to get the values
DECLARE #SQLString NVARCHAR(MAX),
#Filename VARCHAR(500), -- Pass file name here
#ApplicantID VARCHAR(500) --Pass Application ID here
SET #SQLString = '
Insert Into Attachments
(
ApplicantID,
FileName,
FileBinary
)
SELECT #ApplicantID,#Filename,FileBinary
FROM OPENROWSET(BULK N''' + #ImagePath
+ ''',SINGLE_BLOB) AS FileBinary(FileBinary);'
EXEC Sp_executesql
#SQLString,
N'#Filename varchar(500),#ApplicantID varchar(500)',
#Filename =#Filename,
#ApplicantID=#ApplicantID
I am trying to read an XML file using OpenRowSet from a folder and have been unable to do so and get the error
Cannot bulk Load since the "' #FullFilename'' does not exist.
Would appreciate if one could suggest how I can correct the problem
to obtain all data from each of the XML files.
Thanks.
Code:
declare #Directory varchar(50)
select #Directory = 'E:\XML\'
DECLARE #CD TABLE (XMLData XML);
declare #FileExist int
DECLARE #FileName varchar(500),
#DeleteCommand varchar(1000),
#FullFileName varchar(500)
DECLARE #SQL NVARCHAR(1000),#xml xml
--This is so that we know how long the loop lasts
declare #LoopID int, #MaxID int
SELECT #LoopID = min(id),#MaxID = max(ID)
FROM #tempList
WHILE #LoopID <= #MaxID
BEGIN
SELECT #FileNAme = filename
from #tempList
where id = #LoopID
SELECT #FullFileName = #Directory + #FileName
print #FULLFileName
exec xp_fileexist #FullFileName , #FileExist output
if #FileExist =1 --sanity check in case some evil person removed the file
begin
---********************************Problem with #FullFileName----------------
INSERT INTO #CD
SELECT *
FROM OPENROWSET(BULK ''' + #FullFileName +''' ,Single_BLOB) as rs
---********************************------------
select * from #CD
--SET #DeleteCommand = 'del ' + #Directory + #FileName
--maybe you want to delete or move the file to another directory
-- ** here is how to delete the files you just imported
-- uncomment line below to delete the file just inserted
--EXEC MASTER..XP_CMDSHELL #DeleteCommand
-- ** end of here is how to delete the files
end
--Get the next id, instead of +1 we grab the next value in case of skipped id values
SELECT #LoopID = min(id)
FROM #tempList
where id > #LoopID
END
select * from #tempList
This works and I am able to get the XML data from the specified file
DECLARE #CD TABLE (XMLData XML);
Declare #get_GeneralID bigint
INSERT INTO #CD
SELECT *
FROM OPENROWSET(BULK N'E:\XML\TestResult.XML', SINGLE_BLOB) rs;
select * from #CD
PS: I have put together from code that I found from the web.
I don't think you can use T-SQL variables in your OPENROWSET command - those things need to be fully and explicitly spelled out.
If you need to do this over a list of XML files, you'll have to create your T-SQL command as a string and then use dynamic SQL to execute that command.
I try to bulk insert different files.
When i code it like:
SET #xml=( Select * From OPENROWSET(
BULK 'C:\Data\csvToXml.xml',SINGLE_BLOB)x)
it is working.
If i take path as parameter like:
SET #Path= 'C:\Data\csvToXml.xml'
SET #SqlStmt= N' Select #xmlDoc=( Select * From OPENROWSET(
BULK '''+#Path+''' ,SINGLE_BLOB)x)'
exec sp_executesql #SqlStmt, N'#xmlDoc XML',#xmlDoc
#xmlDoc seems empty. I cannot find where I'm wrong.
Thanks for help.
Came here looking for a solution to my own problem but ended up solving this one!
DECLARE #Output int = 0;
DECLARE #params nvarchar(max) = N'#DesiredValue int OUTPUT';
DECLARE #sql_string nvarchar(max) = N'SELECT #DesiredValue = 13';
EXECUTE sp_executesql #sql_string, #params, #DesiredValue = #Output OUTPUT
SELECT #Output -- yields 13
It turns out you need to pass the keyword OUTPUT not only in the #params argument but also on the variables you wish to retrieve. The "#DesiredValue = #Output OUTPUT" looks a little odd because #Output is taking the value of #DesiredValue rather than the other way around.
Documentation.