I have a database table with a column named File Content, and many rows. What I need is to create a text file for each row of File Content column.
Example:
Sr. File Name File Content
1. FN1 Hello
2. FN2 Good Morning
3. FN3 How are you?
4. FN4 Where are you?
Suppose I have 4 rows, then 4 text files should be created (maybe with any name which we want)
File1.txt should have text "hello" in it.
File2.txt should have text "Good Morning" in it.
File3.txt should have text "How are you?" in it.
File4.txt should have text "Where are you?" in it
Although you said you said you need to do it in TSQL, I wouldn't do it that way if possible. Ram has shown you one solution, but it has the disadvantages that you need to use xp_cmdshell and the SQL Server service account needs permission to access the file system in whatever location you want to have the files.
My suggestion would be to write a script or small program in your preferred language (PowerShell, Perl, Python, C#, whatever) and use that instead. TSQL as a language is simply badly suited for manipulating files or handling anything outside the database. It is obviously possible (CLR procedures are another way), but you often run into problems with permissions, encodings and other issues that are much easier to deal with in an external language.
This can be done with BCP OUT syntax of SQL server.
For the setup: just make sure that you have xp_cmdshell exec permissions on the server. This can be checked from master.sys.configurations table. Also change filelocation path corresponding to your server or network share. I checked and was able to generate 4 files as there are 4 records in the table.
use master
go
declare #DSQL Nvarchar(max)
declare #counter int
declare #maxrows int
declare #filename Nvarchar(30)
select #counter=1, #maxrows = 0
create table t1 (
sno int identity(1,1) not null,
filename varchar(5),
filecontent varchar(100)
)
insert into t1
select 'FN1', 'Hello'
UNION
select 'FN2', 'Good Morning'
UNION
select 'FN3', 'How are you?'
UNION
select 'FN14', 'Where are you?'
select #maxrows = count(*) from t1
--SELECT * FROM T1
while (#counter <= #maxrows)
begin
select #filename = filename from t1
where sno = #counter
select #DSQL = N'exec xp_cmdshell' + ' ''bcp "select filecontent from master.dbo.T1 where sno = ' + cast(#counter as nvarchar(10)) + '" queryout "d:\temp\' + #filename + '.txt" -T -c -S home-e93994b54f'''
print #dsql
exec sp_executesql #DSQL
select #counter = #counter + 1
end
drop table t1
Related
I have a view written in SQL Server. I need to export the data to Excel so that there are separate Excel files for every possible two letter country code. Files are named "Report - Country Date.xslx", for example "Report - Australia 06082018.xlsx".
So far I have just connected the view to Excel and repeated following steps for each country:
Change the country code in the SQL view
Save the Excel file with a name matching the country in question
There are some 100 different countries, so the work is quite repetitive and tedious. There are also too many rows in the query results to export several countries' data at the same time. Is there a way to automate this process, even just some part of it?
Okay, normally i would recommend SSIS to do this. Its an excellent tool to handle your question. However if you REALLY want to do it in SQL you can do something like this:
NB's
I could only get it to work by using xls excel files.
It will give you a warning about its not trusted source however you can just click OK
I havent tested it deeply so there can be errors or fixes that needs to be made :)
Prestep: Load sample data
CREATE TABLE [dbo].[TestExcel](
[CountryCode] [varchar](50) NULL,
[Names] [varchar](50) NULL,
[Sales] [int] NULL
)
Insert into dbo.TestExcel (CountryCode,Names,Sales)
Values
('DA','Thomas', 10),
('DA','Jens', 20),
('DA','John', 40),
('EN','Mark', 10),
('EN','Adam', 5 )
Step 1:
Create a stored procedure
Create procedure proc_generate_excel_with_columns
(
#db_name varchar(100),
#table_name varchar(100),
#file_name varchar(100),
#CountryCode varchar(10)
)
as
--Generate column names as a recordset
declare #columns varchar(8000), #sql varchar(8000), #data_file varchar(100)
select
#columns=coalesce(#columns+',','')+column_name+' as '+column_name
from
information_schema.columns
where
table_name=#table_name
select #columns=''''''+replace(replace(#columns,' as ',''''' as '),',',',''''')
PRINT 'Headers'
PRINT #columns
PRINT 'Create a dummy file to have actual data'
select #data_file=substring(#file_name,1,len(#file_name)-charindex('\',reverse(#file_name)))+'\data_file_'+cast(cast(GETDATE() as date) as nvarchar)+'.xls'
print #data_file
print 'Generate column names in the passed EXCEL file'
set #sql='exec master..xp_cmdshell ''bcp " select * from (select '+#columns+') as t" queryout "'+#file_name+'_'+#CountryCode+cast(cast(GETDATE() as date) as nvarchar)+'.xls" -S "EGC25199\SQL2016" -T -c'''
print #sql
exec(#sql)
print 'Generate data in the dummy file'
set #sql='exec master..xp_cmdshell ''bcp "select * from '+#db_name+'..'+#table_name+' where CountryCode = '''''+#CountryCode+'''''" queryout "'+#data_file+'" -S "EGC25199\SQL2016" -T -c'''
print #sql
exec(#sql)
--Copy dummy file to passed EXCEL file
set #sql= 'exec master..xp_cmdshell ''type '+#data_file+' >> "'+#file_name+'_'+#CountryCode+cast(cast(GETDATE() as date) as nvarchar)+'.xls"'''
exec(#sql)
--Delete dummy file
set #sql= 'exec master..xp_cmdshell ''del '+#data_file+''''
exec(#sql)
Step 2: Execute stored procedure
USE [LegOgSpass]
GO
DECLARE #RC int
DECLARE #db_name varchar(100) = 'LegOgSpass'
DECLARE #table_name varchar(100) = 'TestExcel'
DECLARE #file_name varchar(100) = 'D:\Test\TestExcel'
DECLARE #CountryCode varchar(10) = 'EN'
-- TODO: Set parameter values here.
EXECUTE #RC = [dbo].[proc_generate_excel_with_columns]
#db_name
,#table_name
,#file_name
,#CountryCode
Result - Picture of Folder
Result of DA
Result of EN
Python can also help in automating SQL data ->Excel. This sample code creates 4 excel sheets using pyexcelerate.
#•pyexcelerate
import pyexcelerate as px
from datetime import date
wb = px.Workbook()
ws = wb.new_sheet("Report")
n = date.today()
ReportDate = "{:02.0f}{:02.0f}{}".format(n.day,n.month,n.year)
ws.range("B2", "C3").value = [[1, 2], [3, 4]]
Countrylist = ['USA', 'Australia','China','Nigeria']
#"Report - Australia 06082018.xlsx".
for c in countrylist:
wb.save("Report -"+c+' '+str(ReportDate)+".xlsx")
we have a very old ERP system which is badly supported.
Now the warehouse want´s to buy a new "store system" for our goods. It´s a fully automatic store system which need´s data from our ERP system. The support of your ERP system can´t help us, so we have to build a solution of our own.
The idea was to "move" the items for the new storage system to a special storage place called (SHUT1) and output the "part number" and "quantity" to a file (xml) which can be read by the new software.
We can´t change anything in the software of our ERP system, so we have to do it on the SQL Server itself.
(I know, a trigger is not the "best" thing to do, but I have have no other choice)
CREATE TRIGGER tr_LagerShut ON dbo.Lagerverwaltung
AFTER INSERT
AS
BEGIN
IF (SELECT [Lagerort] from Inserted) = 'SHUT1'
BEGIN
DECLARE #Cmd VARCHAR(2000) ;
DECLARE #FormatDate4File VARCHAR(200);
SET #FormatDate4File = (SELECT(SYSUTCDATETIME()));
SET #FormatDate4File = (SELECT REPLACE(#FormatDate4File,' ','-'));
SET #FormatDate4File = (SELECT REPLACE(#FormatDate4File,':','-'));
SET #FormatDate4File = (SELECT REPLACE(#FormatDate4File,'.','-'));
SET #Cmd = ( SELECT [Artikelnummer],[Menge] FROM inserted FOR XML PATH('')) ;
SET #Cmd = 'Echo "' + #Cmd + '" >>"C:\Temp\' + #FormatDate4File +'.xml"' ;
EXEC xp_cmdshell #Cmd ;
END;
END;
The trigger "installs" fine, but if I change a storage place to a new one, the ERP system stalls with "ERROR" (there is no error description :(
If I drop the trigger the system is just running fine again. So I think there is a error in the trigger, but I can´t find it.
Can anybody help please?
Aleks.
Don't know what ERP system stalls with "ERROR" looks like... Frozend GUI? Timeout? Just no file created?
My magic glass bulb tells me the following: You are inserting more than one row at once. If so, this statement will break, because a comparison like this is only valid against a scalar value. If there is more than one row in inserted, this will not work:
IF (SELECT [Lagerort] from Inserted) = 'SHUT1'.
Your trigger can be simplified, but I doubt, that you will like the result. Check this with special characters (like üöä) and check for enclosing "-characters...
CREATE TRIGGER tr_LagerShut ON dbo.Lagerverwaltung
AFTER INSERT
AS
BEGIN
IF EXISTS(SELECT 1 FROM inserted WHERE [Lagerort]='SHUT1')
BEGIN
DECLARE #FileName VARCHAR(255) =REPLACE(REPLACE(REPLACE(SYSUTCDATETIME(),' ','-'),':','-'),'.','-');
DECLARE #Content XML=
(
SELECT [Artikelnummer],[Menge] FROM inserted WHERE [Lagerort]='SHUT1' FOR XML AUTO,ELEMENTS
);
DECLARE #Cmd VARCHAR(4000) = 'Echo "' + CAST(#Content AS VARCHAR(MAX)) + '" >>"c:\temp\' + #FileName + '.xml"' ;
PRINT #cmd;
EXEC xp_cmdshell #Cmd ;
END
END;
This might better be done with BCP.
UPDATE Your comment...
First you should check, if this works at all:
DECLARE #FileName VARCHAR(255) =REPLACE(REPLACE(REPLACE(SYSUTCDATETIME(),' ','-'),':','-'),'.','-');
DECLARE #Content XML=
(
SELECT TOP 5 * FROM sys.objects FOR XML AUTO,ELEMENTS
);
DECLARE #Cmd VARCHAR(4000) = 'Echo "' + CAST(#Content AS VARCHAR(MAX)) + '" >>"c:\temp\' + #FileName + '.xml"' ;
PRINT #cmd;
EXEC xp_cmdshell #Cmd ;
If you find no file in c:\temp\: Are you aware, that SQL-Server will always write in its own context? Might be, that you are awaiting a file in your local c-drive, but the file is written to the Server's machine acutally.
If this works isolatedly, it should work within a trigger too. You might pack the call into BEGIN TRY ... END TRY and add an appropriate CATCH block.
So okay, the "simple" trigger could be really a problem. Now I have this idea:
(one more info: time stamp is not inserted into table "Lagerverwaltung" when a new row is inserted)
Pseudo Code on:
Trigger on Table "Lagerverwaltung"
Check if the storage place(s) is "SHUT1"
If "yes"
DECLARE #FileName VARCHAR(255) =REPLACE(REPLACE(REPLACE(SYSUTCDATETIME(),' ','-'),':','-'),'.','-');
create a new table with name dbo + '.' + #filename
Insert all the data (inserted) + row(SYSUTCDATETIME()) AS TimeStamp where 'Lagerort' = 'SHUT1' into new table named 'dbo.' + #filename
DECLARE #Cmd varchar(4000) = 'bcp "select [Artikelnummer],[Menge],[TimeStamp] FROM [wwsbautest].[dbo].#filename WHERE [Lagerort]=''SHUT1'' AND [Menge] > ''0'' FOR XML AUTO, ELEMENTS" queryout "C:\temp\' + #FileName + '.xml" -c -T';
EXEC xp_cmdshell #Cmd;
Drop table dbo.#filename;
Could somthing like that work?
I've being searching for this on google but I haven't found any good explanation, so here's my issue.
I need to import product images, which are in a folder to SQL Server, I tried to use xp_cmdshell but without success.
My images are in C:\users\user.name\Images and the images have their names as the product id, just like [product_id].jpg and they're going to be inserted in a table with the product ID and the image binary as columns.
I just need to list the images on the folder, convert the images to binary and insert them in the table with the file name (as the product_id)
My questions are:
How do I list the images on the folder?
How do I access the folder with dots in their name (like user.name)
How do I convert the images to binary in order to store them in the database (if SQL Server doesn't do that automatically)
Thanks in advance
I figured I'd try an xp_cmdshell-based approach just for kicks. I came up with something that does appear to work for me, so I'd be curious to know what problems you ran into when you tried using xp_cmdshell. See the comments for an explanation of what's going on here.
-- I'm going to assume you already have a destination table like this one set up.
create table Images (fname nvarchar(max), data varbinary(max));
go
-- Set the directory whose images you want to load. The rest of this code assumes that #directory
-- has a terminating backslash.
declare #directory nvarchar(max) = N'D:\Images\';
-- Query the names of all .JPG files in the given directory. The dir command's /b switch omits all
-- data from the output save for the filenames. Note that directories can contain single-quotes, so
-- we need the REPLACE to avoid terminating the string literal too early.
declare #filenames table (fname varchar(max));
declare #shellCommand nvarchar(max) = N'exec xp_cmdshell ''dir ' + replace(#directory, '''', '''''') + '*.jpg /b''';
insert #filenames exec(#shellCommand);
-- Construct and execute a batch of SQL statements to load the filenames and the contents of the
-- corresponding files into the Images table. I found when I called dir /b via xp_cmdshell above, I
-- always got a null back in the final row, which is why I check for fname IS NOT NULL here.
declare #sql nvarchar(max) = '';
with EscapedNameCTE as (select fname = replace(#directory + fname, '''', '''''') from #filenames where fname is not null)
select
#sql = #sql + N'insert Images (fname, data) values (''' + E.fname + ''', (select X.* from openrowset(bulk ''' + E.fname + N''', single_blob) X)); '
from
EscapedNameCTE E;
exec(#sql);
I started with an empty Images table. Here's what I had after running the above:
Now I'm not claiming this is necessarily the best way to go about doing this; the link provided by #nscheaffer might be more appropriate, and I'll be reading it myself since I'm not familiar with SSIS. But perhaps this will help illustrate the kind of approach you were initially trying for.
I have a table containing base64-encoded jpegs as well as some other data. The base64 string is stored in a VARCHAR(MAX) column.
How can I save these images out to actual files inside folders that are dynamically named using other data from the table in a stored procedure?
I found the answer by combining a lot of small tips from different places and wanted to collate them all here as no one seemed to have the full process.
I have two tables, called Photos and PhotoBinary. Photos contains at least a PhotoID BIGINT, the Base64 data VARCHAR(MAX), and something to get the FolderName from - NVARCHAR(15) - increase as needed. I also have a BIT field to mark them as isProcessed. PhotoBinary has a single VARBINARY(MAX) column and should be empty.
The second table serves two purposes, it holds the converted base64 encoded image in binary format, and allows me to work around the fact that BCP will not let you skip columns when exporting data from a table when using a "format file" to specify the column formats. So in my case the data has to sit in a table all on its own, I did try taking it from a view but had the aforementioned issue with not being allowed to skip the id column.
The main stored procedure has a bcp command that depends upon an .fmt file created using the following SQL. I'm pretty sure I had to edit the resulting file with a plain text editor to change the 8 to a 0 that indicates the prefix length after SQLBINARY. I couldn't just use the -n switch on the command in the main stored procedure as it resulted in an 8 byte prefix being put into the resulting file, which made it an invalid jpeg. So, I used the edited format file to get around that.
DECLARE #command VARCHAR(4000);
SET #command = 'bcp DB.dbo.PhotoBinary format nul -T -n -f "A:\pathto\photobinary.fmt"';
EXEC xp_cmdshell #command;
I then have the following in a stored procedure that I run to export the images into an appropriate folder:
DECLARE #command VARCHAR(4000),
#photoId BIGINT,
#imageFileName VARCHAR(128),
#folderName NVARCHAR(15),
#basePath NVARCHAR(500),
#fullPath NVARCHAR(500),
#dbServerName NVARCHAR(100);
DECLARE #directories TABLE (directory nvarchar(255), depth INT);
-- The location of the output folder
SET #basePath = '\\server\share';
-- The server that the photobinary db is on
SET #dbServerName = 'localhost';
-- #basePath values, get the folders already in the output folder
INSERT INTO #directories(directory, depth) EXEC master.sys.xp_dirtree #basePath;
-- Cursor for each image in table that hasn't already been exported
DECLARE photo_cursor CURSOR FOR
SELECT PhotoID,
'some_image_' + CAST(PhotoID AS NVARCHAR) + '.jpg',
FolderName
FROM dbo.Photos
WHERE isProcessed = 0;
OPEN photo_cursor
FETCH NEXT FROM photo_cursor
INTO #photoId,
#imageFileName,
#folderName;
WHILE (##FETCH_STATUS = 0) -- Cursor loop
BEGIN
-- Create the #basePath directory
IF NOT EXISTS (SELECT * FROM #directories WHERE directory = #folderName)
BEGIN
SET #fullPath = #basePath + '\' + #folderName;
EXEC master.dbo.xp_create_subdir #fullPath;
END
-- move and convert the base64 encoded image to a separate table in binary format
-- it should be the only row in the table
INSERT INTO DB.dbo.PhotoBinary (PhotoBinary)
SELECT CAST(N'' AS xml).value('xs:base64Binary(sql:column("Base64"))', 'varbinary(max)')
FROM DB.dbo.Photos
WHERE PhotoID = #photoId;
-- This command uses the command-line BCP tool to "bulk export" the image data in binary to an "archive" file that just happens to be a jpg
SET #command = 'bcp "SELECT TOP 1 PhotoBinary FROM DB.dbo.PhotoBinary" queryout "' + #basePath + '\' + #folderName + '\' + #imageFileName + '" -T -S ' + #dbServerName + ' -f "A:\pathto\photobinary.fmt"';
EXEC xp_cmdshell #command;
-- clean up the photo data
DELETE FROM DB.dbo.PhotoBinary;
-- mark photo as processed
UPDATE DB.dbo.Photos SET isProcessed = 1 WHERE PhotoID = #photoId;
FETCH NEXT FROM photo_cursor
INTO #photoId,
#imageFileName,
#folderName;
END -- cursor loop
CLOSE photo_cursor
DEALLOCATE photo_cursor
I have a table which has an IDENTITY field and a VARBINARY(MAX) field in it. Is there any way I could dump the content of each VARBINARY(MAX) field to the filesystem (eg. c:\temp\images)?
Table schema:
PhotoId INTEGER NOT NULL IDENTITY
Image VARBINARY(MAX) NOT NULL
DateCreated SMALLDATETIME
Output:
c:\temp\images\1.png
c:\temp\images\2.png
c:\temp\images\3.png
... etc...
What is the best way to approach this?
I've figured it out thanks to this post ...
SET NOCOUNT ON
DECLARE #IdThumbnail INTEGER,
#MimeType VARCHAR(100),
#FileName VARCHAR(200),
#Sqlstmt varchar(4000)
DECLARE Cursor_Image CURSOR FOR
SELECT a.IdThumbnail
FROM tblThumbnail a
ORDER BY a.IdThumbnail
OPEN Cursor_Image
FETCH NEXT FROM Cursor_Image INTO #IdThumbnail
WHILE ##FETCH_STATUS = 0
BEGIN
-- Generate the file name based upon the ID and the MIMETYPE.
SELECT #FileName = LTRIM(STR(#IdThumbnail)) + '.png'
-- Framing DynamicSQL for XP_CMDshell
SET #Sqlstmt='BCP "SELECT OriginalImage
FROM Appian.dbo.tblThumbnail
WHERE IdThumbnail = ' + LTRIM(STR(#IdThumbnail)) +
'" QUERYOUT c:\Temp\Images\' + LTRIM(#FileName) +
' -T -fC:\Temp\ImageFormatFile.txt'
print #FileName
print #sqlstmt
EXEC xp_cmdshell #sqlstmt
FETCH NEXT FROM Cursor_Image INTO #IdThumbnail
END
CLOSE Cursor_Image
DEALLOCATE Cursor_Image
Please note -> u need to have a format file, for the BCP command. This is the content of the file and i've placed it in c:\Temp (as noted in the BCP commandline above).
10.0
1
1 SQLIMAGE 0 0 "" 1 OriginalImage ""
Final note about that format file .. there HAS TO BE A NEW LINE AFTER THE LAST LINE. otherwise u'll get an error.
Enjoy!
The easiest (for me) would be to write a little .NET application that dumped the varbinary fields to a file.
If you are opposed to working in .NET, then you should (I've not tested this) be able to create a new column of type "VARBINARY(MAX) FILESTREAM". Be sure you also have a the corresponding "ROWGUIDCOL" column. Then you can (in theory) do something like:
UPDATE table
SET varfilestream_col = varbinary_col
Hope that works for ya.