I need to copy certain file from a folder A to a folder B, but I need to copy only file based on a condition. The name of the file must start with the same value as a variable I have in my stored procedure.
Here is what I've got at the moment:
DECLARE #SQLFile VARCHAR(1024)
DECLARE #MessageId INT = 3 --copy all the files from the source folder that start with this variable
DECLARE #SourceFolderPath VARCHAR(1024)
DECLARE #DestinationFolderPath VARCHAR(1024)
SET #DestinationFolderPath = '\\mydestination'
SET #SourceFolderPath = '\\mysource'
SET #SQLFile = ' COPY /Y ' + #SourceFolderPath + ' /B ' + #DestinationFolderPath
EXEC master..xp_cmdshell #SQLFile
With this code I copy all the files but I don't know if there is a way to integrate the condition of beginning of name.
Thanks
I'm guessing a simple file glob will do what you want. But as people said in the comments using xp_cmdshell can create all kinds of security issues if used where untrusted data is kicking about. For this reason it is also often blocked by SQL Server security policy.
DECLARE #SQLFile VARCHAR(1024)
DECLARE #MessageId INT = 3 --copy all the files from the source folder that start with this variable
DECLARE #SourceFolderPath VARCHAR(1024)
DECLARE #DestinationFolderPath VARCHAR(1024)
SET #DestinationFolderPath = '\\mydestination'
SET #SourceFolderPath = '\\mysource'
SET #SQLFile = ' COPY /Y ' + #SourceFolderPath + '\' + CAST(#MessageID AS varchar(max)) + '* /B ' + #DestinationFolderPath
EXEC master..xp_cmdshell #SQLFile
Related
I have a folder called "Data" This folder consists of various .CSV Files with same schema. The folder Location is 'C:\Data..'
I want to Import the contents of all files recursively into SQL Server in one table.
Append additional column in the table which consists for each file.
Have tried some solutions found it here
, but it didn't work.
--get the list of files to process:
SET #path = 'C:\Data\'
SET #cmd = 'dir ' + #path + '*.csv /b'
INSERT INTO ALLFILENAMES(WHICHFILE)
EXEC Master..xp_cmdShell #cmd
UPDATE ALLFILENAMES SET WHICHPATH = #path where WHICHPATH is null
--cursor loop
declare c1 cursor for SELECT WHICHPATH,WHICHFILE FROM ALLFILENAMES where WHICHFILE like '%.csv%'
open c1
fetch next from c1 into #path,#filename
While ##fetch_status <> -1
begin
--bulk insert won't take a variable name, so make a sql and execute it instead:
set #sql = 'BULK INSERT Temp FROM ''' + #path + #filename + ''' '
+ ' WITH (
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
FIRSTROW = 2
) '
print #sql
exec (#sql)
fetch next from c1 into #path,#filename
end
close c1
deallocate c1
TWo things which aren't working as expected:
1) It doesn't work for recursive files in the directory.
2) Not able to do the bulk insert to Temp file created.
I have this SQL command which will be used in a stored procedure and will be scheduled:
BULK INSERT Test101.dbo.Test102
FROM 'C:\Bulk\samp.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
It works well but what I want to do is to process all the .csv files from a folder (let's say Bulk folder) automatically. I mean, the user doesn't have to define the exact location with filename instead the stored procedure will process all the .CSV file from that folder and ignore the other files if there were.
Please help.
Thank you.
try these to make sure you have the right permissions and settings to do xp_cmdshell
One Time config change to enable here : https://stackoverflow.com/a/5131503/2628302
--To Test if that worked run these commands. There should be no errors and should return list of files under c:\
declare #files table (ID int IDENTITY, FileName varchar(500))
insert into #files execute xp_cmdshell 'dir c:\ /b'
select * from #files
--if no errors then proceed to create this SP as shown below. This is the one that does all the work
CREATE PROCEDURE dbo.sp_BulkInsAllFilesInDirectory
AS
BEGIN
--a table to hold filenames
Declare #ALLFILENAMES as TABLE (WHICHPATH VARCHAR(255),WHICHFILE varchar(255))
--some variables
declare #filename varchar(255),
#path varchar(255),
#sql varchar(8000),
#cmd varchar(1000)
--get the list of files to process:
SET #path = 'C:\Bulk\'
SET #cmd = 'dir ' + #path + '*.csv /b'
INSERT INTO #ALLFILENAMES(WHICHFILE)
EXEC Master..xp_cmdShell #cmd
UPDATE #ALLFILENAMES SET WHICHPATH = #path where WHICHPATH is null
--cursor loop
declare c1 cursor for SELECT WHICHPATH,WHICHFILE FROM #ALLFILENAMES where WHICHFILE like '%.csv%'
open c1
fetch next from c1 into #path,#filename
While ##fetch_status <> -1
begin
--bulk insert won't take a variable name, so make a sql and execute it instead:
set #sql = 'BULK INSERT Temp FROM ''' + #path + #filename + ''' '
+ ' WITH (
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
FIRSTROW = 2
) '
print #sql
exec (#sql)
fetch next from c1 into #path,#filename
end
close c1
deallocate c1
END
--- TEST it by running it like so (start with just one csv file in C:\BULK\ directory. If it works for one it will most likely work for more than one file.
EXEC dbo.sp_BulkInsAllFilesInDirectory
see if there are errors. Leave a message here and I will check tomorrow. Good luck.
I have a query to move files from one folder(Source) to another(Destination)
I have given permission on both source and destination to network services and local services, I get an error
The system cannot find the file specified
I have tested the folders to see if I am able to to access the files and I was able to, it seems when I get the files it works. It is when I try to move the files to the Destination folder, I get this error.
My query below is as follow, I create a temp table to link to my source folder and insert its contents into it, I then filter it out into another temp table, where I then get the files I want to move and then use a cursor to iterate through it and move the files.
I hope this can assist.
Create Table #tmpDir
(
ID int IDENTITY(1,1),
fName varchar(400)
)
Declare #dir varchar(100)
Declare #folderPath varchar(500)
set #folderPath = '\\Server1\Documents\Ocs\Inbox\'
set #dir = 'DIR ' + '"' + #folderPath + '"'
print #dir
INSERT #tmpDir EXECUTE master.dbo.xp_cmdshell #dir
DELETE FROM #tmpDir where ID < 6
SELECT SUBSTRING(FName,40,100) fileName2
into #THIS
FROM #tmpDir
WHERE FName not like '%<DIR>%'
and FName not like '%bytes%' and FName is not null
Delete #THIS Where fileName2 Not Like '%_MyFiles%'
Declare #FileMove varchar(100)
Declare cur cursor for
Select fileName2 From #THIS
Open cur
Fetch next from cur into #FileMove
WHILE ##FETCH_STATUS = 0
BEGIN
Declare #cmd varchar(1000)
set #cmd = 'Move /y "\\Server1\Documents\Ocs\Inbox\' + #FileMove + ' "\\Server1\ImportMac\Inbox"'
EXECUTE master.dbo.xp_cmdshell #cmd
Fetch next from cur into #FileMove
END
Close cur
Deallocate cur
Drop Table #tmpDir, #THIS
You need to put a double quote at the end of the source file name for your move command:
set #cmd = 'Move /y "\\Server1\Documents\Ocs\Inbox\' + #FileMove + '" "\\Server1\ImportMac\Inbox"'
I'm a SQL Newbie and I'm trying to figure out how to use a Parameter with SQL Thru Command-Line
For my boss/staff I have written Batch Files to run SQL Code for them to export data and the like. In Access all I have to do is [Paramerter] and it prompts for data to be entered.
The #State Variable I'd like to be able to be set dynamically. I'd like the batch file to ask for State and Query use that information. I have no idea how to do it.
Batch File
sqlcmd -E -S ServerName -i C:\Lead$\SQL\MakeSTPhoeLists.sql
pause
The SQL File
Use LeadsDb
Go
Declare #State VarChar(2)
Set #State = 'DE'
DELETE FROM tblzExportPhone
INSERT INTO tblzExportPhone ( Phone )
SELECT tblLeads.Phone
FROM tblLeads
WHERE tblLeads.ST = #State
Declare #FileName VarChar(100)
Set #FileName = 'H:\Leads\Storage\STLists\' + #State +'StatePhoneList.csv'
DECLARE #bcp_cmd4 VARCHAR(400) = ' BCP.EXE LeadsDb..tblzExportPhone out '
SET #bcp_cmd4 = #bcp_cmd4 + 'H:\Leads\SQL\Formats\PhoneTmp.csv' + ' -T -f H:\Leads\SQL\Formats\tblzExportPhone.fmt'
SET #bcp_cmd4 = #bcp_cmd4 + ' & Copy /b H:\Leads\SQL\Formats\ExPhone.csv+H:\Leads\SQL\Formats\PhoneTmp.csv ' + #FileName + ' /y'
Set #bcp_cmd4 = #bcp_cmd4 + ' & Del H:\Leads\SQL\Formats\PhoneTmp.csv'
Thank You.
In your sql file use this notation $(statename)
Add this to your command file -v statename = %1
And execute it passing the parameter mycommanfile.cmd DE
Also read this for a full example.
Essentially what I found out is you have to make a Stored Procedure and Call a query to run it. I'm going to leave an example. BTW SQL Query Creates CSV File from table Using bcp and then Zips it up using RAR Dos Command.
BTW Stored procedure is used due to every which other way I try to do it withoutI failed, but once I added it I was golden. Can use Dos prompt insead of bat file but I'm setting up bat files for 'Semi-Computer Smart people'
Hope this helps a newbie like me out^^
BAT File (cmd prompt)
Echo phone numbers in the State to send to Paramount.
Echo Then after the file is made converts it to Zip.
Echo '
Set /p State=Enter Initials of the State? :
sqlcmd -E -S Titania -i H:\Lead$\SQL\STPhones.sql -v StateName=%State%
SQL Command #1
Use dbNameHere
go
Declare #State NVarChar(2)
EXECUTE [dbo].[STPhoneListB] #State=$(StateName)
SQLCommand #2
Use dbNameHere
Go
CREATE PROCEDURE [dbo].[STPhoneListB]
#State NVarChar(2) = 'DE',
#Folder VarChar(100) = 'H:\Lead$'
AS
BEGIN
Declare #FileName VarChar(150)
Set #FileName = #Folder + '\STPhones_'+ #State +'.csv'
Declare #DosCMD VarChar(150) = 'Del ' + #Folder +'\STPhones'+ #State
+'.Zip /q'
EXEC master..xp_cmdshell #DosCMD
DECLARE #bcp_cmd4 VARCHAR(400)
DELETE FROM tblzExportPhone
INSERT INTO tblzExportPhone ( Phone )
SELECT tblLeads.Phone
FROM tblLeads
WHERE tblLeads.ST = #State
Set #bcp_cmd4 = 'BCP.EXE LeadsDb..tblzExportPhone out '
SET #bcp_cmd4 = #bcp_cmd4 + 'H:\SQL\PhoneTmp.csv' +
' -T -f H:\SQL\tblzExportPhone.fmt'
SET #bcp_cmd4 = #bcp_cmd4
+ ' & Copy /b H:\SQL\ExPhone.csv+H:\SQL\PhoneTmp.csv '
+ #FileName + ' /y'
Set #bcp_cmd4 = #bcp_cmd4 + ' & Del H:\SQL\PhoneTmp.csv'
EXEC master..xp_cmdshell #bcp_cmd4
Set #bcp_cmd4 = 'cd '+ #Folder +
' & "C:\Program Files\WinRAR\rar.exe" m STPhones_'+ #State +'.Zip ' +
'STPhones_'+ #State +'.csv'
EXEC master..xp_cmdshell #bcp_cmd4
DELETE FROM tblzExportPhone
END
I am trying to run the following commands to generate a file and populate the file with varchar(MAX) amount of data. Here is the issue, when I try to run the code it creates the files but they are empty. I know the #classString variable has data in it. Is there a string enclosing sequence I can wrap around the #classString to delimit it? I am trying to generate a script so I am assuming the #classString contains characters that are interfering with the file write.
SET #sqlString = 'echo off > ' + #FilePath + '\' + #table_name + '.cs"'
exec xp_cmdshell #sqlString
SET #sqlString = 'echo ' + #classString + ' >> ' + #FilePath + '\' + #table_name + '.cs"'
exec xp_cmdshell #sqlString
There is no easy way to delimit or quote the whole string, instead you need to escape any shell metacharacters within it using the ^ symbol. For example:
SET #classString = (SELECT REPLACE(#classString, '>', '^>'))
SET #classString = (SELECT REPLACE(#classString, '&', '^&'))
...and so on for any cmdshell metacharacters that may feature in your string.