I created a ssis package to move backup files from c: location to E location:
source file path: C:\Backup\A\xxx.bak
destination file path: E:\DB Backups\A\xxx.bak
I used foreachloop container and created two variables: sourcefileV and destinationV and used file system task to rename file. The package works however after moved the file to destination, the file is not in subfolder E:\DB Backups\A, it's in E:\DB Backups. I have to manually cut and move it to the subfolder. Is there any way to directly move file to the destination subfolder in ssis package?
Thanks,
You can create two variables #[User::DestinationFolder] and #[User::SourceFolder].
#[User::SourceFolder] = C:\Backup\
#[User::DestinationFolder] = E:\DB Backups\
And you can use the following expression to create the destination folder path.
#[User::DestinationFolder] + SUBSTRING(REPLACE(#[User::SourceFileV],#[User::SourceFolder],""),1, LEN(REPLACE(#[User::SourceFileV],#[User::SourceFolder],"")) - FINDSTRING(REVERSE(#[User::SourceFileV]),"\\",1))
And you can use the following expression to get the destination file path
#[User::DestinationFolder] + REPLACE(#[User::SourceFileV],#[User::SourceFolder],"")
First, you have to add an Execute File System Task to create the destination directory. Then you have to add a second Execute File System Task to copy the file.
Related
I have a dataflow that is used to do transformation of multiple flat files from given folder using for each loop container. I have a flat file again as output file. The problem is that every time I execute the the job only the last file that got transformed will be stored in destination file.
Is there a way in SSIS I can create individual transformed output file instead on overwriting on same one over and over again?
For. eg. I have 5 flat files ,test_1.txt,test_2.txt,test_3.txt ,test4_.txt
and test_5.txt in a folder.
After the job ran I can only see the data from last file test_5.txt being
transformed in my destination file.
Here's steps on a working example I tested.
Variables
I have 3 variables defined:
FileName - To be used in the foreach loop
DestinationDir - where are the files going
SourceDir - where are the files I want to process
Foreach Loop Setup
I have a foreach loop configured as:
Expression for "Directory" set to #[User::SourceDir]
Retrieve file name set to "Name and extension"
Then under the "Variable Mappings":
That means as the foreach loop is iterating over the files in the directory it will be setting the "Name and extension" of the file its on to the variable #[User:FileName]
Data Flow Task
The I add a Data Flow Task inside the foreach loop:
Then inside the DFT I have a simple Flat File Source to Flat File Destination. We'll just pass the contents of each file to new files:
During initial development I'll manually pick one file to walk through setting each of the source and destinations. Then come back and change the connection managers and set an expression on the ConnectionString.
Connection Manager Expressions
SourceFile Connection Manager:
ConnectionString gets an expression as: #[User::SourceDir] + #[User::FileName]
DestinationFile Connection Manager:
ConnectionString gets an expression as: #[User::DestinationDir] + #[User::FileName]
Testing
I have 2 test files in my source directory and no files in my destination:
After I execute my package I get success and also get new files in my destination:
There are ways to do what you are asking in SSIS with variables and expressions but there is an easier way to accomplish it using command line.
Since you are just consolidating a text files into 1 you can use a command prompt to better handle your issue:
copy *.txt output.txt
I have an ssis package that loops through files and import the files to a SQL DB. In the loop the file is picked up regardless if it is a duplicate file. In my data flow i create a key that ignores the duplicates and imports new records.
After importing the file and ignoring duplicate i have a script task that check if the file exists in a destination folder.
string filepath;
filepath = Dts.Variables["User::FILE_PATH_VAR"].Value.ToString();
Dts.Variables["User::FILE_EXISTS"].Value = File.Exists(filepath);
Dts.TaskResult = (int) ScriptResults.Success
Up to this point the package is working. What i want to do is, if the file exists in the destination folder move the file to a duplicate destination folder. If the file does not exist move it to the destination folder.
I have tried two file system tasks with constraint editor to the one on File exist = true and the other = False. But it only diverts to the one and moves both files to the duplicate folder.
I am trying to archive a file. included in the image is the settings of my system file task. I'm trying to send a file whose name is stored in a variable within the loop, and send that file into the requisite folder.
The file is supposed to land in F:\DATA\ARCHIVE\WELLBORE\ using the filename it was given. However, it keeps landing in F:\DATA\ARCHIVE with the NAME WELLBORE, which is not what I am looking for.
What do I have missing?
Thanks
Problem
The issue is when you select Move File so you have to select the source full file path and the destination full file path, so the destination variable should contains the destination full path (with file name) and if the variable contains a folder path, the folder name will be considered as a filename and it will throw an exception.
Solution
Create a new variable #[User::DestinationFile] and assign the following expression to it:
#[User::ArchiveFolder] + RIGHT( #[User::WellBoreFile] , FINDSTRING(REVERSE( #[User::WellBoreFile] ) , "\\", 1) - 1)
This expression will add the filename to the destination path. And use this new variable as a Destination
References
SSIS EXPRESSION TO GET FILE NAME FROM FULL PATH
SSIS Expression to get filename from FilePath
Microsoft Docs article*
I am working on a SSIS project that download a file via web and upload the data in sql server then after that move the downloaded file to another folder. Here is an image for a clearer picture
I have a variable:
Source_Folder with a value:C:\Users\T-Aordiz\Documents\DumpDatas\Outbound Dump\Dump
Success_Folder value: C:\Users\T-Aordiz\Documents\DumpDatas\Outbound Dump\Success
For Each Loop values
Expressions:#[User::Source_Folder] , Variable Mappings: User::FileName(which is blank string)
and here is my file system task
However I encounter this error message
Error: 0xC002F304 at Success, File System Task: An error occurred with the following error message: "Could not find file 'C:\Users\T-Aordiz\Documents\DumpDatas\Outbound Dump\Dump'.".
Task failed: Success
I tried changing the expression in File System Task but also encountered an error. Help me guys
Source connection,
Option 1,
When using Foreach File Enumerator - 'Retrieve file name', if you choose Fully Qualified, it will give you a complete path, it means #[User::Filename] includes folder and file name with extension. you can use this variable as source connection variable.
Option 2,
When using Foreach File Enumerator - 'Retrieve file name', if you choose 'Name and Extension', it means you must create another variable FilenameFullpath, expression specified as #[User::Source_Folder] + "\" + #[User::Filename]. (you can always include "\" at the end of your declared folder.)
Destination connection,
You can just specify a folder, no necessary to include file name and extension.
In the file system task you must select variables containnig source filename and destination filename not folders.
Choose #[User::Filename] as source and choose the destination as another variable that contains the destination file path (not folder, must include file with extension)
You can generate destination value using script task or expression task. By concatenating the source filename (without path) + destination folder
Note that #[User::Filename] should be mapped in the Foreach Loop container
I'm trying to create batch file to copy mdb file to another folder then renaming it and run a task scheduler everyday. so everytime the batch runs, the same mdb file will copy and rename. I don't care about the additional characters just copy and rename will do.
folder A (source folder)
test.mdb 06/21/2014
folder B (target folder)
test02.mdb 06/21/2014
test03.mdb 06/22/2014
thanks
copy "c:\folder a\test.mdb" "b:\folder b\test%random%.mdb"
The random number is 0 to 32,000. To do it by dates
Set NewDate=%date:/=%
copy "c:\folder a\test.mdb" "b:\folder b\test%newdate%.mdb"
This will copy and rename a file:
copy "folder A\test.mdb" "folder B\test02.mdb"