My process ceates automatically some new folders named with a date i.e (09-01-2011) in a directory (Archive)
How can I check the date of the folder in the directory and delete those older than a certain date?
Thanks
Use for_each_folder_enumeration technology from here:
http://microsoft-ssis.blogspot.com/2011/01/foreach-folder-enumerator.html
Analyze folder name using Script component
Check if folder is empty using Microsoft suggestion: http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/a3240e00-ac9d-4082-b1e4-72bd7f695524/
Move folder using File System Task (Copy directory)
Related
My company recently migrated from one payroll system to another. I've used excel to dump all the files to a list with their path and mapped the old employees ID's to the new system. What I have now is a spreadsheet with old file name, current file path, and new file name. Is there a way for a .bat script to use this spreadsheet to copy and rename each file?
I have a folder on a webb server that the user can upload files to, I need to have a script that monitor that folder and whenever a new file is uploaded or changed I would like to copy that file to another folder on my server. This script should be monitoring the source folder all the time. Is this possible in a Windows Server 2008R2?
I have an SSIS package that picks up a file from a directory and imports it into a database, pretty straightforward stuff. The only issue is the input file is named based on the current date, e.g \path\to\file\filename_010115.txt, where 010115 is for January 1, 2015.
I was calling this package with a custom bat file that set the connection manager for a Flat File Source to the current date formatted filename and passed it into dtexec.exe directly; however, our environment demands that we use XML configuration files and not bat files. Is there a simple way to set the file source "prefix" in the xml and append the date-formatted filename before attempting to pick up the file?
Have you considered rearchitecting your approach? You're not going to be able to gracefully do this with a Configuration - be it XML, table, environment, or registry key. In the simplest case, a table, before you could even start your SSIS package, a process would need to update that table to use the current date. A scheduling tool like SQL Agent can run SQL Commands. If you're going the XML route though, you're looking at a tiny little app or PowerShell command to modify your configuration file. Again, that's going to have to modify your file every day and before the SSIS package begins as it sets values once in the beginning and never consults the configuration source again.
You could use an Expression in SSIS to use the current date as part of your flat file source's connection string but in the event you need to process more than one day's file or reprocess yesterday's file, you're humped and either have to manually rename source files or change the system clock and nobody's going to do that.
A more canonical approach would be to use a Foreach (file) Loop Container. Point it at the source folder and find all the file(s) that match your pattern. I'm assuming here you move processed files out of the same folder. The Foreach Container finds all matching files and enumerates through the list popping the current one into whatever variable you've chosen.
See a working example on
https://stackoverflow.com/a/19957728/181965
In my actual project, my boss asked me to export data from database to csv flat files.
So, as I don't know .net code, I use SSIS File System task and ForEach Loop Container.
I built a package which:
Select data filtered by year, month and region
Create differents folder to archive files exported like this:
A- Year\MONTH\REGION\ .csvfiles
One difficulty that I have is to create folder and sub folder only if it does not exist.
I'm lazy, I'd drop a Script Task in the Foreach Loop Container and make use of the System.IO.Directory. Specifically, I'd use Exists and CreateDirectory
I have some csv files. I want to write SQL Server script to read the file at certain period and insert in SQL Server db, if record is not found and ignore it if file has already been read previously by scheduler. Each csv will contain one record only.
Like:
1.csv => John,2000,2012/12/12
2.csv => Tom,3000,2012/12/11
It will be great if someone can provide examples of script.
Thanks!
If I was you I would create an SSIS package that uses the multi file input. This input let's you pull data from every file in a directory.
Here are the basic steps for your SSIS package.
Check if there are any files in the "working" directory. If not end the package.
Move every file from your "working" directory to a "staging" directory.
You will do this so that if additional files appear in your "working" directory while you are in the midst of the package you won't lose them.
Read all of the files in the "staging" directory. Use a data flow with the multi file input.
Once the reading has been completed then move all of the files to a
"backup" directory.
This of course assumes you want to keep them for some reason. You could just as easily delete them from the "staging" directory.
Once you have your package completed then schedule it using SQL Server agent to run the package at whatever interval you are interested in.