I have a .7z archive that I manage using 7zFM (File Manager) and I can easily add objects to this archive. However the objects in the archive are encrypted with AES like I want them to be, but when I add something new I cannot seem to add encryption to that file within the archive unless I decrypt the entire archive, then recrypt it.
Is there an easy way to add encryption, or perhaps remove encryption, of specific files within a .7z archive?
Select new files to be added
Right click (context menu) -> 7-Zip -> Add to archive..
Select an existing archive
Encryption, enter password
Click OK
Related
I'm building a SSIS package (using VS 2017) to load a bunch of CSV files from a specific folder. It works great using the ForEach File loop. Data Flow task has a Flat File Source and OLE DB Destination. I want to be able to save the filename in the same table, along the data from the CSV file. How can I do this??
thanks a lot for your time and help
One other method if you want to load the entire file path of the flat files is
Right click on Flat File Source in Data Flow tab
Click "Show Advanced Editor"
Click "Component Properties"
Under Custom Properties you will find "FileNameColumnName".
If you give a name (e.g.: FlatFileName) then that will appear as one of the source output columns in mapping that contains the file name value(full path with file name). It can be mapped to any available column in destination or modified using Derived column task to get only file name.
In my development, I have stored mostly the entire path which helped me in better tracking.
One other method for anyone who comes across this question, instead of using ForEach loop you can also use a simpler method
Right click in Connection Manager
Click on "New Connection"
Select "MULTIFLATFILE" connection type and click Add.
In Connection manager editor, insert location and use wildcard * (E.g.: \\ABC\XYZ\file_*.txt) to pick all the flat files in that folder.
This automatically loops through all the flat files in that folder.
Hope this helps!
The ForEach File Enumerator can capture the file name and assign it to a variable, which can then be routed to the connection string variable of a connection manager for dynamic loading. This same methodology can be applied to write the file name to a database table.
In the data flow, add a Derived Column transformation and add a new column called 'FileName' (or whatever) and then set it's value to the variable value that the ForEach File Enumerator is setting for the file name.
There are many aspects of what I want to do but I think learning one piece will let me derive the rest.
I have an SSIS package that uses powershell to download a publicly available zip file, an execute script to unzip with 7zip and then data flows to load the unzipped files to corresponding tables.
What I want to do is add the file name (and eventually other aspects of the file like creation date, record counts and so on) from any one of the unzipped files to a log table that keeps track of the summary level details of the files.
How do I dynamically store this type of information as part of the package? Derived columns? But what's the input? Thanks!
There are many options for dynamically working with files through SSIS. Below is an overview of one method. Of course this can vary, depending on your specific needs and requirements.
Add a Foreach Loop Container. On the Collection pane, the Folder property can either be set using the
GUI as well as through a parameter or variable with the Directory
expression. Searching sub folders can also be set by checking the "Traverse subfolders" checkbox or using the Recurse expression like the Folder field.
The Files field will indicate the files to use and wildcards can be
used. * will match any number of characters. For
example, *.csv will get all csv files regardless of name and
Test*.txt will return all .txt files with names beginning Test,
regardless of how many or which characters follow. To limit this to
a single character, use ?. The FileSpec expression will allow
this to be set dynamically similar to the directory by variable or parameter.
The Variable Mappings pane will allow for setting a variable to hold a file name from the directory. Add a variable that will hold the file name to index 0 to map these.
You indicated that you wanted to store the file name. The detail of this can be controlled from the "Retrieve file name" field on the Collection window. As their names imply, Fully Qualified will hold the complete file path, Name and Extension will return the file name with extension, and Name Only is just the file name.
As for other aspects of the file, I'd recommend a using a Script Task for this for more functionality. The C# FileInfo class provides options for finding details about the file such as the creation date, last time the file was accessed, and when the file was most recently written to. Additonal information on this can be found here.
For the record counts from the file, you'll need to create a Connection Manager for this and work with the data within the package. I'm assuming these are flat files? If so, creating a Flat File Connection Manager, and setting the same variable from the Variable Mappings pane of the Foreach Loop to the ConnectionString expression of the Connection Manager will allow you to dynamically loop through each file. Make sure that the Fully Qualified option is used for the "Retrieve file name" field as earlier if you decide to do this. You will also want to configure the correct columns and data types for the Connection Manager ahead of time. This same process can be followed for Excel files, however the variable with the file name will be used on the ExcelFilePath expression instead.
As for storing information about a file in a log table, there are a multitude of options for these. A very basic example of an Insert statement within an Execute SQL Task that's placed within the Foreach Loop is below. The 3 part table name is only necessary if you're using a table that differs from the initial catalog of the Connection Manager. The ? is the parameter marker (assuming this is an OLE DB connection). After this, map the same variable/parameter that stores the file name using the Parameter Mapping pane. Set the direction to Input, appropriate data type (likely VARCHAR/NVARCHAR), 0 in the Parameter Name field to indicate this is the first parameter in the SQL statement (additional ? can be used for subsequent parameters in the SQL statement, then increment this field in accordance), and the default Parameter Size can be left at -1. Again, this is a simple example and you'll probably want store more about the files and their contents, but this can get you started.
Sample SQL Insert:
INSERT INTO YourDataBase.YourSchema.YourTable (ColumnToHoldFileName)
VALUES (?)
you can use Variable to store File name when your loop the files, and after file been loaded to table, then u can use current file name to insert/update log table.
figured it out from looking at other posts. I had to expand the parameter size...easy fix!
I'm trying to create simple project in which I'd like to download XML files from given website. I have stored files names in DataBase table. What I have done looking at this tutorial: Implementing Foreach Looping Logic in SSIS is:
a. Read all distinct rows from my Table (let's call it XMLTable)
b. Assign result of this query to User variable called: nameOfFileToDownload
c. Created For Each Loop container
d. Configured to assign localy each row with file name to download to: nameFileForeachLoop variable
e. Download files from concate link as a path using HTTPManager with dynamic file name from nameFileForeachLoop variable.
f. Created XMLFlatFile connection for dummy file - I assumed after reading from above tutorial.
The problem is now that this loop container works but doesn't download files separately - still to one file which at the end is empty. My nameFileForeachLoop variable is not updated during each LOOP iteration. What's more I have noticed that during FLAT FILE creation I have only CSV and TXT extension available. I have tried many approaches but without results. Can you help me how to download XML files?
For example I have following link to XML: nbp.pl/kursy/xml/c001z180102.xml What changes here is last part of this link with XML extension which I get from my XMLTable.
I have configured my components as follows:
You are on the right track, but need some amendements.
Do not create and configure Flat File Destination connection manager unless you are creating tables in .CSV or .TXT files. In provided example author selects data with dynamic queries and stores the results in dynamic txt files. As I understand, this is not your case.
Here are some examples how to download and save files with HTTP in SSIS. Sample download script and Review of different approaches to HTTP download.
As part of migration from traditional system to new technology, I need to rename N number of files[.txt, .pdf, .xl, etc] available in the particular folder using SSIS.
Move the file to destination
Parse the prefix of files which is used as ID for associating with the record in the table.
Ex: 1012BA12_Attach_Emp.doc [ID=1012BA12]
Then I need to go to database and lookup the new ID.
Ex: old ID=1012BA12 and equivalent new ID=512
Then replace the old ID with new one.
Ex: 512_Attach_Emp.doc
Insert one row to some table with respect new name & path.
I have been used the for each file enumerator, Execute sqltask and file system task
but it's consuming a day to do so.
Please guide me best approach.
The issue you are having is likely to be on the database side, not SSIS.
Do you have indexes on the tables you are accessing?
Are the files local to the SSIS instance, or does SSIS access the files remotely?
I have this task that needs investigation as to why the md5 value of a file keeps changing.
Example:
I need to generate the diagnostic file of a certain machine.
After generating the file, it produces a .zip file, say, Diag.zip which contains all the information/files of that certain machine.
Inside Diag.zip file contain a .xls, say, Data.xls which contains all the summary of all files in that certain machine, includes, the directory of the file, file version, file size, create time and md5.
Then save all the information of Data.xls in database.
After a day or so, do it again back in Step 1-4.
Then when I queried all the save data of Data.xls in the database in a 2 weeks range, and it shows that almost all files in that certain machine have its md5 value changed.
The question is: Why is it that md5 value always changed every time I generated a new diagnostic files?
There seems to be an issue with excel files, in particular Excel 2003 xls files. Whenever they get opened in Excel, even if they don't get changed and don't get saved, Excel automatically updates some of the file's metadata, such as the "Document Properties and Personal Information" and "Last Accessed Statistics". Therefore, the file every time it gets opened changes a little bit, and this makes that the MD5 changes also.
One way to avoid this problem is to remove "document properties and personal information".
Remove hidden data and personal information from Office documents. Excel 2007: Remove Hidden Data and Personal Information from Office Documents
Remove hidden data and personal information from Office documents. Excel 2013, Excel 2010: Remove Hidden Data and Personal Information by Inspecting Workbooks
Other way to avoid this would be to use xlsx files. I have been trying to replicate this behavior in xlsx files, but it seems it only happens on xls (2003).
The MD5 is based on a lot of things. But I can assume filesize, filename & creationdate.
If one of those changes, the md5 hash changes. The exact same file will always return the exact same md5 hash. A new file always generates a new md5 hash.