I am writing my first SSIS package to translate data from an SQL View to a flat text file.
The flat text filename needs to be in the date format "[year][month][day][hour][minute][second][ms].txt".
I have added a script component to generate this name which stores it to a variable #[User::Filename] correctly.
Somehow during fiddling with the Script Task the ReadWriteVariables property has been set to the result of the Script (eg. "20130524140500297").
I have tried removing this setting in the property window of the Task and by editing the task:
This seems to 'take' but when I build the package the setting magically reverts to the useless number!
Is there something I am missing or any way to remove this permanently?
I don't want to revert the file to an earlier version.
Yes, you're missing the fact, that ReadWriteVariables property in Script Component should have a list of variables that you want to modify in your script. You however set it to evaluate as expression :).
To correct this, set ReadWriteVariables to #[User::Filename], mark EvaluateAsExpression property of #[User::Filename], and set the expression to the variable, not the script.
edit: evaluate variable as expression location:
Related
Created a package and deployed on SQL Server 2017. my package is having parameters (package level scope) and want to set values from environment file at the time of create a SQL-JOB-STEP.
I have created environment file and defined my variables perfectly "Integration Services Categolog => SSISDB => MyProject => Environments" and those variables can be assigned at the time of configure & execute to my package parameters which is working well.
but I cannot do the same while create SQL-JOB. I can see in "step properties window" there is a option to locate "Environment" file but when I locate the file then I cannot find the way to assign those variables to my package's parameters. can anyone help me in this please?
please refer picture:
Click the ellipses (...) besides the empty Value spot. There you will be presented with three options (if you didn't have an Environment established, there'd be only two).
The design time value is the default here. If your parameter, MyVar01_PackageLevel, had an actual value displayed, the UI would render it in the default font weight/style.
One row above is a manual override radio button and here you can just hard code the value. Doing this will result in the Value displayed to be in bold
One row below the design time value is where you can associate a parameter to a list of parameter values from your Environment. Scroll through the list of values, which are in no discernible order, and find the target property name. Let's call it V01_PL After selecting that, the above will indicate V01_PL in an italicized font. It won't show the value of V01_PL mind you, it'll just tell you it's driven the configured value.
I would like to store an XML retrieved value as a string variable for later use in a child package within SSIS. The situation is as follows:
I'm retrieving an ID from an XML file in a data flow task (A). I need to save this ID as a variable so I can pass it later on to a child package via package configurations and use it there. The issue is that with the 'Recordset Destination', I can only store it as an object type, and then I can't seem to convert it in my child package to a string again.
I think that I can solve this with a foreach loop, but I don't feel like this is very performant. Is there any other way to do this? I attached some screenshots to make the situation a bit more clear.
Thanks!
Screenshots
You can use a Script Component to Store the ID value in a variable, Just add the script component as destination, select the variable as Read\Write variable in the script form. And assign a value to this variable in the PostExecute() function inside the Script Editor.
You can refer to one of the following link for more information:
How to use variables in a Script Component
Using Variables in the Script Component
Read and Write variables in a Script Component in SSIS (SQL Server Integration Services) using C#
I use Visual Studio 2015 and SQL Server 2016 and also I use COZYROC component for SSIS for working with SFTP.
I what move all files between two folder in SFTP. For that. I have created a variable (V_FilesNameList) with object data type. Then, I have used a COZYROC SFTP Task for getting the list of files name on SFTP server:
This is a configuration for above component :
Then, I have created a foreach loop on my variable (V_FilesNameList).
And inside the loop I want to change the name of each file and copy it to the new address.
But I have this error :
Error: ForEach Variable Mapping number 1 to variable
"User::V_File_Name_Old" cannot be applied.
Error: ForEach Variable Mapping number 919997528
to variable "껤ʧ" cannot be applied.
And when I want to watch the value inside the my variable (V_FilesNameList) in Watch windows, I have this error :
V_FilesNameList Unable to evaluate the expression. An error occurred
that usually indicates a corrupt installation (code 0x80004002).
If the problem persists, repair your Visual Studio installation via
'Add or Remove Programs' in Control Panel.
Why I have those errors? And how can I move the files between two folder in SFTP server with SSIS?
And how can I move the files between two folder in SFTP server with
SSIS?
Download Files (File Transfer Task CozyRoc) to your file system inside some folder (configurable Project Parameter)
Now using the Foreach Loop Container (Set Enumerator -> ForEach File Enumerator ) get complete file path for each file.
Using File Transfer Task move file to the SFTP.
Using File System Task delete all files from the local folder (Where files are download in step 1).
Check the Variable Mappings in your foreach loop container, if the object V_FilesNameList contains a list string of file names, then the variable in your Variable Mappings should be in string data type, so that they could be mapped correctly. I am guessing your mapped variable variable 껤ʧ is not set correctly.
I had the error to variable "껤ʧ" cannot be applied when I was working with a ForEach loop container too. The problem is with NULL's in the ResultVariable. Try to handle the NULLS returned from source. In my case, the object variable was populated from a query so all I had to do was use ISNULL sql function.
I'm trying to modify one of the FireDAC sample projects in order to use an existing SQLite file as the database source. The sample works fine unmodified and connects to its database. However, I can't figure out where the database it connects to is specified, in order to change it.
According to the documentation, there should be a Database property on the TFDPhysSQLiteDriverLink component. There isn't: it doesn't exist. I even converted the form to text and looked through all components' customized properties, and there's no path defined anywhere. Nor is there in code - the sample is very small and there's no path defined at all.
The other option on the documentation is to include the FireDAC.Phys.SQLite unit, although that doesn't explain how to set the database, since as far as I can tell that unit just includes the component. And when I search for Database properties (see attached image) none of them in any class in that unit seem to be quite what I'm after. The closest is a string that's for a backup component - I doubt that's what I need. There is a SQLiteDatabase property in the TFDPhysSQLiteConnection class but that's read-only.
List of all Database properties defined in the FireDAC.Phys.SQLite unit
I also tried creating a temporary connection definition at runtime, by double-clicking the TFDConnection component. That only gives an exception:
Exception double-clicking the TFDConnection component
The only solution to this I found is in the XE5 documentation, where it says to set the $(PUBLICDOCUMENTSDIR) environment variable. I already had to do that to get the demo to run (previously, it threw the same exception on the line FDConnection1.Connected := True;; it doesn't now, the demo runs perfectly at runtime.) That change obviously hasn't affected the designer, and I don't even know if I'm looking in the right place, since after all the documentation talks about setting the Database property.
So I'm stumped. Where does it set the database? It's not in the DFM or any streamed properties; it's not in the property defined by the documentation (TFDPhysSQLiteDriverLink.Database doesn't exist, nor does anything that looks like it); it's not in the TFDConnection designtime editor (even though it throws an exception, a file specified as a property here would appear in the streamed DFM, I'd think); it's not in code; ...where else can it be?
(I have never used FireDAC before so am a complete noob, btw. I'm self-teaching via the documentation and samples.)
You don't actually need a TFDPhysSQLiteDriverLink for a minimalist FireDAC project, and using one rather confuses the issue if you're trying to make a connection to a database for the first time.
Try this:
Make a note of the name including path of a Sqlite db.
Start a new VCL project and drop a TFDconnection, TFDQuery, TDataSource & TDBGrid onto its form and connection them up. Set the TDFQuery's Sql to select * from some table you know exists in the db.
Right-click the TFDConnection and select Connection editor from the pop-up.
Set the DriverID to SQLite and insert your db name into the Database Value box.
Open the FDQuery.
If you compile and run the project, you'll get an exception telling you a class factory for a TFDGUIxWaitCursor is missing (this is the sort of thing I love about FireDAC), but that's easily fixed by dropping one onto your form. Notice that you don't have to connect it using the Object Inspector to any of the other FD components.
After that, you can add a TFDPhysSQLiteDriverLink and set its DriverID to the same as for the TFDConnection.
I ussualy roll my own class and handle the OnBeforeConnect event
Something like this
procedure TSQLiteConnection.SQLiteConnectionBeforeConnect(Sender: TObject);
begin
if not(TFile.Exists(DatabaseFilePath)) then
Params.Values['OpenMode'] := 'CreateUTF16'
else
Params.Values['OpenMode'] := 'ReadWrite';
Params.Values['Database'] := DatabaseFilePath;
DriverName := 'SQLite';
end;
The DatabaseFilePath is just a string field of the class, so basically you can put any file path there
TSQLiteConnection is, of course, a TFDConnection descendant
I have one temp folder C:\Mydata inside that i have .csv files with name
demo1.csv demo2.csv...etc
Now I want to insert data from all those files into database table
for this I have taken Foreach Loop Container in the property Directory I mention
C:\Mydata
inside Foreach Loop Container taken a Data Flow Task in that Flat File Source is there in the connection manager property in file name what should I mention, so that it will traverse through all folder and all files whose extension is .csv
I did like C:\DEMO\*.csv which is giving me error :
no file found path is wrong
From what i interpreted, your issue was assigning the resulting file from the foreach to the connector.
To do that you need to assign the filename from the Foreach to a variable.
The first step to be taken is to create the variable. To do that you need to activate the "Variables" window:
Then you need to create a new variable of type string and rename it.
For more information on SSIS variables and how to create them, see http://msdn.microsoft.com/en-us/library/ms140216.aspx
Now you need to assign the new variable (in this case i named it "filename") to the foreach loop:
NOTE THAT WHILE THE OPTION "Name Only" IS SELECTED IN THE FIRST WINDOW, IN YOUR CASE YOU PROBABLY WANT "Fully Qualified"!!
The screenshot was taken from a working project i have and i am using only the name (Name Only) as opposed to the full path (Fully Qualified)
After doing that you need to assign that variable to your flat file connector using expressions:
Which version of MSSQL and SSIS are you using? And why are you using C:\Mydata in one place and C:\DEMO in another?
The basic technique of looping over files is described here.
There is lot of information about these steps in the documentation and on the web. If you need more help, please be as specific as possible about what you've tried, what isn't working, what versions you're working with etc.