An existing SSIS package has Data source with Data access mode: SQL command from variable. User variable is having select statement as value. Now I added a column to select statement in variable and saved the package. This new column is not showing up in data source columns list. Tried closing the package and reopening it, on doing this I no longer see the new column in variable value select statement too. Looking for solutions other than creating a new variable and calling it in data source.
Find the Variable. Right click on it and select Properties.
In the Properties window, you will see EvaluateAsExpression is equal to True. Click the Expression box and then the ellipses/triple dots/.... In there you will need to add the column as part of the string that is being built
Related
I have a table with 3 columns. I need to import this table's data into an Excel sheet. I need to create the Excel sheet dynamically name and timestamp. Is there any reference for this procedure?
If you need to name the Excel file just as a date/time stamp, you can set this up in the SSIS variables.
Create a variable for the path, I've named it ExcelFilePath and set it to C:\Temp\ . Create another called ExcelFileName, and then go into the expression editor, and enter the following:
#[User::ExcelFilePath] + LEFT((DT_WSTR, 30)GETDATE(), 19)
Here's an image of how it should look:
You can then create an Excel Connection Manager, go into the Properties for it, and click the ... button on the Expressions property. A windows will open - under Property select ExcelFilePath, and then click the ... button in the Expression column next to it:
The expression editor will open again, this time enter #[User::ExcelFilePath], like this:
You may need to set up more dynamic parts in the Control Flow or Data Flow if you want to name the file with a dynamic name followed by the timestamp, but you can use the variables in exactly the same way as I've shown here. Likewise, if the path needs to be dynamic, you can set that variable via a script in the Control Flow or Data Flow.
Below link might help you:
Dynamically creating Excel File, ExcelSheets
I have consumed simple web service for addition of two numbers and I am getting the result in xml format that I have stored in variable supposed named as my_data, now I have to insert that result in my sql table named as Data_result by writing query in oledb source editor, but unable to insert data using variable.I just want to store all the result calculated by my web service in sql table.
Since I am newbie to SSIS, any help is greatly appreciated
When I am trying to insert the data by the following query command:
"INSERT INTO Data_Result(Result) SELECT '"+#USER::my_data+"'"
It's giving error:
Error at Data Flow Task [OLE DB Source [55]]: No Column Transformation was return by SQL Command
--------Updated explanation of errors based on the solution given
Error for executing query
If I place the expression without the "" quotes then I get following error
If I place the query in double quotes then following error is shown
And if I remove User from User::Data Variable and place the query in double quotes then I get following screen
although the expression evaluates but after proceeding further on this evaluated expression when I am trying to search for the variable in expression column of Execute sql Task, then I am unable to locate the newly created variable, as shown below
------------Updated question according to other query----------
Here is the picture of my whole work flow
This is what I have did inside the for each loop container under collection tab
And this below setting I have done between Variable mapping tab,
And in below screen shot, I am using Execute SQL Task to enter my data obtained from web service task into database using an insert query, but unable to fire proper insert query,
And below is my XML file,
<?xml version="1.0" encoding="utf-16"?>
Reshma
1988-09-23T00:00:00
,
This name and birthdate I received from web service I want to insert that into database table
--------------Updated question for foreach loop container--------
Below is the foreach loop container I am using,
But still I am not getting node value into variable and also I am suppose to make use of object type variable or it can work with string type of variable.
I suggest you build this SQL statement in another variable, as explained here:
http://consultingblogs.emc.com/jamiethomson/archive/2005/12/09/2480.aspx
Then use that variable in your execute SQL statement
Normally you would try and parameterise the query but in this case (as in many others) it won't work.
Here are some more detailed instructions:
A. Generate the dynamic SQL
Create a new SSIS variable of type string
In properties, set it's 'Evaluate as Expression' to TRUE
In properties, click on 'Expression' and build an expression as follows:
"INSERT INTO Data_Result(Result) VALUES ('"+#[USER::my_data]+"')"
Press Evaluate and verify that the output is valid SQL (you may even wish to paste it into SSMS and test)
B. Get an Execute SQL task to run your SQL:
Drop a execute SQL task on your page. Set it's connection
With the execute SQL task highlighted, go to properties and click expressions
Pick the property 'SQLStatementSource'. In the expression pick your variable that you built in step A and press evaluate.
Press OK on everything
Now whatever change occurs in my_data will automatically be applied to your execute SQL task
Try that for starters and get back to me.
I have some variables (at package level, at dataflow level, doesn't matter) and I need to load them by running an Execute Sql Task.
I added an Execute Sql Task, I have wrote my sql statement in filed SqlStatement, I have changed the ForcedExecutionValueType to string.
Now I want that when the Execute Sql Task is executed the return value that that select returns, to be the exact value for my variable.
example:
Select employer_name from employers where id=1
Returs="John Doe"
value for #employer_name must be "John Doe"
Any hints?
Your more detailed discussion of the issue is a little difficult to follow, so I aplogize if my following steps cover items with you are already familiar.
Steps:
Under the Control Flow tab, drag over an Execute SQL Task and open it.
Select General on the left.
Define the Connection (based on a database source created in Connection Managers below).
Define the SQL Statement.
For Result Set, select Single Row.
Select Result Set on the left.
Select the Add button and notice that a new record was added above.
Under the Result name column header for the new record, change "NewResultname" to "0".
Under Variable Name column header for the new record, select either an existing User variable you've already created or scroll to the top and create a new one.
Select OK and you're done.
Running the Execute SQL Task by itself will now populate the variable. Make certain you've verified that the SQL will return only one value and no more. Otherwise, you will need to modify your SQL with "TOP 1" in it to be on the safe side. When multiple values are expected, then you to apply a variable defined with a Data Type of "Object" and use "Full Result set" instead of "Single Row" in the Execute SQL Task.
Hope this helps.
I am working with SSIS 2008. I have a select query name sqlquery1 that returns some rows:
aq
dr
tb
This query is not implemented on the SSIS at the moment.
I am calling a stored procedure from an OLE DB Source within a Data Flow Task. I would like to pass the data obtained from the query to the stored procedure parameter.
Example:
I would like to call the stored procedure by passing the first value aq
storedProdecure1 'aq'
then pass the second value dr
storedProdecure1 'dr'
I guess it would be something like a cycle. I need this because the data generated by the OLE DB Source through the stored procedure needs to be sent to another destination and this must be done for each record of the sqlquery1.
I would like to know how to call the query sqlquery1 and pass its output to call another stored procedure.
How do I need to do this in SSIS?
Conceptually, what your solution will look like is an execute your source query to generate your result set. Store that into a variable and then you'll need to do iterate through those results and for each row, you'll want to call your stored procedure with that row's value and send the results into a new Excel file.
I'd envision your package looking something like this
An Execute SQL Task, named "SQL Load Recordset", attached to a Foreach Loop Container, named "FELC Shred Recordset". Nested inside there I have a File System Task, named "FST Copy Template" which is a precedence for a Data Flow Task, named "DFT Generate Output".
Set up
As you're a beginner, I'm going to try and explain in detail. To save yourself some hassle, grab a copy of BIDSHelper. It's a free, open source tool that improves the design experience in BIDS/SSDT.
Variables
Click on the background of your Control Flow. With nothing selected, right-click and select Variables. In the new window that pops up, click the button that creates a New Variable 4 times. The reason for clicking on nothing is that until SQL Server 2012, the default behaviour of variable creation is to create them at the scope of the current object. This has resulted in many lost hairs for new and experienced developers alike. Variable names are case sensitive so be aware of that as well.
Rename Variable to RecordSet. Change the Data type from Int32 to Object
Rename Variable1 to ParameterValue. Change the data type from Int32 to String
Rename Variable2 to TemplateFile. Change the data type from Int32 to String. Set the value to the path of your output Excel File. I used C:\ssisdata\ShredRecordset.xlsx
Rename Variable 4 to OutputFileName. Change the data type from Int32 to String. Here we're going to do something slightly advanced. Click on the variable and hit F4 to bring up the Properties window. Change the value of EvaluateAsExpression to True. In Expression, set it to "C:\\ssisdata\\ShredRecordset." + #[User::ParameterValue] + ".xlsx" (or whatever your file and path are). What this does, is configures a variable to change as the value of ParameterValue changes. This helps ensure we get a unique file name. You're welcome to change naming convention as needed. Note that you need to escape the \ any time you are in an expression.
Connection Managers
I have made the assumption you are using an OLE DB connection manager. Mine is named FOO. If you are using ADO.NET the concepts will be similar but there will be nuances pertaining to parameters and such.
You will also need a second Connection Manager to handle Excel. If SSIS is temperamental about data types, Excel is flat out psychotic-stab-you-in-the-back-with-a-fork-while-you're-sleeping about data types. We're going to wait and let the data flow actually create this Connection Manager to ensure our types are good.
Source Query to Result Set
The SQL Load Recordset is an instance of the Execute SQL Task. Here I have a simple query to mimic your source.
SELECT 'aq' AS parameterValue
UNION ALL SELECT 'dr'
UNION ALL SELECT 'tb'
What's important to note on the General tab is that I have switched my ResultSet from None to Full result set. Doing this makes the Result Set tab go from being greyed out to usable.
You can observe that I have assigned the Variable Name to the variable we created above (User::RecordSet) and I the Result Name is 0. That is important as the default value, NewResultName doesn't work.
FELC Shred Recordset
Grab a Foreach Loop Container and we will use that to "shred" the results that were generated in the preceding step.
Configure the enumerator as a Foreach ADO Enumerator Use User::RecordSet as your ADO object source variable. Select rows in the first table as your Enumeration mode
On the Variable Mappings tab, you will need to select your variable User::ParameterValue and assign it the Index of 0. This will result in the zerotth element in your recordset object being assigned to the variable ParameterValue. It is important that you have data type agreement as SSIS won't do implicit conversions here.
FST Copy Template
This a File System Task. We are going to copy our template Excel File so that we have a well named output file (has the parameter name in it). Configure it as
IsDestinationPathVariable: True
DestinationVarible: User::OutputFileName
OverwriteDestination: True
Operation: Copy File
IsSourcePathVariable: True
SourceVariable: User::TemplateFile
DFT Generate Output
This is a Data Flow Task. I'm assuming you're just dumping results straight to a file so we'll just need an OLE DB Source and an Excel Destination
OLEDB dbo_storedProcedure1
This is where your data is pulled from your source system with the parameter we shredded in the Control Flow. I am going to write my query in here and use the ? to indicate it has a parameter.
Change your Data access mode to "SQL Command" and in the SQL command text that is available, put your query
EXECUTE dbo.storedProcedure1 ?
I click the Parameters... button and fill it out as shown
Parameters: #parameterValue
Variables: User::ParameterValue
Param direction: Input
Connect an Excel Destination to the OLE DB Source. Double click and in the Excel Connection Manager section, click New... Determine if you're needing 2003 or 2007 format (.xls vs .xlsx) and whether you want your file to have header rows. For you File Path, put in the same value you used for your #User::TemplatePath variable and click OK.
We now need to populate the name of the Excel Sheet. Click that New... button and it may bark that there is not sufficient information about mapping data types. Don't worry, that's semi-standard. It will then pop up a table definition something like
CREATE TABLE `Excel Destination` (
`name` NVARCHAR(35),
`number` INT,
`type` NVARCHAR(3),
`low` INT,
`high` INT,
`status` INT
)
The "table" name is going to be the worksheet name, or precisely, the named data set in the worksheet. I made mine Sheet1 and clicked OK. Now that the sheet exists, select it in the drop down. I went with the Sheet1$ as the target sheet name. Not sure if it makes a difference.
Click the Mappings tab and things should auto-map just fine so click OK.
Finally
At this point, if we ran the package it would overwrite the template file every time. The secret is we need to tell that Excel Connection Manager we just made that it needs to not have a hard coded name.
Click once on the Excel Connection Manager in the Connection Managers tab. In the Properties window, find the Expressions section and click the ellipses ... Here we will configure the Property ExcelFilePath and the Expression we will use is
#[User::OutputFileName]
If your icons and such look different, that's to be expected. This was documented using SSIS 2012. Your work flow will be the same in 2005 and 2008/2008R2 just the skin is different.
If you run this package and it doesn't even start and there is an error about the ACE 12 or Jet 4.0 something not available, then you are on a 64bit machine and need to tell BIDS/SSDT that you want to run in 32 bit mode.
Ensure the Run64BitRuntime value is False. This project setting can be found by right clicking on the project, expand the Configuration Properties and it will be an option under Debugging.
Further reading
A different example of shredding a recordset object can be found on How to automate the execution of a stored procedure with an SSIS package?
I am new with SSIS packages, There is a specific process I want to accomplish.
1st) I want to get a record set from a SQL Server DB and Mark that I have retrieved the specific rows.
2nd) Loop through each row
3rd) for each row looped through call a web service and pass two fields to the web service as input parameters.
Any help will be much appreciated.
The first step to do this is to create a few SSIS variables. Select View > Other Windows > Variables to display the variables window. Add a variable that will contain your result set from the SQL query. Set the data type to Object. Create additional variables for each column selected in the query.
In the Connection Managers tab, add a new Connection Manger. Edit and configure the new Connection Manager with the SQL Server connection information (e.g., Server name, authentication and database, etc.)
Add an Execute SQL Task to the Control Flow. Edit the task. In the Connection property, select your new Connection Manager. Enter the SQL SELECT statement in the SQLStatement property. Set the ResultSet property to Full result set.
Select the Result Set tab in the Execute SQL Task. Add a new result set. Set the Result Name to 0 and the Variable Name to the object variable you created earlier. Close the editor.
Add a Foreach Loop Container to Control Flow. Create a Precedence Constraint (green line) from the Execute SQL Task to the Foreach Loop Container. This will ensure that the SQL task must succeed before the Foreach Loop starts.
Edit the Forach Loop Container and select the Collection tab. Set the Enumerator property to Foreach ADO Enumerator and select your object variable in the ADO object source variable. Select the Variable Mappings tab. Select your variables that will retrieve the column value for each row. Associate the first variable with the first column by selecting index 0. Index 1 will be the second column, etc.
Add a Web Service Task in the Foreach Loop Container. This task will run once for each row in the result set. You can use the variable values to configure the Web Service Task. You may need to work with expressions if the property doesn't directly accept variables.
If you want to mark your rows one at a time, you can add another Execute SQL Task and place it into the Foreach Loop Container. Configure it similarly to the first Execute SQL Task, but create an SQL UPDATE statement. You might consider updating all rows the rows at once to get better performance. If so, then move the Execute SQL Task out of the Foreach Loop Container and configure the UPDATE statement so that it updates the set of rows.