I edited the question based on the solution that Hadi gave.
I am using SSIS in VS 2013.
I have a user variable called MyVariableList and Query.
This is Expression in user variable Query: "SELECT cola, colB FROM myTable WHERE myID IN (" + #[User::MyVariableList] + ")"
I have a Script Task that set the value of #[User::MyVariableList].
Dts.Variables["User::MyVariableList"].Value = sList;
After that, I have A Data Flow Task with OLE DB Source (from 1 database) to another OLE DB Destination (another database on another server).
In the OLE DB Source Editor, I set
Data access mode: SQL Command from variable
Variable name: User:: Query
In the OLE DB Source connection, I have set the DelayValidation to True
Before I even can run the package, I am getting this error
How can I fix this issue ? Thank you
First of all, you are working with a project parameter not a project variable
You cannot achieve this using a parameterized sql command, you have to build the whole query inside a variable, and use it as source (SQL Command from variable)
Parameters are used to specify a single value, not concatenating the query string
Create a SSIS variable (User::Query), change the variable Evaluate As Expression property to True and write the expression in the variable Expression property. Like the following
"SELECT cola, colB FROM myTable WHERE myID IN (" + #[$Project::MyProjectVariable] + ")"
Note: to use a project parameter inside an expression use the following syntax : #[$Project::MyProjectVariable]
Related
Pretty new to BI and SQL in general, but a few months ago I didn't even know what a model is and now here I am...trying to build a package that runs daily.
Currently running this is Excel via PowerQuery but because the data is so much, I have to manually change the query every month. Decided to move it into SSIS.
Required outcome: Pull the last date in my Database and use it as a variable in the model (as I have millions of rows, I only want to load lines with dates greater than what I have in my table already).
Here is my Execute SQL Task:
I set up a variable for the SQL query
and trying to use it in my OLE DB query like this
Execute SQL Task: results, are fine - returns date as "dd/mm/yyyy hh24:mi:ss"
SELECT MAX (CONVACCT_CREATE_DATE) AS Expr1 FROM GOMSDailySales
Variable for OLE DB SQL Query:
"SELECT fin_booking_code, FIN_DEPT_CODE, FIN_ACCT_NO, FIN_PROD_CODE, FIN_PROG_CODE, FIN_OPEN_CODE, DEBIT_AMT, CREDIT_AMT, CURRENCY_CODE, PART_NO, FIN_DOC_NO, CREATE_DATE
FROM cuown.converted_accounts
WHERE (CREATE_DATE > TO_DATE(#[User::GetMaxDate],'yyyy/mm/dd hh24:mi:ss'))
AND (FIN_ACCT_NO LIKE '1%')"
Currently getting missing expression error, if I add " ' " to my #[User::GetMaxDate], I get a year must be between 0 and xxxx error.
What am I doing wrong / is there a cleaner way to get this done?
In the OLEDB source use the following, change the data access mode to SQL command, and use the following command:
SELECT fin_booking_code, FIN_DEPT_CODE, FIN_ACCT_NO, FIN_PROD_CODE, FIN_PROG_CODE, FIN_OPEN_CODE, DEBIT_AMT, CREDIT_AMT, CURRENCY_CODE, PART_NO, FIN_DOC_NO, CREATE_DATE
FROM cuown.converted_accounts
WHERE (CREATE_DATE > TO_DATE(?,'yyyy/mm/dd hh24:mi:ss'))
AND (FIN_ACCT_NO LIKE '1%')
And click on the parameters button and map #[User::GetMaxDate] to the first parameter.
For more information, check the following answer: Parameterized OLEDB source query
Alternative method
If parameters are not supported in the OLE DB provider you are using, create a variable of type string and evaluate this variable as the following expression:
"SELECT fin_booking_code, FIN_DEPT_CODE, FIN_ACCT_NO, FIN_PROD_CODE, FIN_PROG_CODE, FIN_OPEN_CODE, DEBIT_AMT, CREDIT_AMT, CURRENCY_CODE, PART_NO, FIN_DOC_NO, CREATE_DATE
FROM cuown.converted_accounts
WHERE CREATE_DATE > TO_DATE('" + (DT_WSTR, 50)#[User::GetMaxDate] +
"' ,'yyyy/mm/dd hh24:mi:ss') AND FIN_ACCT_NO LIKE '1%'"
Then from the OLE DB source, change the data access mode the SQL Command from variable and select the string variable you created.
Your trying to use the SSIS variable like a variable in the query. When constructing a SQL query in a string variable you simply need to concatenate the strings together. The expression for your query string variable should look like this.
"SELECT fin_booking_code, FIN_DEPT_CODE, FIN_ACCT_NO, FIN_PROD_CODE, FIN_PROG_CODE, FIN_OPEN_CODE, DEBIT_AMT, CREDIT_AMT, CURRENCY_CODE, PART_NO, FIN_DOC_NO, CREATE_DATE
FROM cuown.converted_accounts
WHERE CREATE_DATE > " + #[User::GetMaxDate] +
"AND (FIN_ACCT_NO LIKE '1%')"
I have to consolidate data from from 1000+ databases having the same structure/tables in one unique DB.
DBs may be added and removed potentially on a daily basis so I need to retrieve the list of DBs dynamically and run the dynamically generated SQL query to extract data on each of them.
I designed the Data Flow with a query from a variable that is working fine if executed with a static value:
With a SQL task I get the list of instances, I loop over the them and with a nested Foreach Loop/SQL task I retrieve the database names and create the dynamic SQL with the following statement (DB name is anonymized):
SELECT 'select ''' + name + ''' as DatabaseName, ID from ' + name + '.[dbo].[Orders] as querytext FROM sys.databases WHERE name LIKE ( 'XXX%_%' );
This part is also working fine:
How can I use the result of the SQL task "Execute SQL Task - Get query text" as query to be executed in the Source "OLE DB Source 1" (part of "Data Flow Task 3")?
I tried mapping an Object variable "User::SqlCommandFromSQLTask" in the result set of the SQL task, then set it up as ADO object source variable and with a Script task convert it to string and pass the value to the variable SqlStringFromSQLTask3 (used as source in "OLE DB Source 1") but I get the error Violation of PRIMARY KEY constraint, like if the data flow is always running with a static value I set up as default:
While, if I remove the value from the variable panel, I get the error "Command text was not set for the command object.", even changing the property DelayValidation of the Data Flow to false.
Any help is much appreciated.
When I have used SSIS to connect to multiple SQL Server boxes, I have stored those SQL Server connection strings in a table in a central database. Then I use a query of that table as the input to the foreach loop data flow task. If we ever have to change a sql server connection string, which does happen, we just update that table with the newest value.
I am creating a SSIS package and trying to extract data by calling stored procedures from one database and inserting the result set values into another table of different database. I have created a Execute SQL task to extract the data, a for each loop container to loop through the result set and Execute SQL task within the for loop container to insert the result set data into another database table. I am getting an the following error while inserting the records. I guess its the issue with the mapping.
[Execute SQL Task] Error: Executing the query "insert into EmployeeCount (companyId..." failed with the following error: "Parameter name is unrecognized.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Following the screenshot of the template design
Following is the edit window of execute sql task which in inside the foreach container
The insert statement
insert into EmployeeCount (companyId,dataItemName,dataItemvalue,fiscalYear,fiscalQuarter,PeriodTypeId) values(companyId,dataItemName,dataItemvalue,fiscalYear,fiscalQuarter,PeriodTypeId)
You will have to set "Parameter Name" in chronological order.
i.e companyID parameter must be 0, dataItemvalue to 1 ....PeriodTypeId to 5
Sample :
In the 'Parameter Mapping' for Parameter Name, instead of '?' give the values in a 0 based index i.e. 0,1,2,... till the end.
As an option:
You may be trying to insert too long values into shorter columns
For example if you create Source DB you need to be sure that each column have sufficient maximum length and you don't put largest values (too long values) into this column.
So unfortunately SSIS don't specify the problematic column, and you need to find it by yourself or enlarge maximum length of every column.
Simple Workaround
You can achieve this without using parameters, you can use expressions:
Double click on the Execute SQL Task, Go To Expressions.
add the following expression to SqlStatementSource:
"insert into EmployeeCount (companyId,dataItemName,dataItemvalue,fiscalYear,fiscalQuarter,PeriodTypeId)
values(" + (DT_WSTR,50)#[User::companyId] + ",'" + #[User::dataItemName] + "'," + (DT_WSTR,50)#[User::dataItemvalue] + "," + (DT_WSTR,50)#[User::fiscalYear] + "," + (DT_WSTR,50)#[User::fiscalQuarter] + "," + (DT_WSTR,50)#[User::PeriodTypeId] + ")"
Be aware! This approach makes your solution vulnerable to SQL injections
I have a table that contains just under a million rows. I'm building a form using SSIS that asks for user input and uses the values as parameters to build a view from source data. I'm having trouble getting SSIS to create the view from a variable.
The purpose of this 'tool' is to provide a dialogue that programmatically builds a view and later an update statement based upon parameters defined via a form that will execute an SSIS package. A number of the ppl on my team know 0 SQL. Therefore this circumvents any SQL knowledge. Creating an entirely standalone app is not ideal as it would require too much additional overhead on my side and would deviate from a number of our existing processes that currently use SSIS/SQL to achieve similar results.
With that here is what I've tried/trying.
I have an SSIS package that contains 'Execute SQL Task'
This task brings up a form with 5 inputs (variables)
var1,var2,var3,var4,var5.
some vars are strings others are doubles, ints etc... (they all vary)
You populate the fields and hit okay.
These variables are passed to an 'Execute Package Task'.
Inside this package (Package B)
the vars are used in an 'Execute SQL Task'.
This task is attempting to take the users input and create a view with a where clause containing 4 other variables.
example:
Create View ? AS Select col1,col2,col3,col4 WHERE
col1 = ?
AND col2 =?
AND col3 =?.........
First it appears that using ? in the create view is invalid.
The error being:
Error: 0x0 at Build_Query: Incorrect syntax near '#P1'.
Error: 0xC002F210 at Build_Query, Execute SQL Task: Executing the query "CREATE VIEW ? as Select * from S_t_equip_template
..." failed with the following error: "The batch could not be analyzed because of compile errors.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established
correctly.
Task failed: Build_Query
If I use the create view variable as an expression and remove the variables/paramerters for the where statements, I can create the view no problem.
However the where statements throw errors once I add them back in. I've tried evaluating these as an expression in the 'Execute SQL Task' but as these are of various types I get the error:
[Execute SQL Task] Error: Executing the query "
CREATE VIEW testing AS SELECT
P.label,P.uniq..." failed with the following error: "The metadata could not
be determined because statement 'CREATE VIEW testingagain AS SELECT
P.label,P.uniqueid,C.label as Child_Label,C.uniqueid as Child_uni' does not
support metadata discovery.". Possible failure reasons: Problems with the
query, "ResultSet" property not set correctly, parameters not set correctly,
or connection not established correctly.
No idea what is going on. Any help would be appreciated
I've googled the error and found some info but the other use-cases are so different that it's difficult to understand the actual cause, or another work around.
AS requested (simplified example):
I've created a package variable (datatype string) called: View_Name
Execute SQL Task:
CREATE VIEW #[User::View_Name] AS SELECT
* from table1
where col1 = 100;
Specifically it does not like that I use a variable here.
If I set the View name everything works until: I move on to my Where clauses that contain variables.
Create a variable called type (datatype int)
I map the variable/parameter in my sql task
Example:
CREATE VIEW tempTable AS SELECT
* from table1
where col1 = ?;
This won't work, same error.
If i attempt to do the above via an expression or expressions I get the following error
[Execute SQL Task] Error: Executing the query "CREATE VIEW test_45678 AS SELECT P.label,P.uniquei..." failed with the following error: "Must declare the scalar variable "#".". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Generally having to due with the variables cannot be evaluated this way. I'm guessing I'd need to evaluate each piece individually and build the expression piece by piece. That's fine but very inefficient and not maintainable.
I had some fundamental misunderstanding in my attempt to evaluate my expressions. Specifically syntax issues and having to cast each variable to a string.
My SQLstatement variable
ON C.pid = P.ID
where
C.width >="+(DT_WSTR, 8)#[User::Width] +"-"+ (DT_WSTR, 8)#[User::Range]+........
The final expression looks like so:
"Create View "+#[User::View_Name] + " AS SELECT " + #[User::SQLStatement]
I am facing issue in loading-data-from-multiple-db-to-another-server-using-ssis
I have referred the below link
Loading data from multiple db to another server using SSIS
SSIS Package FLow :
Unfortunately i am getting the error in "Execute SQL Task" as below:
[Execute SQL Task] Error: There is an invalid number of result
bindings returned for the ResultSetType: "ResultSetType_Rowset".
Appreciate if you could help me with the solutions.
Thanks
Based on the comments, to solve your issue
1. Evaluate your SQL Statement. For instance :
2. Once your variable query is evaluating, move to Execute SQL Task. It should look like this:
3. Next the resultSet should look like this (object_variable is of object type)
Why are we not using anything in parameter mapping ?
Answer: If we had a SQL query like Select col1, col2 from table1 where col3 = ?, Then we would be replacing ? with either a parameter or a variable.
In your case, delete everything inside parameter mapping.
Updated : Also, since you're query is Select * into tbl2 from tbl1, ResultSet property should be None instead of any other thing.
You have 2 Execute SQL Tasks
1, First Execute SQL Task, get list of tables and schema, it expects a Full Result set and map it to a object type variable.
2, Foreach loop container, ADO Enumerator, ADO source is the object type variable. Variable Mappings to 2 string type variables, 1 is for table name and 1 is for schema name.
Second Execute SQL Task, it has no Full Result set, no parameter mapping. Because table name/schema name changes are taken care of by Foreach loop container.