I have a Stored Procedure activity in my Azure Data Factory pipeline that recalls the following stored procedure on Azure Synapse:
CREATE PROCEDURE schema.procedure_name #portafoglio INT [...]
The activity in configured in order to use a variable (portafoglioElab), previously set in the pipeline:
My problem is how to pass a Null value as input parameter to the procedure via the Azure Data Factory Pipeline in the Store Procedure activity.
I've tried by setting the variable portafoglioElab with null, "null", None, #coalesce(null) but I'm not finding the right value corresponding to sql NULL.
This is the step of the pipeline where I set the variable:
It is a step inside the IF condition of the whole pipeline:
Indeed I'm getting this error:
Cannot create Sql Source. Please double check the connection string, stored procedure are set with correct format. Error: The value of the property 'Value' is invalid for the stored procedure parameter 'portafoglio'.
Can please someone help me?
Thanks in advance
I've found a solutions with the dynamic content.
It is not only possible to put dynamic content in the value of the parameter, but even the parameter itself cam be dynamic.
I solved after many tests with this value:
#json(concat('{"portafoglio": {"value": ', variables('portafoglioElab'), '}}'))
In this way I can handle null and not null values.
Pay attention that the documentation is wrong, the content has to be a dictionary, not a string.
Pass -1 or some other sentinel and handle it in the proc
Related
I have an Execute SQL Task which tries to execute a stored procedure, like this:
EXEC usp_stored_proc ?, ?, ? OUTPUT, ? OUTPUT;
I have 4 variables mapped to parameters. Ignoring the output parameters, these are both strings mapped to NVARCHAR params (as expected by the stored procedure).
When I run the package, an error tells me that execution failed with the message input string is not in the correct format. However, when I use a breakpoint to find the runtime values of the input parameters (or at least the variables mapped to them) and execute the same line of SQL in SSMS using the runtime values, it works fine.
Can anyone help? I'm at the end of my tether with this. I can't even find out the exact parameter causing the issue although it's probably both as the values follow the same format.
More details:
Connection type: OLE DB
Input Variable: String = schema.table
Mapped Param: NVARCHAR, ParamName = 0, ParamSize = -1
UPDATE
Solved the issue by making a new execute sql component that calls a stripped down procedure. I then slowly added lines of code to the procedure and additional parameters until arriving at the same component I started with and now it works. Comparing the original and rebuilt tasks, I see absolutely no differences (same with the procedure), so I don't know why this issue was occuring.
Try changing the parameter size (ParamSize) to match the parameter size within the stored procedure; if nvarchar(50) then set it to 50.
Solved the issue by making a new execute sql component that calls a stripped down procedure. I then slowly added lines of code to the procedure and additional parameters until arriving at the same component I started with and now it works. Comparing the original and rebuilt tasks, I see absolutely no differences (same with the procedure), so I don't know why this issue was occurring.
I’m experiencing a frustrating issue when trying to call a proc in an OLE DB source task. I’m using the SQL command from variable data access mode but I can see that it isn’t evaluating my variable correctly.
My variable (with ValidateAsExpression set to True) uses an expression to create a sql command like “EXEC ProcName ‘Param'” where the value of Param comes from a variable who’s value I set using an EXEC SQL task. Below is the expression:
“EXEC ProcName ” + “‘” + #[User::vDateThreshold] + “‘”
If I use a variable in my source that references a static value it works fine, but the issue seems to be when I use a variable which reference another variable in its expression.
Has anyone else come across this issue?
I’m using this method because I’ve had a similar issue when trying to use a parameter with the sql command data access method.
Thanks in advance
I’m using this method because I’ve had a similar issue when trying to use a parameter with the sql command data access method.
The right way to do that is by using SQL Command with parameters:
EXEC ProcName ?
And select #[User::vDateThreshold] as parameter.
Parameterized OLEDB source query
If it is not working then check your procedure code and make sure it generate a specific result set. If the result set is dynamic and columns are not fixed then you have to define it in the query using WITH RESULTSETS keyword.
https://www.mssqltips.com/sqlservertip/2356/overview-of-with-result-sets-feature-of-sql-server-2012/
From the name of #[User::vDateThreshold it seems like an SSIS datetime variable. Try setting this to a variable with an explicit cast and then executing the stored procedure with the variable. Make sure there that are single quotes (') within the CAST function as you would use if this was done in SSMS. When concatenating a datetime variable within a string variable in SSIS, the datetime variable must be converted to text, which is done with (DT_STR, length, codepage) in the sample expression below. I'm not sure what version you're using, but testing this out on SSDT for Visual 2017 worked fine for me. This will cover if you still want to hold the SQL in a variable, however the solution that #Hadi posted is a good option if you'd prefer to go that route.
"DECLARE #pDate DATETIME
SET #pDate = CAST('" + (DT_STR, 50, 1252)#[User::vDateThreshold] + "' AS DATETIME)
EXEC ProcName #pDate"
Thank you for the responses to my question.
I actually found the issue was with the ordering of my tasks in the package. When I looked closer at the values assigned to the relevant variables by using a break point on my exec SQL task I could see the wrong date was being passed to my proc. When I set the value of vDateThreshold at an earlier point the correct date value was assigned.
I think this was a case of looking at something for long enough that I was missing the obvious.
I'm working in report builder, calling a stored procedure that has a parameter that, when null, sets itself to a certain value. I want to display what this parameter is set to on the report. From experimenting, Report Builder's parameter collection only shows the parameter as it is sent from the report.
Alternatives that I've considered but can't get to work or are sub-optimal:
Adding the parameter to the select statement. The main drawback is this won't display a value if there are no results.
Using a return value or output parameter. There doesn't seem to be a way to do this.
Re-creating the "null" logic in the stored procedure. The correct output is displayed but this is a code fork.
How can I display this value? Is there a way to show a return value or output value?
You could change the procedure to return the parameter value in a UNION ALL select so that a row with the parameter value will always be returned. That row could have NULL for all the other columns so that you can filter it out in the rest of the report.
Another possibility is to add a second dataset to the report that does nothing but get the value of the parameter based on what you pass. That, however, is also a sort of code fork. The fork could be mitigated, however, by putting it in a UDF, and resourcing the same UDF in both datasets.
Yet another possibility is to replicate the logic of populating the parameter in a Custom Code block in the report. However, that is also a code fork.
I've never worked in Report Builder interface but I do have quite a bit of experience building reports in BIDS/VS... There it's a simple matter of setting the parameter default, in the rdl, to match the default in the stored procedure.
I'm learning SSIS and this seems like an easy task but I'm stuck.
I have a CSV file Orders.csv with this data:
ProductId,Quantity,CustomerId
1,1,104
2,1,105
3,2,106
I also have a stored procedure ssis_createorder that takes as input parameters:
#productid int
#quantity int
#customerid int
What I want to do is create an SSIS package that takes the .csv file as input and calls ssis_createorder three times for each row in the .csv file (the first row contains column names).
Here is what I have done so far.
I have created an SSIS package (Visual Studio 2005 & SQL Server 2005).
In Control Flow I have a Data Flow Task.
The Data Flow has a Flat File source of my .csv file. All of of the columns are mapped.
I have created a variable named orders of type Object. I also have variables CustomerId, ProductId, & Quantity of type int32.
Next I have a Recordset Destination that is assigning the contents of the .csv file into the varialbe orders. I'm not sure about how to use this tool. I'm setting the VariableName (under Customer Properties) to User::orders. I think that now orders holds an ADO record set made up of the contents from the original .csv file.
Next I'm adding a ForEach Loop Container on the Control Flow tag and linking it to the Data Flow Task.
Inside of the ForEach Loop Container I'm setting the Enumerator to "ForEach ADO Enumerator". I'm setting "ADO object source variable" to User::orders". For Enumeration mode I'm selecting "Rows in the first table".
In the Variable Mapping tab I have User::ProductId index 0, User::Quantity index 1, User::CustomerId index 2. I'm not sure if this is correct.
Next I have a Script Task inside of the ForEach Loop Container.
I have ReadOnlyVariables set to ProductId.
In the Main method this is what I'm doing:
Dim sProductId As String = Dts.Variables("ProductId").Value.ToString
MsgBox("sProductId")
When I run the package my ForEach Loop Container turns Bright Red and I get the following error messages
Error: 0xC001F009 at MasterTest: The type of the value being assigned to variable "User::ProductId" differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object.
Error: 0xC001C012 at Foreach Loop Container: ForEach Variable Mapping number 1 to variable "User::ProductId" cannot be applied.
Error: 0xC001F009 at MasterTest: The type of the value being assigned to variable "User::Quantity" differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object.
Error: 0xC001C012 at Foreach Loop Container: ForEach Variable Mapping number 2 to variable "User::Quantity" cannot be applied.
Error: 0xC001F009 at MasterTest: The type of the value being assigned to variable "User::CustomerId" differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object.
Error: 0xC001C012 at Foreach Loop Container: ForEach Variable Mapping number 3 to variable "User::CustomerId" cannot be applied.
Warning: 0x80019002 at MasterTest: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (12) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "Package.dtsx" finished: Failure.
Dts.TaskResult = Dts.Results.Success
Any help would be appreciated
One of my coworkers just give me the answer.
You don't need the the ForEach Loop Container or the RecordSet Container.
All you need is the Flat File Source and an OLE DB Command. Connect to your database and inside the OLE DB Command select the appropriate connection.
In the Component Properties enter the following SQLCommand:
exec ssis_createorder ?, ?, ?
The "?" are place holders for the parameters.
Next under the Column Mappings tab map the .csv file columns to the stored procedure parameters.
You are finished go ahead and run the package.
Thanks Gary if you were on StackOverFlow I would give you an upvote and accept your answer.
If I understand correctly, what you want to do is execute a stored procedure 3 times for each row in the data source.
What if you just create a data flow with a flat file data source and pipe the data through 3 execute sql command tasks? Just map the columns in the data to the input params of your stored procedure.
Maybe I'm not seeing it correctly in your question and I'm thinking too simple, but in my experience you need to avoid using the foreach task in SSIS as much as possible.
I suspect that you need to look at your Data Flow task. It's likely that the values from the source CSV file are being interpreted as string values. You will probably need a Derived Column component or a Data Conversion component to convert your input values to the desired data type.
And, I think #StephaneT's solution would be good for executing the SP.
I'm not sure if this answers your question. But I was looking to do this and I achieved it using the BULK INSERT command. I created a staging table with all of the columns in the csv file, and instead of a stored procedure I used a INSTEAD OF INSERT trigger to handle the logic of inserting it into many tables.
So here is the error...
An error occurred while saving the
Panel.
System.ArgumentOutOfRangeException:
Index was out of range. Must be
non-negative and less than the size of
the collection. Parameter name: index
at
System.ThrowHelper.ThrowArgumentOutOfRangeException()
at
System.Collections.Generic.List`1.get_Item(Int32
index) at PanelController.Save(Int32
ID, FormCollection FormValues)
During debugging, I checked the parameter it was adding to the save stored proc, and the ID it was adding was 0. Then, when it used this
oDal.Execute("Lending.uspPanelSave")
Item.PanelId = oDal.Parameters("#PanelId").Value
To retrieve the ID to return it, it set it as 1000? Anyone know what the problem is?
Use SqlServer Profiler (in performance tools) to see what is realy send to SQL Server.
Sounds like it's not finding an output parameter called #PanelId in your command object. Was the parameter added and declared with the correct direction? Is the parameter declared properly in the stored procedure?