I have an existing Snowflake task that is scheduled to run once an hour and is using the WHEN condition to check if a particular stream has data. I'd like to ALTER this task so that it no longer uses the when condition, however, unset does not seem to work and I cannot modify it to NULL. The one workaround we've found is you can modify it to TRUE which seems to work (the statement executes and the task does run), but I'd prefer to have it unset like the other tasks that do not have a when condition set. I also recognize I could drop the task and recreate it, however, then I'd lose the history. This is not a significant problem, but seems strange that it cannot be modified after the fact and I've not seen any documentation that indicates this is expected behavior.
Note: I've made sure the root task is suspended before trying to make any changes.
alter task example_task unset when;
SQL compilation error: invalid property 'when' for 'TASK'
alter task example_task modify when NULL;
Invalid expression for task condition expression. Expecting one of
the following: [SYSTEM$STREAM_HAS_DATA]
alter task example_task unset condition;
SQL compilation error: invalid property 'condition' for 'TASK'
(Note: tested ^ because the column is labeled 'condition' when you run show tasks)
alter task example_task remove when;
SQL compilation error: syntax error line 1 at position 31 unexpected
'when'.
(Note: tested ^ because you use 'remove' to change the 'after' parameter)
alter task example_task modify when TRUE;
Statement executed successfully.
As a side note, this came to our attention while trying to use the snowflake terraform provider which unsuccessfully tried to update the when condition. Now I think I know why.
As an alternative approach, why not replace the whole task using CREATE OR REPLACE TASK...?
Related
we have a requirement where SSIS job should trigger based on the availability of value in the status table maintained,point to remember here that we are not sure about the exact time when the status is going to be available so my SSIS process must continuously look for the value in status table,if value(ex: success) is available in status table then job should trigger.here we have 20 different ssis batch processes which should invoke based on respective/related status value is available.
What you can do is:
Scheduled the SSIS package that run frequently.
For that scheduled package, assign the value from the table to a package variable
Use either expression for disabling the task or constraint expression to let the package proceeds.
Starting a SSIS package takes some time. So I would recommend to create a package with the following structure:
Package variable Check_run type int, initial value 1440 (to stop run after 24 hours if we run check every minute). This is to avoid infinite package run.
Set For Loop, check if Check_run is greater than zero and decrement it on each loop run.
In For loop check your flag variable in Exec SQL task, select single result value and assign its result to a variable, say, Flag.
Create conditional execution branches based on Flag variable value. If Flag variable is set to run - start other packages. Otherwise - wait for a minute with Exec SQL command waitfor delay '01:00'
You mentioned the word trigger. How about you create a trigger when that status column meets the criteria to run the packages:
Also this is how to run a package from T-SQL:
https://www.timmitchell.net/post/2016/11/28/a-better-way-to-execute-ssis-packages-with-t-sql/
You might want to consider creating a master package that runs all the packages associated with this trigger.
I would take #Long's approach, but enhance it by doing the following:
1.) use Execute SQL Task to query the status table for all records that pertain to the specific job function and load the results into a recordset. Note: the variable that you are loading the recordset into must be of type object.
2.) Create a Foreach Loop enumerator of type ADO to loop over the recordset.
3.) Do stuff.
4.) When the job is complete, go back to the status table and mark the record complete so that it is not processed again.
5.) Set the job to run periodically (e.g., minute, hourly, daily, etc.).
The enhancement hear is that no flags are needed to govern the job. If a record exists then the foreach loop does its job. If no records exist within the recordset then the job exits successfully. This simplifies the design.
I started a new job a few months ago and am noticing some of the older SSIS packages are built with dummy tasks. For example, if there is a Execute SQL Task and it produces a Boolean result, the True result may go to the task that needs something done. However, there is also a dummy task for False that does nothing and is labeled as such.
Here's an example of what I'm seeing.
Is this a best practice or proper way to code these types of Tasks? The way I code these types of tasks is to only code the needed task and ignore the other. In the case of my image example, I would code the True but not add a task for the False.
Just curious if I'm doing something wrong here.
Why adding a dummy task that does nothing?
(1) Informing Developers
If the goal is Informing developers that edits the package then you can write an annotation beside of the True precedence constraint, and write that if False result is not considered or does or does nothing
(2) For Debugging purposes
If the adding a dummy task is for debugging purposes, then it is not a bad idea. Because it will make the package flow more clear for the viewer. But you have to disable this component before deploying the package on the production server.
(3) To apply an expression
If the goal is to apply an expression in the precedence constraint like mentioned in the comments, then the proper way is to use an Expression Task (in case that it is connected to a dummy task)
Also based on this Microsoft article:
A precedence constraint can use an expression to define the constraint between two executables
So if expression are needed, then write it in the True precedence constraint, or if expression is needed to assign some values to a variable then like i mentioned above use an Expression Task
In one line standard practise is to create the log if something does not go according to desired logic. It should be logged for future debugging.
If you are expecting true as a result of some boolean logic and it turns out to be false then there should be some structure to log it.
Now coming to use of dummy task it is of no harm but instead of that, some task should be used which can create some logs for future reference.
I have 3 tasks in an SSIS package. The first task is an Execute Process Task that runs a PowerShell command to download a file from a website. If the command fails the task fails. So I have two connections from it to two other tasks: one for success and one for failure.
The package works as expected. If the Execute Process Task succeeds it runs the associated success task. If the fails it runs the associated failure task.
The problem is that if the Execute Process Task fails, it shows the task as failing even though it correctly/properly ran the failure task connected to it.
Is there anyway to get it to stop showing as failure but still run the associated failure task even when the Execute Process Task fails?
Update 1
If you need that the package doesn't result failure then you should set FailPackageOnFailure and FailParentOnFailure to False on the task you should also set MaximumErrorCount on the package itself to something greater than 1. If the Execute Process Task fails it will increment the package's error count and if the error count exceeds MaximumErrorCount then the package can/will still fail. Else if you only need that the task doesn't shows failure it cannot be done
SSIS -- Allow a task to fail but have the package succeed? (See all answers, not only the accepted one)
Initial Answer
You cannot use a failure precedence constraint if the precedent task will always succeed
I don't think you can do this. because the failure precedence constraint (connector) only works if the precedent task fail. You can use ForceExecutionResult property to let a task always success, but the failure connector will never be used.
Workaround
I am not sure if this can help, but instead of using failure precedence constraint, store the Execution Value in a Variable, using ExecValueVariable property, and add Expressions to the precedence constraints *(both connector will have the Success constraint and a similar Expression:
#[User::ResultValue] = 1
OR
#[User::ResultValue] = 0
Side Note: ExecValueVariable and ForceExecutionResult and other properties are found in the properties Tab, click on the Task and press F4 to show it
So it seems that you want the task to be shown as success even though the parent task fails and the task corresponding to the failure output is successful.
To do this you need to edit the field FailTaskReturnCodeIsNotSuccessValue to False so that even though the parent task doesn't return the successValue the task is not shown as failed.Hope this helps!!.
I have a question on how to set up precedence constraints in SSIS.
I have the following flow in the package:
The execute SQL task returns a string value in this format '1111' that is then stored in the variable called "data"
The point of it all is to control which script tasks gets to execute
For example, if the value is "1111" then all 4 scripts get to run.
If the value is "1011" then scripts 1,3,4 get to run... you get the picture
The scripts DEPEND on the previous one in all cases.
The constraints evaluate with an expression such as this: SUBSTRING(#[User::data], 2,1)=="1" for script 2, for example.
The problem:
If one of the scripts dont run, then the next one wont either (because the constraint never got to the evaluated). For example, for data = "1011", the scripts 3 and 4 never get to run because number 2 never ran...
Do you know a better way to make this work?
Using SQL Server 2008 with BIDS
I agree with #Iamdave for his direction that says modify your script in each script do the check there if it should or should not execute rather than using expression constraints.
However, because someone might want to do this with Data Flow Tasks or something here is a way to do it with SSIS components. Your problem is that you want to conditionally execute a task but whether or not that task gets executed you then want to conditionally execute another task. By placing each script task in a Sequence Container the precedence between sequence containers will always make the other container execute as long as that prior container does not fail or does not have a expression as a constraint. But once in the container you need to be able to set a Conditional Precedence. You can do that by adding a dummy task of some kind ahead of the script task and adding the constraint on that precedence.
Add some conditional code into your scripts.
For example, at the start of script task 1:
if firstcharacter(Variable) = 1
do things
else
do nothing
And then in script task 2:
if secondcharacter(Variable) = 1
do things
else
do nothing
and just make sure that all the data passes through, only being transformed if the right character in your variable is met.
Let me also back up a step - I'm trying to implement a sanity check inside an IS package. The idea is that the entire package runs in a read uncommitted transaction, with the final step being a check that determines that certain row counts are present, that kind of stuff. If they are NOT, I want to raise an exception and rollback the transaction.
If you can tell me how to do this, or, even better, suggest a better way to implement a sanity check, that would be great.
In order to fail the package if your observed rowcount differs from your expected rowcount:
Create a Package Global Variable to hold your expected rowcount. This could be derived from a RowCount step in your DFT, set manually, etc.
Edit your Execute SQL Task that provides the observed rowcount, and set the Result Set to Single Row.
In the Result Set tab of your Execute SQL Task, assign this Result Set to a variable.
Edit your constraint condition prior to your final step. Set the Evaluation Operation to Expression and Constraint. Set the Value to Failure. In your expression, evaluate ResultSetVariable <> ExpectedRowCountVariable.
If the observed rowcount does not equal the expected rowcount, the package will fail.
You can raise an error and roll back the transaction in an SSIS "Execute SQL" task with the following SQL:
Raiserror('Something went wrong',16,1)
This will cause the "Execute SQL" task to return the error to the SSIS package and the task will follow the "red" (failure) path. You can then use this path to rollback the transaction and do any tidying-up.
This approach has the advantage you can do all your processing in the Execute SQL task then call Raiserror if you need to.