I have over 2 dozen tasks in our Snowflake database, all having the names in a similar pattern ending with a number (example : TSK_x, where x = 1,2,...,27).
I am trying to trying to write a procedure or anonymous block in Snowflake (without using Javascript stored proc) to generate a descending order task number statements and execute them from inside the procedure like :
ALTER TASK TSK_27 RESUME;
ALTER TASK TSK_26 RESUME;
...
ALTER TASK TSK_1 RESUME;
The task (TSK_1) is the parent task and needs to be enabled last.
As a background, that script will be included in Jenkins as part of our build. Our Jenkins does not allow multiple SQL statements in one file and so I am thinking of a stored proc like the one mentioned above.
Any help/suggestion will be much appreciated. I am new to Snowflake.
Query to "generate a descending order task number statements"
First execute -
show tasks;
created_on
name
state
2022-06-02 12:53:23.662 -0700
T1
started
2022-06-13 20:11:11.032 -0700
TASK_1
started
2022-06-13 20:24:20.211 -0700
TASK_10
started
2022-06-13 20:11:17.883 -0700
TASK_2
started
2022-06-13 20:24:10.871 -0700
TASK_2A
suspended
2022-06-13 20:11:22.769 -0700
TASK_3
started
2022-06-13 20:11:26.497 -0700
TASK_4
started
2022-06-13 20:11:30.725 -0700
TASK_5
started
2022-06-13 20:11:34.765 -0700
TASK_6
started
2022-06-13 20:11:38.313 -0700
TASK_7
started
Query (change order clause as needed - add desc in end) -
select "name" as name,"state" from table(result_scan(LAST_QUERY_ID()))
where regexp_like("name",'TASK_[[:digit:]]+$')
order by substr("name",1,4), to_number(substr("name",6));
NAME
state
TASK_1
started
TASK_2
started
TASK_3
started
TASK_4
started
TASK_5
started
TASK_6
started
TASK_7
started
TASK_10
started
Anonymous procedure to set tasks to resume -
show tasks;
EXECUTE IMMEDIATE $$
DECLARE
p_tsk string;
c1 CURSOR FOR select "name" as name from table(result_scan(LAST_QUERY_ID())) where regexp_like("name",'TASK_[[:digit:]]+$') order by substr("name",1,4), to_number(substr("name",6)) desc;
BEGIN
for record in c1 do
p_tsk:=record.name;
execute immediate 'alter task '||:p_tsk ||' suspend';
end for;
RETURN p_tsk;
END;
$$
;
To recursively resume all dependent tasks tied to a root task in a simple tree of tasks, query the SYSTEM$TASK_DEPENDENTS_ENABLE function rather than enabling each task individually (using ALTER TASK … RESUME).
Example:
select system$task_dependents_enable('mydb.myschema.mytask');
Related
The task should run at 2:27am UTC, but it did not executed.
GRANT EXECUTE TASK ON ACCOUNT TO ROLE SYSADMIN;
CREATE or replace TASK TASK_DELETE3
WAREHOUSE = TEST
SCHEDULE = 'USING CRON 27 2 * * * UTC' as
CREATE OR REPLACE TABLE TEST2."PUBLIC"."DELETE"
CLONE TEST1."PUBLIC"."DELETE";
ALTER TASK TASK_DELETE3 RESUME;
The task [state] = started. Does anyone know why?
If the status shows that the task is started, that means it is enabled and will run in the scheduled times.
You can check the task history to see the previous runs and next run of the task using the following query:
select *
from table(information_schema.task_history(
task_name=>'TASK_DELETE3'));
I was using a different role when I was checking the table in the database. The Task successfully completed at scheduled time.
We are using Control-M to submit several batch jobs to a legacy application, but due to limitations, the only way to monitor their status is by querying the Process table of the DB.
Process table:
JobNum
JobStat
Batch
1
Finished
ABC
2
Failed
ABC
3
Started
ABC
4
Started
ABC
I am trying use Cyclic Database job to query the Jobs, and rerun ever 5min while there are still jobs in Started, but have it break when:
The query result is empty (No jobs exist for that batch) - set to Not OK
All jobs are in either Finished or Failed in the batch - set to OK
Currently, I am trying to do something like:
SELECT 'TotalJobs', COUNT(JobNum)
FROM Process
WHERE Batch = 'ABC'
SELECT 'StartedJobs', COUNT(JobNum)
FROM Process
WHERE Batch = 'ABC'
AND JobStat = 'Started'
SELECT 'CompletedJobs', COUNT(JobNum)
FROM Process
WHERE Batch = 'ABC'
AND JobStat IN ('Finished', 'Failed')
Then using On-Do Actions with Specific statements like -
Statement: *
Code: TotalJob,0
Set-NotOK
Statement: *
Code: StartedJobs,0
Set-OK
But it does both actions...
Is this possible to do this more complex analysis with On-Do Actions?
Thanks
Looks like it should work, could you share a screenshot of the output (sysout) and the job log when it matches both scenarios (this will show what it has matched). Possibly the output is including more than intended.
I created the task below and am having trouble getting it to execute all lines. It looks like it just does the first delete from productweekly_upload then completes. Anyone have any ideas? This is my first time using tasks
CREATE OR REPLACE TASK WeeklySymphony_Load
WAREHOUSE = UPLOADWAREHOUSE
SCHEDULE = 'USING CRON 10 8 * * MON America/New_York'
as
--run every monday at 8:10 am
delete from Productweekly_Upload;
delete from Factsweekly_Upload;
delete from Productweekly;
delete from Factsweekly;
copy into ProductWeekly_Upload
from #symphony_s3_stage/prasco_phast_it_prdct_wk_;
copy into FactsWeekly_Upload
from #symphony_s3_stage/prasco_phast_it_wk_;
insert into ProductWeekly
select * from ProductWeekly_Upload;
insert into FactsWeekly
select * from FactsWeekly_Upload;
You can only execute 1 command in a TASK. If you want to create multiple steps, you can either wrap these into a stored procedure and call the SP from the TASK, or you can create each step as a TASK and make those dependencies, so they execute in order.
I recommend a read-through of this document:
https://docs.snowflake.com/en/user-guide/tasks-intro.html
If you want multiple statement, the 3 solutions are :
Stored proc
Chain multiple tasking AFTER statement
Use Procedural logic with AS DECLARE // BEGIN // END; block
https://docs.snowflake.com/en/sql-reference/sql/create-task.html#procedural-logic-using-snowflake-scripting
I have a task scheduled to run every 15 minutes:
CREATE OR REPLACE TASK mytask
WAREHOUSE = 'SHARED_WH_MEDIUM'
SCHEDULE = '15 MINUTE'
STATEMENT_TIMEOUT_IN_SECONDS = 3600,
QUERY_TAG = 'KLIPFOLIO'
AS
CREATE OR REPLACE TABLE mytable AS
SELECT * from xxx;
;
alter task mytask resume;
I see from the output of task_history() that the task is SCHEDULED:
select * from table(aftonbladet.information_schema.task_history(task_name => 'MYTASK')) order by scheduled_time;
QUERY_ID NAME DATABASE_NAME SCHEMA_NAME QUERY_TEXT CONDITION_TEXT STATE ERROR_CODE ERROR_MESSAGE SCHEDULED_TIME COMPLETED_TIME RETURN_VALUE
*** MYTASK *** *** *** SCHEDULED 2020-01-21 09:58:12.434 +0100
but I want it to run right now without waiting for the SCHEDULED_TIME , is there any way to accomplish that?
Snowflake now supports running tasks manually. Just use the EXECUTE TASK command:
EXECUTE TASK manually triggers an asynchronous single run of a scheduled task (either a standalone task or the root task in a task tree) independent of the schedule defined for the task. A successful run of a root task triggers a cascading run of child tasks in the tree as their precedent task completes, as though the root task had run on its defined schedule.
Also, there is no need for the task in started mode. Even tasks in suspended mode can be executed manually.
There is no way currently to execute a task manually. You could, however, alter the task schedule to 1 minute, let it run, and then alter it back to 15 minutes, so that you're not waiting the full 15 minutes. I have seen this request multiple times, and there is an Idea on Lodge (https://community.snowflake.com/s/ideas) that you should upvote (search for 'Tasks' and I think it'll be one of the top ideas). Since Tasks are still in Public Preview, it's likely that these types of ideas will be reviewed and prioritized if they have a lot of votes.
To build on Mike's answer:
You could have a task that executes every minute, but only if there's data on the stream!
For this you can create a table and stream just to decide if the task will be triggered every minute or not.
This root task should delete the data inserted in the stream to prevent the task running again.
So then you can have dependent tasks that execute every time you bring data into the stream, but only when the stream has new data.
This relies on the ability to run a task only when SYSTEM$STREAM_HAS_DATA()
-- stream so this task executes every minute, but only if there's new data
create table just_timestamps_stream_table(value varchar);
create stream just_timestamps_stream on table just_timestamps_stream_table;
-- https://docs.snowflake.com/en/user-guide/tasks-intro.html
create or replace task mytask_minute
warehouse = test_small
schedule = '1 MINUTE'
when SYSTEM$STREAM_HAS_DATA('just_timestamps_stream')
as
-- consume stream so tasks doesn't execute again
delete from just_timestamps_stream_table;
-- the real task to be executed
create or replace task mytask_minute_child1
warehouse = test_small
after mytask_minute
as
insert into just_timestamps values(current_timestamp, 'child1');
Full example:
https://github.com/fhoffa/snowflake_snippets/blob/main/stream_and_tasks/minimal.sql
I am trying to use the oracle SQL access advisor utility.
For recommendation on performance tuning and partitioning of tables.
But when i try to use it gives me no results and says there are no result for the task.
I am trying below code to generate the recommendation this is an example from SCOTT schema which also giving me same error
declare
v_sql varchar2(2000) := 'select * from emp where empno in (7369,7499)';
v_tuning_task varchar2(200) := 'tune_task_advisor_view7';
v_tune_result clob;
begin
dbms_advisor.quick_tune ( dbms_advisor.sqlaccess_advisor , v_tuning_task, v_sql );
DBMS_ADVISOR.reset_task(v_tuning_task);
dbms_advisor.set_task_parameter(v_tuning_task, 'ANALYSIS_SCOPE', 'ALL');
dbms_advisor.set_task_parameter(v_tuning_task, 'STORAGE_CHANGE', '10000000');
dbms_advisor.set_task_parameter ( v_tuning_task, 'MODE', 'COMPREHENSIVE');
dbms_output.put_line ('Quick Tune Completed');
end;
SELECT DBMS_ADVISOR.get_task_script ('tune_task_advisor_view7') AS script FROM dual;
The Error --
SELECT DBMS_ADVISOR.get_task_script ('tune_task_advisor_view7') AS script FROM dual
Error report -
SQL Error: ORA-13631: The most recent execution of task tune_task_advisor_view7 contains no results.
ORA-06512: at "SYS.PRVT_ADVISOR", line 3350
ORA-06512: at "SYS.DBMS_ADVISOR", line 641
ORA-06512: at line 1
13631. 00000 - "The most recent execution of task %s contains no results."
*Cause: The user attempted to create a report or script from a task that
has not successfully completed an execution.
*Action: Execute the task and then retry the operation
There is one more that when i comment all the SET_TASK_PARAMETER procedure then it run but does not provide any recommendations the output is like --
SCRIPT
--------------------------------------------------------------------------------
Rem SQL Access Advisor: Version 11.2.0.3.0 - Production
Rem
Rem Username:
Is there any parameter which i missed to define here.
For RESET_TASK if i do not rest_task then it gives me error for it also so i have used it here.
Thanks in advance
When I get this error, this typically means that the DBMS_ADVISOR does not have any advise for the profile based on its diagnostic so it doesn't return a report. I typically don't try to produce a ADDM advisor report, but rather look for all the HASH plans I can find for the query, run an explain plan on those hash values, and then profile the one with the best plan.
I forgot to execute the task !
exec dbms_advisor.execute_task('tune_task_advisor_view7');
and then
SELECT DBMS_ADVISOR.get_task_script ('tune_task_advisor_view7') AS script FROM
dual;
OUTPUT will be
SCRIPT
--------------------------------------------------------------------------------
Rem SQL Access Advisor: Version 12.1.0.2.0 - Production
Rem
Rem Username: CHRIS
Rem Task: tune_task_advisor_view7
Rem Execution date:
Rem
/* RETAIN INDEX "SCOTT"."PK_EMP" */
Find address and hash for the query SQL_ID
select address, hash_value, plan_hash_value from v$sqlarea where sql_id = '3mx8whn1c5jbb';
Purge the plan out of the cursor cache (only if it’s determined that it’s the wrong plan). Statistics should be up to date prior to purging out the plan.
exec sys.dbms_shared_pool.purge(',','C');
exec sys.dbms_shared_pool.purge('00000005DF37F740,46318955','C');
Create the tuning plan if one does not exist.
variable stmt_task VARCHAR2(64);
SET SERVEROUTPUT ON LINESIZE 200 PAGESIZE 20000 LONG 9999
EXEC :stmt_task := DBMS_SQLTUNE.CREATE_TUNING_TASK(sql_id => '3mx8whn1c5jbb',task_name => '3mx8whn1c5jbb_AWR_tuning_task');
SELECT DBMS_SQLTUNE.REPORT_TUNING_TASK('3mx8whn1c5jbb_AWR_tuning_task', 'TEXT', 'TYPICAL', 'FINDINGS') FROM DUAL;
If the tuning plan is reported. Accept the new plan.
DECLARE
sqlprofile_name VARCHAR2(30);
BEGIN
sqlprofile_name := DBMS_SQLTUNE.ACCEPT_SQL_PROFILE (
task_name => '3mx8whn1c5jbb_AWR_tuning_task'
, name => 'sql_profile_1'
, force_match => true
);
END;
/
If creating a profile doesn’t work or if the plan does not exist in the cursor cache, check the AWR repository to see if a better plan exists.
If you have an AWR repository with the correct plan, you can rerun the queries above and just use the option to get the hash and sql_id from the AWR repository.
If that doesn't work. THEN I used ADDM (As you did above).