I have created a task:
CREATE OR REPLACE TASK TASK_1
WAREHOUSE = WAREHOUSE
SCHEDULE = 'USING CRON 30 1 * * * America/Detroit'
AS
....
This runs at 1:30 am daily.
Is there a way to execute this query on demand?
i.e. something like:
TRIGGER TASK TASK_1;
mRainey's answer is right. You can't schedule a task outside of a schedule or task dependency. That's the correct answer to the OP's question.
For others though who stumble upon this answer, you can make scheduling a task at a different time easier on yourself:
CREATE OR REPLACE PROCEDURE "SCHEDULE_TASK_AT_TIME"(TASK_NAME VARCHAR, HOUR float, MINUTE float)
RETURNS VARIANT
LANGUAGE JAVASCRIPT
AS $$
var return_rows = [];
var task_name = TASK_NAME;
var h = HOUR;
var m = MINUTE;
var default_timezone = 'America/Los_Angeles';
var new_chron = 'USING CRON ' + m + ' ' + h + ' * * * ' + default_timezone;
var stmt = snowflake.createStatement({sqlText: `
DESCRIBE TASK IDENTIFIER(:1)
`, binds:[task_name]});
res = stmt.execute();
res.next();
var old_chron = res.getColumnValue(8);
var stmt = snowflake.createStatement({sqlText: `
ALTER TASK IDENTIFIER(:1) SUSPEND
`, binds:[task_name]});
res = stmt.execute();
var stmt = snowflake.createStatement({sqlText: `
ALTER TASK IDENTIFIER(:1) SET SCHEDULE = :2
`, binds:[task_name, new_chron]});
res = stmt.execute();
var stmt = snowflake.createStatement({sqlText: `
ALTER TASK IDENTIFIER(:1) RESUME
`, binds:[task_name]});
res = stmt.execute();
return_rows.push('Old Chron: ' + old_chron);
return_rows.push('New Chron: ' + new_chron);
return return_rows;
$$;
Then you can schedule your task like this, which would run at the next 22:38:
call SCHEDULE_TASK_AT_TIME('DEMO_TASK', 22, 38);
The output of this procedure gives you the old chron time and the new chron time so you can easily set it back when you're done.
Just be sure to be careful w/ this and notice its limitations - in my version you have to hard-code the timezone for example.
Also, I didn't look into whether you can set up a chron to execute just once, so whatever hour and minute you set it to, it'll run like this every day unless you take further action.
Currently, there’s no way to explicitly execute a task outside of either a schedule or task dependency.
Related
I'm trying to come up with a SP that creates a certain task, appended by year, for a generic approach.
I can create the tasks outside, alone, with the $$ marks,
but I can't do it inside JS SP like this:
CREATE OR REPLACE PROCEDURE create_exec_tasks_by_year_range()
RETURNS varchar
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS
$$
var return_value = "";
var range_years = Array.from(Array(new Date().getUTCFullYear() - 2006), (_, i) => (i + 2007).toString());
//CREATE a task TO CALL SP BY YEAR, FROM 2007-current year
range_years.forEach((year_elem) => {
rs = snowflake.createStatement( {
sqlText: `CREATE OR REPLACE TASK MY_TSK_YEAR_`+year_elem+`
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
$$
DECLARE
year_track float;
rs resultset;
BEGIN
year_track := :1;
rs := (execute IMMEDIATE 'INSERT INTO MY_TABLE VALUES(?)' using (year_track));
return TABLE(rs);
END;
$$
;`
, binds: [year_elem] }).execute();
rs.next();
//rs.getColumnValue(1);
return_value += "MY_TSK_YEAR_"+year_elem+", ";
to_exec = snowflake.createStatement( {
sqlText: `EXECUTE TASK MY_TSK_YEAR_`+year_elem+`
to_exec.next();
return_value += to_exec.getColumnValue(1)+", ";
});
return return_value;
$$;
because it throws me
syntax error line ...at position 2 unexpected 'DECLARE'
while creating the TASK manually, works, because I don't have a conflict between $$?
CREATE OR REPLACE TASK SHARED.SRC_EXT_WEATHER.TSK_DUMMY
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
$$
DECLARE
year_track float;
rs resultset;
BEGIN
year_track := 2007;
rs := (execute IMMEDIATE 'INSERT INTO MY_TABLE VALUES(?)' using (year_track));
return TABLE(rs);
END;
$$;
Is it possible to make the SP to work for the TASK created with EXECUTE_IMMEDIATE block and bind parameter? The problem seems to be the way I write in inside the $$ scope of the stored procedure, no?
The problem is you're using the $$ string terminator for the stored procedure. Then inside the body of the stored procedure you're using $$ to terminate the string containing the body of the task. As soon as the compiler runs into that, it sees it as the end of the string defining the stored procedure body. To use $$ in the body of a stored procedure, you can do something like this:
CREATE OR REPLACE PROCEDURE create_exec_tasks_by_year_range()
RETURNS varchar
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS
$$
const double_dollar = '$' + '$';
var return_value = "";
var to_exec;
var rs_exec;
var range_years = Array.from(Array(new Date().getUTCFullYear() - 2006), (_, i) => (i + 2007).toString());
//CREATE a task TO CALL SP BY YEAR, FROM 2007-current year
range_years.forEach((year_elem) => {
rs = snowflake.createStatement( {
sqlText: `CREATE OR REPLACE TASK MY_TSK_YEAR_${year_elem}
WAREHOUSE = MOS_WEATHER
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
${double_dollar}
DECLARE
year_track float;
rs resultset;
BEGIN
year_track := :1;
rs := (execute IMMEDIATE 'INSERT INTO MY_TABLE VALUES(?)' using (year_track));
return TABLE(rs);
END;
${double_dollar};`
, binds: [year_elem] }).execute();
rs.next();
//rs.getColumnValue(1);
return_value += "MY_TSK_YEAR_"+year_elem+", ";
to_exec = snowflake.createStatement({sqlText: `EXECUTE TASK MY_TSK_YEAR_${year_elem}`});
rs_exec = to_exec.execute();
rs_exec.next();
return_value += rs_exec.getColumnValue(1)+", ";
});
return return_value;
$$;
#Greg Pavlik I don't know my final approach is OK, but it's working, as far as I run this on SF UI!
The above code, for me worked with a minor change from :1 to ${year_elem} , but then I face issue with binding a string value (single quoted) ... like this:
CREATE OR REPLACE PROCEDURE create_exec_tasks_by_year_range(str_arg string)
RETURNS varchar
LANGUAGE JAVASCRIPT
COMMENT ='[DRAFT] SP that creates tasks since a certain rangw with exec immediate block using 2 args: one is year in range, other is a string to isnert as well'
EXECUTE AS OWNER
AS
$$
const double_dollar = '$' + '$';
var return_value = "";
var to_exec;
var rs_exec;
var arg = STR_ARG;
var range_years = Array.from(Array(new Date().getUTCFullYear() - 2020), (_, i) => (i + 2021).toString());
//CREATE a task TO CALL SP BY YEAR, FROM 2007-current year
range_years.forEach((year_elem) => {
rs = snowflake.createStatement( {
sqlText: `CREATE OR REPLACE TASK MY_TSK_YEAR_${year_elem}
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
${double_dollar}
DECLARE
year_track float;
rs resultset;
my_str varchar;
BEGIN
year_track := ${year_elem};
my_str:= ${arg} ;
rs := (execute IMMEDIATE 'INSERT INTO Z_TEST_TASK_INSERT2 VALUES(?,?)' using (year_track,my_str));
return TABLE(rs);
END;
${double_dollar};`
//, binds: [year_elem]
}).execute();
rs.next();
return_value += rs.getColumnValue(1);
});
return return_value;
$$;
CALL create_exec_tasks_by_year_range('testing_String');
EXECUTE TASK MY_TSK_YEAR_2021;
EXECUTE TASK MY_TSK_YEAR_2022;
000904 SQL compilation error: error line 8 at position 21
invalid identifier 'testing_String'
One thing I would like to understand is:
How to use bind variable as a single quoted string? How to escape it or at leats distinguish it from the query prefixed by ' alread?
The SP creation succeeds via SQL Editor (DBEaver for instance) or SF UI
But the CALL is only working for me on SF UI (and SNOWSQL must happen the same, I assume). Is that something to do with the drivers used by JDBC vs. SF UI?
I have created a stored procedure that returns a create table sql statement; I want to be able to now call that procedure and assign the result to a variable like:
set create_table_statement = call sp_create_stage_table(target_db, table_name);
snowflake will not let me do this, so is there a way I can.
Context
We have just been handed over our new MDP which is built on AWS-S3, DBT & Snowflake, next week we go into production but we have 200+ tables and snowlpipes to code out. I wanted to semi automate this by generating the create table statements based off the tables metadata and then calling the results from that to create the tables. At the moment we're having to run the SQL, copy+paste the results in and then run that, which is fine in dev/pre-production mode when it's a handful of tables. but with just 2 of us it will be a lot of work to get all those tables and pipes created.
so I've found a work around, by creating a second procedure and calling the first one as a se=ql string to get the results as a string - then calling that string as a sql statement. like:
create or replace procedure sp_create_stage_table("db_name" string, "table_name" string)
returns string
language javascript
as
$$
var sql_string = "call sp_get_create_table_statement('" + db_name + "','" + table_name + "');";
var get_sql_query = snowflake.createStatement({sqlText: sql_string});
var get_result_set = get_sql_query.execute();
get_result_set.next();
var get_query_value = get_result_set.getColumnValue(1);
sql_string = get_query_value.toString();
try {
var main_sql_query = snowflake.createStatement({sqlText: sql_string});
main_sql_query.execute();
return "Stage Table " + table_name + " Successfully created in " + db_name + " database."
}
catch (err){
return "an error occured! \n error_code: " + err.code + "\n error_state: " + err.state + "\n error_message: " + err.message;
}
$$;
It is possible to assign scalar result of stored procedure to session variable. Instead:
SET var = CALL sp();
The pattern is:
SET var = (SELECT * FROM TABLE(RESULT_SCAN(LAST_QUERY_ID())));
Sample:
CREATE OR REPLACE PROCEDURE TEST()
RETURNS VARCHAR
LANGUAGE SQL
AS
BEGIN
RETURN 'Result from stored procedrue';
END;
CALL TEST();
SET variable = (SELECT * FROM TABLE(RESULT_SCAN(LAST_QUERY_ID())));
SELECT $variable;
-- Result from stored procedrue
It looks like Snowflake doesn't process parameter binding for SHOW USER statements in JS like this.
var sql_cmd = "SHOW USERS LIKE ?;";
var username = "user.name"
var stmt = snowflake.createStatement({sqlText: sql_cmd, binds:[username]});
var users = stmt.execute();
It just gives me an error saying that
SQL compilation error: syntax error line 1 at position 16 unexpected '?'. At Statement.execute, line 14 position 18
How do I make it work?
Is there a more accurate docs on what is supported by the binds feature? I feel like it should support all SQLs but looks like it doesn't work on CREATE either on another thread I found here.
Can you try this as an alternative? Note to use the SHOW USERS command you must execute the proc as a CALLER: https://docs.snowflake.com/en/sql-reference/stored-procedures-rights.html#caller-s-rights-stored-procedures
CREATE OR REPLACE PROCEDURE user_bind(username VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
var sql_command = "SHOW USERS LIKE '" + USERNAME + "'";
var stmt = snowflake.createStatement( {sqlText: sql_command} );
var result1 = stmt.execute();
result1.next();
return result1.getColumnValue(1);
$$
;
alternatively you dont need a parameter and you could just use the variable within the SP
var username= 'USER1';
var sql_command = "SHOW USERS LIKE '" + username + "'";
call the stored proc
CALL user_bind ('USER1'); --or
CALL user_bind ();
Trying to run Describe table and running RESULT_SCAN on the query id of the describe table query.
Procedure:
var qry = ` describe table TEST_TABLE `;
var qry_rslt = snowflake.execute({sqlText: qry});
var qry_id= qry_rslt.getQueryId();
var qry2 = ` select * from table(result_scan('`+qry_id+`')) `
snowflake.execute({sqlText: qry2});
The procedure is returning Null and not running the SQL. On manually running the result scan query it says statement not found.
ANy idea how to read describe result.
You're not actually reading the results of the second query. It's running it but not collecting the results. This will collect the first column only of the result set:
create or replace procedure test()
returns string
language javascript
as
$$
var qry = ` describe table TEST_TABLE `;
var qry_rslt = snowflake.execute({sqlText: qry});
var qry_id= qry_rslt.getQueryId();
var qry2 = ` select * from table(result_scan('${qry_id}')) `;
rs = snowflake.execute({sqlText: qry2});
var out = "";
var i = 0;
while (rs.next()) {
if (i++ > 0) out += ",";
out += rs.getColumnValue(1);
}
return out;
$$;
call test();
Are you looking to get the entire DDL in one statement? If so you can run get_ddl and then read just the first row, first column. It will have the DDL for the entire table. If you want it as a table, you'll need to read the rows and columns to do what needs to be done with them.
How to schedule a sql script in the snowflake database to run every day, and set the output file name to include the current date. E.g. if the code ran today then the file name should be 20200906*****.csv.gz, similary for tomorrow 20200907******.csv.gz.
You could use Snowflake TASKS in order to schedule execution of SQL statements.
Task can execute a single SQL statement, including a call to a stored procedure.
Tasks run according to a specified execution configuration, using any combination of a set interval and/or a flexible schedule using a subset of familiar cron utility syntax.
For your goal I would create a Stored Procedure (so that you could use variables for managing changing filename and for any more complex things).
SF Doc: https://docs.snowflake.com/en/sql-reference/sql/create-task.html
--create a new task that executes a single SQL statement based on CRON definition
CREATE TASK mytask_hour
WAREHOUSE = mywh
SCHEDULE = 'USING CRON 0 9-17 * * SUN America/Los_Angeles'
TIMESTAMP_INPUT_FORMAT = 'YYYY-MM-DD HH24'
AS
INSERT INTO mytable(ts) VALUES(CURRENT_TIMESTAMP);
--create a new task that executes a Stored Procedure every hour
create task my_copy_task
warehouse = mywh
schedule = '60 minute'
as
call my_unload_sp();
After creating a task, you must execute ALTER TASK … RESUME in order to enable it.
Use SHOW TASKS to check your task's definition/configuration and then query TASK_HISTORY in order to check executions.
Your Snowflake JS Stored Procedure could be something like this:
create or replace procedure SP_TASK_EXPORT()
RETURNS VARCHAR(256) NOT NULL
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
as $$
function getToday_yyyymmdd()
{
var v_out_Today;
rs = snowflake.execute ( { sqlText: `SELECT to_char(current_date,'yyyymmdd');` } );
if( rs.next())
{
v_out_Today = rs.getColumnValue(1); // get current date as yyyymmdd
}
return v_out_Today;
}
var result = new String('Successfully Executed');
var v_Today = getToday_yyyymmdd();
try {
var sql_command = `copy into #unload_gcs/LH_TBL_FIRST` + v_Today + `.csv.gz from ........`;
var stmt = snowflake.createStatement({sqlText: sql_command});
var res = stmt.execute();
}
catch (err) {
result = "Failed: Code: " + err.code + " | State: " + err.state;
result += "\n Message: " + err.message;
result += "\nStack Trace:\n" + err.stackTraceTxt;
}
return result;
$$;
Before creating your task and schedule it, test your Stored Procedure invoking it:
call SP_TASK_EXPORT();