I have created a stored procedure that returns a create table sql statement; I want to be able to now call that procedure and assign the result to a variable like:
set create_table_statement = call sp_create_stage_table(target_db, table_name);
snowflake will not let me do this, so is there a way I can.
Context
We have just been handed over our new MDP which is built on AWS-S3, DBT & Snowflake, next week we go into production but we have 200+ tables and snowlpipes to code out. I wanted to semi automate this by generating the create table statements based off the tables metadata and then calling the results from that to create the tables. At the moment we're having to run the SQL, copy+paste the results in and then run that, which is fine in dev/pre-production mode when it's a handful of tables. but with just 2 of us it will be a lot of work to get all those tables and pipes created.
so I've found a work around, by creating a second procedure and calling the first one as a se=ql string to get the results as a string - then calling that string as a sql statement. like:
create or replace procedure sp_create_stage_table("db_name" string, "table_name" string)
returns string
language javascript
as
$$
var sql_string = "call sp_get_create_table_statement('" + db_name + "','" + table_name + "');";
var get_sql_query = snowflake.createStatement({sqlText: sql_string});
var get_result_set = get_sql_query.execute();
get_result_set.next();
var get_query_value = get_result_set.getColumnValue(1);
sql_string = get_query_value.toString();
try {
var main_sql_query = snowflake.createStatement({sqlText: sql_string});
main_sql_query.execute();
return "Stage Table " + table_name + " Successfully created in " + db_name + " database."
}
catch (err){
return "an error occured! \n error_code: " + err.code + "\n error_state: " + err.state + "\n error_message: " + err.message;
}
$$;
It is possible to assign scalar result of stored procedure to session variable. Instead:
SET var = CALL sp();
The pattern is:
SET var = (SELECT * FROM TABLE(RESULT_SCAN(LAST_QUERY_ID())));
Sample:
CREATE OR REPLACE PROCEDURE TEST()
RETURNS VARCHAR
LANGUAGE SQL
AS
BEGIN
RETURN 'Result from stored procedrue';
END;
CALL TEST();
SET variable = (SELECT * FROM TABLE(RESULT_SCAN(LAST_QUERY_ID())));
SELECT $variable;
-- Result from stored procedrue
Related
It looks like Snowflake doesn't process parameter binding for SHOW USER statements in JS like this.
var sql_cmd = "SHOW USERS LIKE ?;";
var username = "user.name"
var stmt = snowflake.createStatement({sqlText: sql_cmd, binds:[username]});
var users = stmt.execute();
It just gives me an error saying that
SQL compilation error: syntax error line 1 at position 16 unexpected '?'. At Statement.execute, line 14 position 18
How do I make it work?
Is there a more accurate docs on what is supported by the binds feature? I feel like it should support all SQLs but looks like it doesn't work on CREATE either on another thread I found here.
Can you try this as an alternative? Note to use the SHOW USERS command you must execute the proc as a CALLER: https://docs.snowflake.com/en/sql-reference/stored-procedures-rights.html#caller-s-rights-stored-procedures
CREATE OR REPLACE PROCEDURE user_bind(username VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
var sql_command = "SHOW USERS LIKE '" + USERNAME + "'";
var stmt = snowflake.createStatement( {sqlText: sql_command} );
var result1 = stmt.execute();
result1.next();
return result1.getColumnValue(1);
$$
;
alternatively you dont need a parameter and you could just use the variable within the SP
var username= 'USER1';
var sql_command = "SHOW USERS LIKE '" + username + "'";
call the stored proc
CALL user_bind ('USER1'); --or
CALL user_bind ();
I am trying to write a Snowflake UDF that accepts a stage name and specific folder name as input parameters and returns the latest file id ( striping from full file name) as the output. Could anyone help me with a simple code to achieve this?
I'm not sure if you want a UDF or stored procedure. The syntax to create would be similar so I think this can help. Here is a stored procedure which will fetch latest staged file from a given stage and path. Just be aware of the limit 1 in query, multiple staged files may share the same last modified date while this procedure returns a scalar (single) value.
Stored Procedure Definition
create or replace procedure "MYDB"."MYSCHEMA"."LATEST_STAGED_FILE"(stage_name text, folder text)
returns string not null
language javascript
execute as caller
as
$$
var sql_text = "list #" + STAGE_NAME + "/" + FOLDER ;
var sql_command0 = snowflake.createStatement({ sqlText: sql_text});
var sql_command1 = snowflake.createStatement({ sqlText:`SELECT "name" FROM table(result_scan(last_query_id())) WHERE "last_modified" = (select MAX("last_modified") from table(result_scan(last_query_id()))) LIMIT 1;`});
try {
sql_command0.execute();
var resultSet = sql_command1.execute();
while(resultSet.next())
{
var resultFile = resultSet.getColumnValue('name').split("/")
return resultFile[resultFile.length - 1]
}
}
catch (err) {
return "Failed: " + err;
}
$$;
You can then call the stored procedure like
call "MYDB"."MYSCHEMA"."LATEST_STAGED_FILE"('MYDB.MYSCHEMA.MYSTAGE', 'mypath/myotherpath');
References
select from list #
list stage via SP
How to schedule a sql script in the snowflake database to run every day, and set the output file name to include the current date. E.g. if the code ran today then the file name should be 20200906*****.csv.gz, similary for tomorrow 20200907******.csv.gz.
You could use Snowflake TASKS in order to schedule execution of SQL statements.
Task can execute a single SQL statement, including a call to a stored procedure.
Tasks run according to a specified execution configuration, using any combination of a set interval and/or a flexible schedule using a subset of familiar cron utility syntax.
For your goal I would create a Stored Procedure (so that you could use variables for managing changing filename and for any more complex things).
SF Doc: https://docs.snowflake.com/en/sql-reference/sql/create-task.html
--create a new task that executes a single SQL statement based on CRON definition
CREATE TASK mytask_hour
WAREHOUSE = mywh
SCHEDULE = 'USING CRON 0 9-17 * * SUN America/Los_Angeles'
TIMESTAMP_INPUT_FORMAT = 'YYYY-MM-DD HH24'
AS
INSERT INTO mytable(ts) VALUES(CURRENT_TIMESTAMP);
--create a new task that executes a Stored Procedure every hour
create task my_copy_task
warehouse = mywh
schedule = '60 minute'
as
call my_unload_sp();
After creating a task, you must execute ALTER TASK … RESUME in order to enable it.
Use SHOW TASKS to check your task's definition/configuration and then query TASK_HISTORY in order to check executions.
Your Snowflake JS Stored Procedure could be something like this:
create or replace procedure SP_TASK_EXPORT()
RETURNS VARCHAR(256) NOT NULL
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
as $$
function getToday_yyyymmdd()
{
var v_out_Today;
rs = snowflake.execute ( { sqlText: `SELECT to_char(current_date,'yyyymmdd');` } );
if( rs.next())
{
v_out_Today = rs.getColumnValue(1); // get current date as yyyymmdd
}
return v_out_Today;
}
var result = new String('Successfully Executed');
var v_Today = getToday_yyyymmdd();
try {
var sql_command = `copy into #unload_gcs/LH_TBL_FIRST` + v_Today + `.csv.gz from ........`;
var stmt = snowflake.createStatement({sqlText: sql_command});
var res = stmt.execute();
}
catch (err) {
result = "Failed: Code: " + err.code + " | State: " + err.state;
result += "\n Message: " + err.message;
result += "\nStack Trace:\n" + err.stackTraceTxt;
}
return result;
$$;
Before creating your task and schedule it, test your Stored Procedure invoking it:
call SP_TASK_EXPORT();
I am trying to assign a simple sql query result in stored procedure to a variable which I will use it later in another sql statement. If I execute below statement with out putting in procedure it works but not with stored procedure, I get an error while calling sp , Can someone please help me here ?
Procedure code:
$$
VAR NAME = 'ABC'
SET (COUNT_VALUE) = (SELECT COUNT(*) FROM COUNT_TABLE)
With out Procedure below code works
SET (COUNT_VALUE) = (SELECT COUNT(*) FROM COUNT_TABLE)
select $COUNT_VALUE
Thanks
Here's an example that uses a simple helper function to return a result set from a query. You can read the Snowflake docs to see what the result set API has in it. The getResultSet function returns a variable that has the results of the query. Since the SQL is a count, you need to use rs.next() once to get to the first row and then read the value in the aliased count(*). You could also use rs.getColumnValue(1) to get a column by ordinal position, but I recommend using column names with SQL aliases is necessary.
create or replace procedure COUNT_EXAMPLE()
returns string
language javascript
as
$$
var rowCount = 0;
var sql = "select count(*) as ROW_COUNT from SNOWFLAKE_SAMPLE_DATA.TPCH_SF10000.ORDERS;";
try {
var rs = getResultSet(sql);
if (rs.next()) {
var rowCount = rs.getColumnValue("ROW_COUNT");
} else {
return "Error: Count query failed.";
}
}
catch(err) {
return "Error: " + err.message;
}
return "The table has " + rowCount + " rows.";
//--------------------------- End of main function ---------------------------
function getResultSet(sql){
cmd1 = {sqlText: sql};
stmt = snowflake.createStatement(cmd1);
var rs;
rs = stmt.execute();
return rs;
}
$$;
call COUNT_EXAMPLE();
I need help with a practical scenario. I have a table called CONFIG_TBL in Snowflake. This table has SQL statements, 1 per row. My goal is to use Snowflake Stored Procedures or Snowflake UDF, or a combination of the two, and return a result set that will be obtained after execution of that statment.
The statements are simple select statements, like "select * from ABC'.
I could have done this, very easily in SQL server, since procedures can return table values. However, I don't know how to do this in Snowflake.
Any help will be greatly appreciated.
Thanks in advance.
Here's something to get you started at least. Procedures use javascript (SQL Stored Procedures are coming soon), but they can be used to run dynamic queries like you are looking for.
You can get the results in a couple of ways. By either returning a variant object or by using result_scan after calling the procedure.
This example just runs one query so your final solution will be different depending on just what you want the output to look like.
CREATE OR REPLACE PROCEDURE SCHEMA.PROCEDURE_NAME()
RETURNS VARIANT
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
retrieve_queries_sql = "select top 1 query from CONFIG_TBL";
retrieve_queries_result_set = snowflake.execute({sqlText: retrieve_queries_sql });
query_to_run = retrieve_queries_result_set.next().getColumnValue(1);
rs = snowflake.execute({sqlText: query_to_run})
var return_value = "";
if (rs.next()) {
return_value += rs.getColumnValue(1);
return_value += ", " + rs.getColumnValue(2);
}
while (rs.next()) {
return_value += "\n";
return_value += rs.getColumnValue(1);
return_value += ", " + rs.getColumnValue(2);
}
}
return return_value;
$$
CALL SCHEMA.PROCEDURE_NAME()
SELECT *
FROM table(result_scan(last_query_id()))
Edit: Fixed to have example correctly return a result which can then be used by the result_scan. Example taken from here. There are various more examples for getting results out of a procedure, including using JSON output.
We can use language as SQL and return the value to table, Which can return table values.
CREATE OR REPLACE PROCEDURE DDL_TEST_SQL()
RETURNS TABLE(CARRIERFILEID integer, BLOBPATH varchar)
LANGUAGE SQL
EXECUTE AS OWNER
AS
$$
DECLARE
QUERY STRING;
res resultset;
BEGIN
QUERY := 'SELECT TOP 10 CARRIERFILEID,BLOBPATH from CONFIG.CARRIERFILE';
res := (EXECUTE IMMEDIATE :QUERY);
return table(res);
END;
$$;
CALL DDL_TEST_SQL()