We prefer coding Snowflake stored procedures and javascript UDF's using the $$ notation. It is easier as we do not have to worry about escaping every single quote in our code. However, when we retrieve the DDL using GET_DDL, Snowflake removes the $$ and places the body of the SP in single quotes and also escapes every single quote.
Is there a way to get the SP DDL from Snowflake in the $$ format?
Example, below is the SP we create. Notice the $$ sign and that we do not hae
CREATE OR REPLACE PROCEDURE "SP_TEST"()
RETURNS VARCHAR(16777216)
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
var result = snowflake.execute({sqlText: "select 'Hello World!' as test;"})
result.next();
return String(result.getColumnValue(1));
$$;
Then, when we retrieve the DDL from Snowflake using SELECT GET_DDL('PROCEDURE','SP_TEST()'), we get the following. Notice the $$ has been replaced by single quotes and that all other single quotes are now escaped.
CREATE OR REPLACE PROCEDURE "SP_TEST"()
RETURNS VARCHAR(16777216)
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS '
var result = snowflake.execute({sqlText: "select ''Hello World!'' as test;"})
result.next();
return String(result.getColumnValue(1));
';
Thanks
I've not found a built-in way to do it, but you can use a UDF to do this. This should work provided that the $$ start and end of the code section are always on lines by themselves:
create or replace function CODE_DDL_TO_TEXT(CODE_TEXT string)
returns string
language javascript
as
$$
var lines = CODE_TEXT.split("\n");
var out = "";
var startCode = new RegExp("^AS '$", "ig");
var endCode = new RegExp("^'\;$", "ig");
var inCode = false;
var isChange = false;
var s;
for (i = 0; i < lines.length; i++){
isChange = false;
if(!inCode) {
inCode = startCode.test(lines[i]);
if(inCode) {
isChange = true;
out += "AS $" + "$\n";
}
}
if (endCode.test(lines[i])){
out += "$" + "$;";
isChange = true;
inCode = false;
}
if(!isChange){
if(inCode){
s = lines[i].replace(/''/g, "'") + "\n";
s = s.replace(/\\\\/g, "\\");
out += s;
} else {
out += lines[i] + "\n";
}
}
}
return out;
$$;
select CODE_DDL_TO_TEXT(get_ddl('function', 'CODE_DDL_TO_TEXT(string)'));
execute immediate $$
DECLARE
PROCS RESULTSET;
ALL_PROCS RESULTSET;
DB_NAME STRING DEFAULT 'PROC_TEST';
TABLE_NAME STRING;
QUERY STRING;
ALL_QUERY STRING;
COUNT INT DEFAULT 0;
OUTPUT STRING DEFAULT '';
c1 CURSOR FOR select DATABASE_NAME from snowflake.information_schema.databases;
BEGIN
TABLE_NAME := CONCAT(DB_NAME, '.INFORMATION_SCHEMA.SCHEMATA');
OPEN c1;
FOR rec IN c1 DO
QUERY := 'SELECT PROCEDURE_CATALOG,PROCEDURE_NAME,PROCEDURE_OWNER,PROCEDURE_DEFINITION FROM ' || rec.DATABASE_NAME || '.INFORMATION_SCHEMA.procedures';
OUTPUT := OUTPUT || QUERY || '\n UNION ALL \n';
--PROCS := (EXECUTE IMMEDIATE :QUERY);
-- ALL_PROCS := ALL_PROCS + PROCS;
END FOR;
--ALL_PROCS := (EXECUTE IMMEDIATE :OUTPUT);
--RETURN table(ALL_PROCS);
RETURN OUTPUT;
END;
$$;
Related
I'm trying to come up with a SP that creates a certain task, appended by year, for a generic approach.
I can create the tasks outside, alone, with the $$ marks,
but I can't do it inside JS SP like this:
CREATE OR REPLACE PROCEDURE create_exec_tasks_by_year_range()
RETURNS varchar
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS
$$
var return_value = "";
var range_years = Array.from(Array(new Date().getUTCFullYear() - 2006), (_, i) => (i + 2007).toString());
//CREATE a task TO CALL SP BY YEAR, FROM 2007-current year
range_years.forEach((year_elem) => {
rs = snowflake.createStatement( {
sqlText: `CREATE OR REPLACE TASK MY_TSK_YEAR_`+year_elem+`
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
$$
DECLARE
year_track float;
rs resultset;
BEGIN
year_track := :1;
rs := (execute IMMEDIATE 'INSERT INTO MY_TABLE VALUES(?)' using (year_track));
return TABLE(rs);
END;
$$
;`
, binds: [year_elem] }).execute();
rs.next();
//rs.getColumnValue(1);
return_value += "MY_TSK_YEAR_"+year_elem+", ";
to_exec = snowflake.createStatement( {
sqlText: `EXECUTE TASK MY_TSK_YEAR_`+year_elem+`
to_exec.next();
return_value += to_exec.getColumnValue(1)+", ";
});
return return_value;
$$;
because it throws me
syntax error line ...at position 2 unexpected 'DECLARE'
while creating the TASK manually, works, because I don't have a conflict between $$?
CREATE OR REPLACE TASK SHARED.SRC_EXT_WEATHER.TSK_DUMMY
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
$$
DECLARE
year_track float;
rs resultset;
BEGIN
year_track := 2007;
rs := (execute IMMEDIATE 'INSERT INTO MY_TABLE VALUES(?)' using (year_track));
return TABLE(rs);
END;
$$;
Is it possible to make the SP to work for the TASK created with EXECUTE_IMMEDIATE block and bind parameter? The problem seems to be the way I write in inside the $$ scope of the stored procedure, no?
The problem is you're using the $$ string terminator for the stored procedure. Then inside the body of the stored procedure you're using $$ to terminate the string containing the body of the task. As soon as the compiler runs into that, it sees it as the end of the string defining the stored procedure body. To use $$ in the body of a stored procedure, you can do something like this:
CREATE OR REPLACE PROCEDURE create_exec_tasks_by_year_range()
RETURNS varchar
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS
$$
const double_dollar = '$' + '$';
var return_value = "";
var to_exec;
var rs_exec;
var range_years = Array.from(Array(new Date().getUTCFullYear() - 2006), (_, i) => (i + 2007).toString());
//CREATE a task TO CALL SP BY YEAR, FROM 2007-current year
range_years.forEach((year_elem) => {
rs = snowflake.createStatement( {
sqlText: `CREATE OR REPLACE TASK MY_TSK_YEAR_${year_elem}
WAREHOUSE = MOS_WEATHER
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
${double_dollar}
DECLARE
year_track float;
rs resultset;
BEGIN
year_track := :1;
rs := (execute IMMEDIATE 'INSERT INTO MY_TABLE VALUES(?)' using (year_track));
return TABLE(rs);
END;
${double_dollar};`
, binds: [year_elem] }).execute();
rs.next();
//rs.getColumnValue(1);
return_value += "MY_TSK_YEAR_"+year_elem+", ";
to_exec = snowflake.createStatement({sqlText: `EXECUTE TASK MY_TSK_YEAR_${year_elem}`});
rs_exec = to_exec.execute();
rs_exec.next();
return_value += rs_exec.getColumnValue(1)+", ";
});
return return_value;
$$;
#Greg Pavlik I don't know my final approach is OK, but it's working, as far as I run this on SF UI!
The above code, for me worked with a minor change from :1 to ${year_elem} , but then I face issue with binding a string value (single quoted) ... like this:
CREATE OR REPLACE PROCEDURE create_exec_tasks_by_year_range(str_arg string)
RETURNS varchar
LANGUAGE JAVASCRIPT
COMMENT ='[DRAFT] SP that creates tasks since a certain rangw with exec immediate block using 2 args: one is year in range, other is a string to isnert as well'
EXECUTE AS OWNER
AS
$$
const double_dollar = '$' + '$';
var return_value = "";
var to_exec;
var rs_exec;
var arg = STR_ARG;
var range_years = Array.from(Array(new Date().getUTCFullYear() - 2020), (_, i) => (i + 2021).toString());
//CREATE a task TO CALL SP BY YEAR, FROM 2007-current year
range_years.forEach((year_elem) => {
rs = snowflake.createStatement( {
sqlText: `CREATE OR REPLACE TASK MY_TSK_YEAR_${year_elem}
SCHEDULE = 'USING CRON 30 22 * * SUN UTC'
AS
EXECUTE IMMEDIATE
${double_dollar}
DECLARE
year_track float;
rs resultset;
my_str varchar;
BEGIN
year_track := ${year_elem};
my_str:= ${arg} ;
rs := (execute IMMEDIATE 'INSERT INTO Z_TEST_TASK_INSERT2 VALUES(?,?)' using (year_track,my_str));
return TABLE(rs);
END;
${double_dollar};`
//, binds: [year_elem]
}).execute();
rs.next();
return_value += rs.getColumnValue(1);
});
return return_value;
$$;
CALL create_exec_tasks_by_year_range('testing_String');
EXECUTE TASK MY_TSK_YEAR_2021;
EXECUTE TASK MY_TSK_YEAR_2022;
000904 SQL compilation error: error line 8 at position 21
invalid identifier 'testing_String'
One thing I would like to understand is:
How to use bind variable as a single quoted string? How to escape it or at leats distinguish it from the query prefixed by ' alread?
The SP creation succeeds via SQL Editor (DBEaver for instance) or SF UI
But the CALL is only working for me on SF UI (and SNOWSQL must happen the same, I assume). Is that something to do with the drivers used by JDBC vs. SF UI?
In Snowflake, when I create a store proc like so
create procedure stack_overflow_question(select_table varchar)
returns varchar
language sql
as
declare
select_statement varchar;
begin
select_statement := '
SELECT * FROM ' || :select_table || '
';
end;
Then, later when I use select get_ddl('procedure', 'stack_overflow_question(varchar)'); function to make edits to the store proc, the result of this function call has extra single quotes.
Here is the result
CREATE OR REPLACE PROCEDURE "STACK_OVERFLOW_QUESTION"("SELECT_TABLE" VARCHAR(16777216))
RETURNS VARCHAR(16777216)
LANGUAGE SQL
EXECUTE AS OWNER
AS 'declare
select_statement varchar;
begin
select_statement := ''
SELECT * FROM '' || :select_table || ''
'';
end';
Note the difference between the two! The extra single quotes. Also double quotes in the name of the store proc.
Is there something that I can do to prevent this from happening? I am using Snowsight - but don't think that this actually is the problem. Also, I am using snowflake as the language for store procs.
Any ideas?
I wrote a UDF that you can wrap around get_ddl that will convert the DDL from using doubled single quotes to single quotes and wrap the body with $$:
create or replace function CODE_DDL_TO_TEXT(CODE_TEXT string)
returns string
language javascript
as
$$
var lines = CODE_TEXT.split("\n");
var out = "";
var startCode = new RegExp("^AS '$", "ig");
var endCode = new RegExp("^'\;$", "ig");
var inCode = false;
var isChange = false;
var s;
for (i = 0; i < lines.length; i++){
isChange = false;
if(!inCode) {
inCode = startCode.test(lines[i]);
if(inCode) {
isChange = true;
out += "AS $" + "$\n";
}
}
if (endCode.test(lines[i])){
out += "$" + "$;";
isChange = true;
inCode = false;
}
if(!isChange){
if(inCode){
s = lines[i].replace(/''/g, "'") + "\n";
s = s.replace(/\\\\/g, "\\");
out += s;
} else {
out += lines[i] + "\n";
}
}
}
return out;
$$;
You can then call the UDF by wrapping it around the get_ddl function. Here is an example of fishing its own DDL out of get_ddl:
select CODE_DDL_TO_TEXT(get_ddl('function', 'CODE_DDL_TO_TEXT(string)'));
Edit:
You can also use this SQL to reconstruct a stored procedure from the INFORMATION_SCHEMA:
select 'create or replace procedure ' || PROCEDURE_NAME || ARGUMENT_SIGNATURE ||
'\nreturns ' || DATA_TYPE ||
'\nlanguage ' || PROCEDURE_LANGUAGE ||
'\nas $' || '$\n' ||
PROCEDURE_DEFINITION ||
'\n$' || '$;'
from INFORMATION_SCHEMA.PROCEDURES
;
This only returns body -
SELECT PROCEDURE_DEFINITION
FROM INFORMATION_SCHEMA.PROCEDURES
WHERE PROCEDURE_SCHEMA = 'SCHEMA_NAME' AND PROCEDURE_NAME = upper('stack_overflow_question');
i'm working with Snowflake,
i created this Stored Procedure, it's purpose is to do the following steps:
extract the relevant 'application_id' values from the SQL query
use a FOR loop over a SQL query, with the 'application_id' values that i extracted
in step 1, and delete the relevant rows from the target table.
when i call the Stored Procedure, it runs without errors, but it doesn't do anything ( the relevant records are not been deleted).
i added my SP code,
please let me know if you see any syntax / logic errors,
thanks
CREATE OR REPLACE PROCEDURE DWH.sp_testing()
RETURNS string
LANGUAGE javascript
strict
EXECUTE AS owner
AS
$$
try
{
var application_list = ` SELECT application_id
FROM PUBLIC.my_source_table_name
WHERE my_flag = 1
`
var query_statement = snowflake.createStatement({sqlText: application_list});
var application_list_result = query_statement.execute();
for (i = 1; i <= query_statement.getRowCount(); i++)
{
application_list_result.next()
application_id = application_list_result.getColumnValue(1)
var delete_daily_records = `
DELETE FROM PUBLIC.my_target_table_name AS target
WHERE target.application_id = ${application_id}
`
snowflake.execute({sqlText: delete_daily_records});
}
}
catch (err)
{
throw "Failed: " + err;
}
$$
;
CALL DWH.sp_testing();
Are you sure your query is returning data? Also are you sure the target table has data matching your DELETE statement?
The following test works for me using your stored procedure:
select count(*) from citibike_trips where end_station_id=6215;
returns: 14565
Now, I created your stored proc:
CREATE OR REPLACE PROCEDURE sp_testing()
RETURNS string
LANGUAGE javascript
strict
EXECUTE AS owner
AS
$$
try
{
var application_list = `SELECT end_station_id
FROM citibike_trips
WHERE end_station_id=6215 limit 10
`
var query_statement = snowflake.createStatement({sqlText: application_list});
var application_list_result = query_statement.execute();
for (i = 1; i <= query_statement.getRowCount(); i++)
{
application_list_result.next()
end_station_id = application_list_result.getColumnValue(1)
var delete_daily_records = `
DELETE FROM citibike_trips AS target
WHERE target.end_station_id = ${end_station_id}
`
snowflake.execute({sqlText: delete_daily_records});
}
}
catch (err)
{
throw "Failed: " + err;
}
$$
;
Run it:
CALL SP_TESTING();
Shows NULL as result (expected since it is not returning anything).
But then when I check the table again:
select count(*) from citibike_trips where end_station_id=6215;
returns: 0
I am trying to do error handling for snowflake stored procedure. The stored procedure has 4 SQL statement inside. I need check errors for all for statement. When error pops, i need to know which Statement error. How to achieve this? Thanks in advance. I appreciate it.
CREATE OR REPLACE PROCEDURE dw.sp_fidctp_audit_trail_load(SYSDATE varchar)
RETURNS STRING
LANGUAGE javascript
AS
'
var sql_command = "truncate table STG.tb_fidctp_audit_trail_input";
var result = snowflake.execute ( {sqlText: sql_command, binds: [SYSDATE]});
sql_command =`COPY INTO STG.tb_fidctp_audit_trail_input (record_id,ORDER_ID,PARENT_ORDER_ID,EVENT_TYPE,EVENT_DATETIME,PRIMARY_STATE,SECONDARY_STATE,CURRENT_SERVICE_ID,CURRENT_EXECUTOR_ID,TRADING_QUANTITY,LIMIT_PRICE,EVENT_TEXT,ORDER_NOTES,REASON_TEXT,ROUTED_ORDER_CODE,ROOT_ORDER_ID,INSTRUMENT_CODE,ORDER_SOURCE,LEAVES,ACCT_ID,CUST_CROSS_ID,ORDER_FLAGS)
from #STG.CTP_STAGE/AUDIT_TRAIL.TOP.`;
sql_command += SYSDATE;
sql_command += ".psv.gz file_format = (FORMAT_NAME = ''STG.CTP_AUDIT_TRAIL_FF'', ERROR_ON_COLUMN_COUNT_MISMATCH = TRUE, encoding = ''iso-8859-1'') ";
try{
result = snowflake.execute (
{sqlText: sql_command, binds: [SYSDATE]}
);
return "Succeeded."; // Return a success/error indicator.
}
catch (err) {
return "Failed: " + err + "ERROR ON COPY STATEMENT"; // Return a success/error indicator.
}
sql_command = `delete from DW.tb_fidctp_audit_trail
where SYSDATE = to_date((:1), ''YYYYMMDD'')`;
result = snowflake.execute ( {sqlText: sql_command, binds: [SYSDATE]});
sql_command = ` insert into DW.tb_fidctp_audit_trail(sysdate, order_id, parent_order_id, root_order_id, event_datetime, instrument_code, event_text,reason_text, current_service_id, current_executor_id, trading_quantity, leaves )
select to_date(left(event_datetime, 8), ''YYYYMMDD''), order_id, parent_order_id, root_order_id, to_timestamp(left(event_datetime, 24), ''YYYYMMDD HH24:MI:SS.FF''), instrument_code, event_text, reason_text, current_service_id, current_executor_id, cast(trading_quantity as NUMBER(18,0)), cast(leaves as NUMBER(18,0))
from STG.tb_fidctp_audit_trail_input
`;
try{
snowflake.execute (
{sqlText: sql_command, binds: [SYSDATE]}
);
return "Succeeded."; // Return a success/error indicator.
}
catch (err) {
return "Failed: " + err + "ERROR ON INSERT STATEMENT"; // Return a success/error indicator.
}
'
;
There are a few things to do here. Let's start with the create header:
CREATE OR REPLACE PROCEDURE dw.sp_fidctp_audit_trail_load(SYSDATE varchar)
RETURNS STRING
LANGUAGE javascript
AS
'
When you use a single quote to enclose the body of a JavaScript procedure, it makes using single quotes in SQL statements inside the body very cumbersome, having to double the single quotes. To avoid that, you can use an alternate string terminator in Snowflake, $$ like this:
CREATE OR REPLACE PROCEDURE dw.sp_fidctp_audit_trail_load(SYSDATE varchar)
RETURNS STRING
LANGUAGE javascript
AS
$$
// Body goes here. There's no longer a need to escape single quotes
$$;
The issue with the return values is in this part:
return "Succeeded."; // Return a success/error indicator.
}
catch (err) {
return "Failed: " + err + "ERROR ON COPY STATEMENT"; // Return a success/error indicator.
}
In JavaScript, when the return statement is used in the main function, it leaves the main function, which terminates the stored procedure. This means if the copy statement succeeds, the procedure will exit and return "Succeeded." If the copy statement fails, it will hit a different return statement and exit with the error message.
Either way, it will terminate execution after that code block.
Take the return statements out of the successful branches of the try/catch blocks except for the last one.
You can just print the statement in the CATCH statement, either by printing the sql_command variable, or using getSqlText() function from the Statement object.
CREATE OR REPLACE PROCEDURE test_catch_error_sp(SYSDATE varchar)
RETURNS STRING
LANGUAGE javascript
AS
$$
var commands = [];
var params = [];
var results = {"success": [], "failed": []};
commands[0] = "create or replace table test1 (a int)";
params[0] = [];
commands[1] = "INSERT INTO test1 values (?)";
params[1] = [SYSDATE];
commands[2] = "drop table test1";
params[2] = [];
var stmt = null;
for (i = 0; i < commands.length; i++) {
try {
stmt = snowflake.createStatement({sqlText: commands[i], binds: params[i]});
var result = stmt.execute();
results['success'].push(stmt.getSqlText());
stmt = null; // reset it
} catch (err) {
// if createStatement failed, then stmt will still be null
results['failed'].push(
"Failed: " + err + ". ERROR ON STATEMENT: '" + stmt ? stmt.getSqlText() : commands[i] + "'"
);
}
}
return JSON.stringify(results);
$$
;
call test_catch_error_sp('2020-01-01');
-- returns:
-- {"success":["create or replace table test1 (a int)","drop table test1"],"failed":["INSERT INTO test1 values (?)"]}
I am trying to assign a simple sql query result in stored procedure to a variable which I will use it later in another sql statement. If I execute below statement with out putting in procedure it works but not with stored procedure, I get an error while calling sp , Can someone please help me here ?
Procedure code:
$$
VAR NAME = 'ABC'
SET (COUNT_VALUE) = (SELECT COUNT(*) FROM COUNT_TABLE)
With out Procedure below code works
SET (COUNT_VALUE) = (SELECT COUNT(*) FROM COUNT_TABLE)
select $COUNT_VALUE
Thanks
Here's an example that uses a simple helper function to return a result set from a query. You can read the Snowflake docs to see what the result set API has in it. The getResultSet function returns a variable that has the results of the query. Since the SQL is a count, you need to use rs.next() once to get to the first row and then read the value in the aliased count(*). You could also use rs.getColumnValue(1) to get a column by ordinal position, but I recommend using column names with SQL aliases is necessary.
create or replace procedure COUNT_EXAMPLE()
returns string
language javascript
as
$$
var rowCount = 0;
var sql = "select count(*) as ROW_COUNT from SNOWFLAKE_SAMPLE_DATA.TPCH_SF10000.ORDERS;";
try {
var rs = getResultSet(sql);
if (rs.next()) {
var rowCount = rs.getColumnValue("ROW_COUNT");
} else {
return "Error: Count query failed.";
}
}
catch(err) {
return "Error: " + err.message;
}
return "The table has " + rowCount + " rows.";
//--------------------------- End of main function ---------------------------
function getResultSet(sql){
cmd1 = {sqlText: sql};
stmt = snowflake.createStatement(cmd1);
var rs;
rs = stmt.execute();
return rs;
}
$$;
call COUNT_EXAMPLE();