Snowflake Syntax when passing array into insert query - snowflake-cloud-data-platform

My code obtains the column list of a table I have created
the attributes of this table are the contextual values of a session in snowflake
such as USER,DATABASE,WAREHOUSE...ETC
Afterwards it places those attribute names into an array
I then try to call on these names in making an insert query and this is where I am struggling with the syntax
Because each value in my array is USER, DATABASE,WAREHOUSE
I am trying to call on the context functions
like CURRENT_WAREHOUSE()
Can someone please help me with the syntax
for(i=0;i<arr.length;i++){
v_sqlCode = `INSERT INTO SESSION_ATTRIBUTES( arr[i] )
"VALUES ("CALL CURRENT_"+arr[i]+"()")';
}

You can't directly use the output of a Snowflake stored procedure programmatically. If you need to use the output value, you have to collect it using the RESULT_SCAN table function. That can get a bit complex to handle, directly in code, so it's far better to place it into a helper function. See this example of one SP calling another one and using its output value. It does this by calling the SP using the callSP helper function. Use the entire SQL statement including the call command into the SQL parameter for that function. It will run the SP, grab the result from the result_scan table function and return it.
create or replace procedure HELLO_WORLD()
returns string
language javascript
as
$$
return "Hello, world.";
$$;
create or replace procedure CALL_HELLO_WORLD()
returns string
language javascript
execute as caller
as
$$
return callSP(`call HELLO_WORLD()`);
// ---- Main function above, helper functions below.
function callSP(sql){
let cmd = {sqlText: sql};
let stmt = snowflake.createStatement(cmd);
stmt.execute();
let result_scan = `select $1 from table(result_scan(last_query_id()));`;
let result_cmd = {sqlText: result_scan};
let result_stmt = snowflake.createStatement(result_cmd);
let rs = stmt.execute();
if(rs.next()) {
return rs.getColumnValue(1);
} else {
return null;
}
}
$$;
call call_hello_world();

Related

How to bind JavaScript based date column in Snowflake SQL

I am creating snowflake JavaScript based store procedure. How can i refer the date data type variable in snowflake sql.
Here is the sample code:
In the below code ,please suggest how can i use 'dnblatestdt' variable in sql statement.
create or replace procedure test_proc_registration_master_perished_dt(PARAM_REG_SUB_UUID VARCHAR)
returns varchar not null
language javascript
as
$$
/*get latest ingestion_uuid for the given state*/
var step01=`select distinct dnb_applicable_dt,ingestion_uuid from temp_registration_hash_master `;
var statement01=snowflake.createStatement( {sqlText: step01,binds: [PARAM_REG_SUB_UUID]} );
variable1= statement01.execute();
variable1.next();
dnblatestdt=variable1.getColumnValue(1);
ingsuuid=variable1.getColumnValue(2);
/* check if the ingestion is successful or not*/
var step02=`select INGESTION_SUCCESSFUL from FILE_INGESTION_HISTORY where ingestion_uuid=:1 and date=:2::TIMESTAMP_LTZ::DATE`;
var statement02=snowflake.createStatement( {sqlText: step02,binds: [ingsuuid,dnblatestdt]} );
variable2= statement02.execute();
variable2.next();
ingsindc=variable2.getColumnValue(1);
return 'success'
$$
So I wrote a much simpler function that uses a similar pattern to your code:
create or replace procedure test_proc()
returns varchar not null
language javascript
as
$$
var step01 = `SELECT 6::number, '2022-01-27'::timestamp_ntz;`;
var statement01 = snowflake.createStatement( {sqlText: step01} );
results1 = statement01.execute();
results1.next();
ingsuuid = results1.getColumnValue(1);
dnblatestdt = results1.getColumnValue(2);
/* check if the ingestion is successful or not*/
var step02=`SELECT :1 * 2, DATEADD(year,-1, :2::timestamp_ntz);`;
var statement02 = snowflake.createStatement( {sqlText: step02,binds: [ingsuuid , dnblatestdt]} );
results2 = statement02.execute();
results2.next();
ingsindc = results2.getColumnValue(1);
return 'success'
$$
;
and using it works for me:
call test_proc();
TEST_PROC
success
I swapped the order of the reading parameters on the first function, but that should not be a problem.
this makes me thing your casting on the second instance is not working
:2::TIMESTAMP_LTZ::DATE
so I would suggest moving that casting to the first function, which you can test outside the stored procedure, thus.
SELECT DISTINCT dnb_applicable_dt::TIMESTAMP_LTZ::DATE, ingestion_uuid
FROM temp_registration_hash_master
when that is happy, you shouldn't need any casting on the second used of the values.

Query `list stage` in Snowflake UDF

I am trying to write a Snowflake UDF that accepts a stage name and specific folder name as input parameters and returns the latest file id ( striping from full file name) as the output. Could anyone help me with a simple code to achieve this?
I'm not sure if you want a UDF or stored procedure. The syntax to create would be similar so I think this can help. Here is a stored procedure which will fetch latest staged file from a given stage and path. Just be aware of the limit 1 in query, multiple staged files may share the same last modified date while this procedure returns a scalar (single) value.
Stored Procedure Definition
create or replace procedure "MYDB"."MYSCHEMA"."LATEST_STAGED_FILE"(stage_name text, folder text)
returns string not null
language javascript
execute as caller
as
$$
var sql_text = "list #" + STAGE_NAME + "/" + FOLDER ;
var sql_command0 = snowflake.createStatement({ sqlText: sql_text});
var sql_command1 = snowflake.createStatement({ sqlText:`SELECT "name" FROM table(result_scan(last_query_id())) WHERE "last_modified" = (select MAX("last_modified") from table(result_scan(last_query_id()))) LIMIT 1;`});
try {
sql_command0.execute();
var resultSet = sql_command1.execute();
while(resultSet.next())
{
var resultFile = resultSet.getColumnValue('name').split("/")
return resultFile[resultFile.length - 1]
}
}
catch (err) {
return "Failed: " + err;
}
$$;
You can then call the stored procedure like
call "MYDB"."MYSCHEMA"."LATEST_STAGED_FILE"('MYDB.MYSCHEMA.MYSTAGE', 'mypath/myotherpath');
References
select from list #
list stage via SP

Snowflake stored procedure argument is not recognized

I'm constructing a snowflake stored procedure and I'm facing difficulty in using the passed argument in the snowflake procedure.
create or replace procedure dumper(n float)
returns float
language javascript
execute as caller
as
$$
var text = "select file_name from table(information_schema.COPY_HISTORY(TABLE_NAME=> 'records', start_time=> dateadd(hours, ?, current_timestamp())));";
var statement = snowflake.createStatement({sqlText: text, binds: [n]});
var result = statement.execute();
return statement.getRowCount();
$$
;
attempting to call the above procedure
call dumper(-2);
result in the following error
JavaScript execution error: Uncaught ReferenceError: n is not defined in DUMPER at ' var statement = snowflake.createStatement({sqlText: text, binds: [n]});' position 70 stackstrace: DUMPER line: 4
I tried using the interpolation one discussed over here but that too had no success.
Any clue on how to work with passed argument.
You have to capitalize "N" in your JavaScript code:
var statement = snowflake.createStatement({sqlText: text, binds: [N]});
Variables passed into Snowflake stored procedures behave like other object names until they're inside the JavaScript. If they're not double quoted, then Snowflake implicitly capitalizes them. Remember to uppercase all parameters passed into SPs and UDFs in Snowflake. Variables defined inside the SP or UDF using JavaScript follow the normal rules for the language.
Since the regular rules apply to Snowflake identifiers as they do to variables passed into procedures and functions, you can double quote parameters if you want to use lower or mixed case variable names:
create or replace function echo_string("n" string)
returns string
language javascript
as
$$
return n; // This works because "n" is double quoted in the signature
$$;
select echo_string('Hello world.');

Dynamic file paths for Snowflake stages

I am copying data from a Snowflake table into an S3 external stage:
COPY INTO '#my_stage/my_folder/my_file.csv.gz' FROM (
SELECT *
FROM my_table
)
However this code runs daily and I don't want to overwrite my_file.csv.gz but rather keep all the historical versions. However I haven't found a way to create dynamic paths:
SET stage_name=CONCAT('#my_stage/my_folder/my_file', '_date.csv.gz');
COPY INTO $stage_name FROM (
SELECT *
FROM my_table
);
COPY INTO IDENTIFIER($stage_name) FROM (
SELECT *
FROM my_table
);
None of the later 2 queries work!
My question: How can I create dynamic Stage paths in Snowflake? Thanks
Here's a stored procedure you can use and modify. Note that the line with the comment to modify your copy into statement uses backticks instead of single or double quotes. In JavaScript, that allows use of single or double quotes in the string, multi-line constants, and replacement tokens in the form ${variable_name}
create or replace procedure COPY_TO_STAGE(PATH string)
returns variant
language javascript
as
$$
class Query{
constructor(statement){
this.statement = statement;
}
}
// Start of main function
var out = {};
// Change your copy into statement here.
var q = getQuery(`copy into '${PATH}' from (select * from my_table);`);
if (q.resultSet.next()) {
out["rows_unloaded"] = q.resultSet.getColumnValue("rows_unloaded");
out["input_bytes"] = q.resultSet.getColumnValue("input_bytes");
out["output_bytes"] = q.resultSet.getColumnValue("output_bytes");
} else {
out["Error"] = "Unknown error";
}
return out;
// End of main function
function getQuery(sql){
cmd1 = {sqlText: sql};
var query = new Query(snowflake.createStatement(cmd1));
query.resultSet = query.statement.execute();
return query;
}
$$;
Once you define it, you can use SQL variables as the input if you want:
SET stage_name=CONCAT('#my_stage/my_folder/my_file', '_date.csv.gz');
call copy_to_stage($stage_name);
This won't work. Unfortunately using variables for identifiers does not work for stages. You might need to create a Stored procedure with Dynamic SQL:
https://docs.snowflake.com/en/sql-reference/stored-procedures-usage.html#label-example-of-dynamic-sql-in-stored-procedure
So you can just call this procedure every day or generating a SP with several parameters for the path (Stage), the query which will be executed and the target filename.

Executing a SQL statement, present in a table, and returning the resulting record set in Snowflake

I need help with a practical scenario. I have a table called CONFIG_TBL in Snowflake. This table has SQL statements, 1 per row. My goal is to use Snowflake Stored Procedures or Snowflake UDF, or a combination of the two, and return a result set that will be obtained after execution of that statment.
The statements are simple select statements, like "select * from ABC'.
I could have done this, very easily in SQL server, since procedures can return table values. However, I don't know how to do this in Snowflake.
Any help will be greatly appreciated.
Thanks in advance.
Here's something to get you started at least. Procedures use javascript (SQL Stored Procedures are coming soon), but they can be used to run dynamic queries like you are looking for.
You can get the results in a couple of ways. By either returning a variant object or by using result_scan after calling the procedure.
This example just runs one query so your final solution will be different depending on just what you want the output to look like.
CREATE OR REPLACE PROCEDURE SCHEMA.PROCEDURE_NAME()
RETURNS VARIANT
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
retrieve_queries_sql = "select top 1 query from CONFIG_TBL";
retrieve_queries_result_set = snowflake.execute({sqlText: retrieve_queries_sql });
query_to_run = retrieve_queries_result_set.next().getColumnValue(1);
rs = snowflake.execute({sqlText: query_to_run})
var return_value = "";
if (rs.next()) {
return_value += rs.getColumnValue(1);
return_value += ", " + rs.getColumnValue(2);
}
while (rs.next()) {
return_value += "\n";
return_value += rs.getColumnValue(1);
return_value += ", " + rs.getColumnValue(2);
}
}
return return_value;
$$
CALL SCHEMA.PROCEDURE_NAME()
SELECT *
FROM table(result_scan(last_query_id()))
Edit: Fixed to have example correctly return a result which can then be used by the result_scan. Example taken from here. There are various more examples for getting results out of a procedure, including using JSON output.
We can use language as SQL and return the value to table, Which can return table values.
CREATE OR REPLACE PROCEDURE DDL_TEST_SQL()
RETURNS TABLE(CARRIERFILEID integer, BLOBPATH varchar)
LANGUAGE SQL
EXECUTE AS OWNER
AS
$$
DECLARE
QUERY STRING;
res resultset;
BEGIN
QUERY := 'SELECT TOP 10 CARRIERFILEID,BLOBPATH from CONFIG.CARRIERFILE';
res := (EXECUTE IMMEDIATE :QUERY);
return table(res);
END;
$$;
CALL DDL_TEST_SQL()

Resources