I am trying to create an UDTF where I am planning to pass the dynamic where clause as an argument to the UDTF
CREATE OR REPLACE FUNCTION FUNCTION_NAME(where_clause VARCHAR)
RETURNS TABLE ()
LANGUAGE JAVASCRIPT
AS
$$ var sql_command=SELECT COL1, COL2 FROM TABLE_1 JOIN TABLE_2 ON ....
+where_clause+ GROUP BY ...;
var stmt = snowflake.createStatement( {sqlText: sql_command} );
var resultSet = stmt.execute(); resultSet.next();
$$
Is this something possible and also I want to be able to handle quotes in the arguments during calling of the function
for example if the call is:
SELECT * FROM TABLE(FUNCTION_NAME('WHERE COL_1='value''));
How do I handle quotes during calling of the function. Thanks for the help in advance
I wanted to underline the answer of Felipe. JavaScript UDTFs do not support "Stored Procedures API".
https://docs.snowflake.com/en/sql-reference/stored-procedures-api.html
So you can not create and run a dynamic SQL inside of a JavaScript UDTF. It must have "processRow" callback function to process the values coming as parameters and return them as rows.
https://docs.snowflake.com/en/sql-reference/udf-js-table-functions.html#basic-hello-world-examples
You can create a stored procedure to run a dynamic SQL and return the result as an array/json, but it won't be effective.
For your original question, you just need to escape the quote by repeating it or using slash:
call FUNCTION_NAME('WHERE i=''1''');
call FUNCTION_NAME('WHERE i=\'1\'');
Related
SPLIT_PART function used in a select statement in Snowflake Stored procedure returns null value.
var stmt5 = snowflake.CreateStatement({sqltext:`Select SPLIT_PART(SPLIT_PART(:1,''/'',2),''.gz'',1)`,binds:[stagename]});
var queryText=stmt5.getSqlText();
var x=stmt5.execute();
x.next();
stagename is retrieved from the List #my_stage command result and the result of the Split Part is used in the Copy Command for inserting records into a snowflake table.
If somebody responds will share the code thru an email to help me fix the issue. Thanks in advance.
You're using JavaScript with backticks, so there's no need to escape the single quotes. By doing so, it actually looks like an empty string. This is how it would look in your SQL history tab:
Select SPLIT_PART(SPLIT_PART(:1,''/'',2),''.gz'',1)
If you put a file name in and try that:
Select SPLIT_PART(SPLIT_PART('mypath/myfile.gz',''/'',2),''.gz'',1)
This doesn't actually return a null value. It's a syntax error, so maybe this execute call is in a try/catch block. That would return a null value when you try to use .next(),
Try it with the quotes single:
var stmt5 = snowflake.CreateStatement({sqltext:`Select SPLIT_PART(SPLIT_PART(:1,'/',2),'.gz',1)`,binds:[stagename]});
I am trying to create a stored procedure to copy from the external stage (s3 bucket) and use a pattern for the file name. The pattern is based on the concatenated current date but I need to set a variable to use as a pattern. Is it possible to do something like this?
CREATE OR REPLACE PROCEDURE test_copy()
RETURNS STRING
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
SET my_Date=(select concat('.*', regexp_replace(current_date(),'-',''), '.*.parquet' );
var sql_command = '
COPY INTO table1
FROM '#s3bucket'
(file_format => PARQUET, pattern=>$my_Date)
);
'
snowflake.execute(
{
sqlText: sql_command
});
return "Successfully executed.";
$$;
As you can generate the current date in javascript, why not create my_Date purely in javascript?
You then need to create sql_command by concatenating the required strings and variables together
I am creating a snowflake stored procedure where I need to store the output if the SQL query into an array , whose values will be later used to create SQL Command which will be the output. Please suggest how do I store the output in an array.
Please refer to the doc on how to setup SP in Snowflake:
https://docs.snowflake.com/en/sql-reference/stored-procedures-api.html
https://docs.snowflake.com/en/sql-reference/stored-procedures-usage.html
There are lots of examples over there.
You just need to know how to query Snowflake and get results, and the rest are just standard Javascript.
Example as below:
create or replace procedure test_sp()
returns string
language javascript
as
$$
var my_data = [];
var stmt = snowflake.createStatement({sqlText: "SELECT * FROM table_name" });
var rs = stmt.execute();
while (rs.next()) {
my_data.push(rs.getColumnValue(1));
my_data.push(rs.getColumnValue(2));
}
// then you can use my_data array in later part of your code
return '';
$$;
Here is my sample code below:
CREATE OR REPLACE PROCEDURE database.schema.sp_sample(dynamic_columns VARCHAR, dynamic_where_clause VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
rs="Success";
try {
retrieve_queries_sql = `SELECT COL_1, COL_2, ? FROM table ? GROUP BY ?`;
var stmt = snowflake.createStatement( {sqlText: retrieve_queries_sql, binds:[DYNAMIC_COLUMNS, DYNAMIC_WHERE_CLAUSE, DYNAMIC_COLUMNS]} );
var rs = stmt.execute();
}
catch(err) {
rs= "Failed Message: "+err.message;
}
return rs;
$$;
When I try to bind these columns Snowflake throws an error Unexpected ? and I even tried using :1, :2 even that didnt work same error Unexpected :
Dont want to prefer interpolation ${dynamic_columns} or ${dynamic_where_clause} as it can lead to sql injection concerns
How to bind the arguments in the best possible way? Any suggestions appreciated.
in other UDF the parameters to the JavaScript function need to be ALL_CAPS in the JavaScript body, as the SQL part (the function declaration) the SQL is case insensitive, while java it's "how it really is" which is upper case. Or you can double quote then in the declaration, as lower case, to have then lower case in the body.
Given the following procedure
CREATE OR REPLACE PROCEDURE z()
RETURNS STRING NOT NULL
LANGUAGE JAVASCRIPT
AS
$$
const what = snowflake.execute( {sqlText: "select $1 as value from values (1), (2), (3);"} )
return 'Hi There';
$$
;
If I run the 2 statements below
CALL z();
select * from table(result_scan(-2));
I get an error "SQL Error [709] [02000]: Statement 01912a0c-01c1-0574-0000-4de50036137e not found"
If I run the 2 statements below
CALL z();
SELECT LAST_QUERY_ID(), LAST_QUERY_ID(-1), LAST_QUERY_ID(-2);
It shows me that LAST_QUERY_ID() and LAST_QUERY_ID(-1) are identical but also that LAST_QUERY_ID(-2) returns NULL...
Any idea why it returns NULL rather than something that would allow me to retrieve the result of my query "select $1 as value from values (1), (2), (3);"?
Thanks
It shows me that LAST_QUERY_ID() and LAST_QUERY_ID(-1) are identical but also that LAST_QUERY_ID(-2) returns NULL
It's expected to see the same query ID for LAST_QUERY_ID() and LAST_QUERY_ID(-1), because they are same (the default value for LAST_QUERY_ID is "-1").
On the other hand, the LAST_QUERY_ID(-2) should not returns NULL, and the "select * from table(result_scan(-2))" query should not fail.
If you define the function with CALLER rights, you can see that the LAST_QUERY_ID(-2) returns the query ID and the "select * from table(result_scan(-2))" query works:
CREATE OR REPLACE PROCEDURE z()
RETURNS STRING NOT NULL
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
const what = snowflake.execute( {sqlText: "select $1 as value from values (1), (2), (3);"} )
return 'Hi There';
$$
;
As a workaround, you can use the history window to check the latest queries. You should be able to see your query (01912a0c-01c1-0574-0000-4de50036137e) in the history tab. You will notice that these queries are executed by same user, and in the same session. Therefore, there shouldn't be a restriction on listing these queries.
I am able to reproduce the issue on my test environment, and I will report the issue to the development team. If you have access to Snowflake support, it could be better to submit a support case regarding to this issue, so you can easily follow the process.
For the record, here is the answer I got from Snowflake Support
This is an expected behavior because of security limitation placed on purpose (queries from inside a SP do not have access to queries outside and vice versa).
We put a blanket limitation that the world of inside and outside such SPs should be separated. From one the "result" of a query from another cannot be accessed.
Yes, If we declare the procedure with caller rights, everything (LAST_QUERY_ID and RESULT_SCAN) works fine in this case.
There is a discussion going on to implement this use-case when the owner and caller of an "owner's right" SP are the same, all this limitations should be lifted. But we cannot be sure about the timeline, I can link this case with that discussion so that we have this record.
Sounds good to me :-)