(Submitting on behalf of a Snowflake User...)
QUESTION:
Is it possible to nest multiple functions inside of a function and pass all the parameters required?
for example...
CREATE OR REPLACE FUNCTION "udf_InteractionIndicator"("ROH_RENEWAL_SYSTEM_STATUS1" VARCHAR(100), "GOLS_OPPORTUNITY_LINE_STATUS" VARCHAR(100)
, "ROH_CLIENT_CURRENT_TEMPERATURE1" VARCHAR(100)
, "ROH_PO_ATTACHED" VARCHAR(100)
, "ROH_PO_NUMBER" VARCHAR(100)
, "RT_PAID_OVERRIDE" VARCHAR(100), "ROH_RENEWAL_OPPORTUNITY_STATUS1" VARCHAR(100)
, "ROH_RENEWAL_CONVERSATION_DATE" DATE, "ROH_APPROVAL_RECEIVED_DATE" DATETIME)
RETURNS NUMBER(1,0)
AS
$$
CASE WHEN ("udf_RenewalNoticeSentIndicator"("ROH_RENEWAL_SYSTEM_STATUS1", "ROH_CLIENT_CURRENT_TEMPERATURE1"
, "GOLS_OPPORTUNITY_LINE_STATUS"
, "ROH_PO_ATTACHED", "RT_PAID_OVERRIDE"
, "ROH_RENEWAL_OPPORTUNITY_STATUS1")) = 1
AND (ROH_RENEWAL_CONVERSATION_DATE IS NOT NULL
OR ("udf_AuthorizedIndicator"(ROH_APPROVAL_RECEIVED_DATE, "ROH_PO_ATTACHED", "ROH_PO_NUMBER")) = 1
OR ("udf_PaidIndicator"("GOLS_OPPORTUNITY_LINE_STATUS")) = 1
OR ("udf_ChurnIndicator"("GOLS_OPPORTUNITY_LINE_STATUS")) = 1
)
THEN 1 ELSE 0 END
$$
;
I've received the recommendation to:
...create a SQL UDF or JavaScript UDF. A JavaScript UDF can only
contain JavaScript code, and an SQL UDF can contain only one SQL
statement (no DML and DDL). In case of nesting, SQL UDF can call
another SQL UDF or a JavaScript UDF but the same is not true with the
JavaScript UDF(it only contains JavaScript code).
CREATE OR REPLACE FUNCTION udf_InteractionIndicator_nested(ID DOUBLE)
RETURNS DOUBLE
AS
$$
SELECT ID
$$;
create or replace function js_factorial(d double)
returns double
language javascript
strict
as '
if (D <= 0) {
return 1;
} else {
var result = 1;
for (var i = 2; i <= D; i++) {
result = result * i;
}
return result;
}
';
CREATE OR REPLACE FUNCTION udf_InteractionIndicator(ID DOUBLE)
RETURNS double
AS
$$
select udf_InteractionIndicator_nested(ID) + js_factorial(ID)
$$;
select udf_InteractionIndicator(4);
+-----------------------------+
| UDF_INTERACTIONINDICATOR(4) |
|-----------------------------|
| 28 |
+-----------------------------+
HOWEVER, I'm trying to accomplish this with a SQL UDF. It makes sense that a nested function can be created as long as they use the same parameter. I'd like to create a function that accepts say 8 parameters and the underlying functions may reference all, some or none of the parent function parameters. That is where I run into an issue... THOUGHTS?
(A consultant in our community offered the following answer...)
With a JavaScript UDF the design will be much more compact and maintainable, if your use case is that there is a "main" function that breaks down work into subfunctions which will only be invoked from main.
Then you simply define all underlying functions within the main function, which is possible with JavaScript but not with an SQL UDF, and then you are free to use the main parameters everywhere.
CREATE OR REPLACE FUNCTION MAIN_JS(P1 STRING, P2 FLOAT)
RETURNS FLOAT
LANGUAGE JAVASCRIPT
AS '
function helper_1(p) { return p * 2; }
function helper_2(p) { return p == "triple" ? P2 * 3 : P2; }
return helper_1(P2) + helper_2(P1);
';
SELECT MAIN_JS('triple', 4); -- => 20
Related
create or replace function sppi()
returns VARCHAR
language javascript
as
$$
var A= regexp_replace('Customers - (NY)','\\(|\\)','');
return A;
$$
;
call sppi();
Well your REGEXP is valid from the console/WebUI perspective:
select 'Customers - (NY)' as str, regexp_replace(str,'\\(|\\)','');
STR
REGEXP_REPLACE(STR,'\(|\)','')
Customers - (NY)
Customers - NY
so in javascipt functions you cannot directly call SQL functions, so if we flip to a Snowflake Scripting we can though.
BEGIN
let A := regexp_replace('Customers - (NY)','\\(|\\)','');
RETURN :A;
END;
anonymous block
Customers - NY
where-as if you want to stay in Javasript, lets use a Javascript replace:
create or replace function sppi()
returns VARCHAR
language javascript
as
$$
var A= 'Customers - (NY)'.replace(/\(|\)/g,'');
return A;
$$
;
select sppi();
SPPI()
Customers - NY
I have to add a variable MaxDate in my SQL Stored Proc (shown below). The code gets errored out since MaxDate is not represented by its value. Any idea on how I can pass a variable in a stored proc?
create or replace procedure Load_Employee()
returns varchar not null
language javascript
EXECUTE AS CALLER
as
$$
//Variable Initialization
var IntegrationTable ='EMPLOYEE';
var TypeID=0;
var MaxDate=' ';
var cmd = "Select max(COMPLETED_DATE) from SCHEMA.TABLE where TARGET_TABLE_NAME= " + "'" + IntegrationTable + "'" ;
var sql = snowflake.createStatement({sqlText: cmd});
var result = sql.execute();
result.next();
MaxDate=result.getColumnValue(1);
var cmd=` Insert into PersonTable
select SHA1(concat(Person_id,'|','Person')) ,12345678,SHA1(concat('Payroll','|','Pay','|', Load_Date)) ,current_timestamp() , Tenant
from Schema.PERSONTABLE where Date_Added >= MaxDate
where TYPE='ABC' ;`;
$$
;
If your query to get MaxDate works right, then the value should be in the variable. The problem is it's not being replaced in the sql variable defining the insert statement.
Since you're using backticks to open and close the string, you can use a special JavaScript notation to replace the variable with its value, ${MaxDate}.
Your definition of the insert statement would look like this:
var cmd=` Insert into PersonTable
select SHA1(concat(Person_id,'|','Person')) ,12345678,SHA1(concat('Payroll','|','Pay','|', Load_Date)) ,current_timestamp() , Tenant
from Schema.PERSONTABLE where Date_Added >= ${MaxDate}
where TYPE='ABC' ;`;
If that doesn't work, try cutting the SP short with return MaxDate; to see what got assigned to that variable. Also it's very helpful to check the query history view to see what SQL actually ran inside a stored procedure.
Also, I think this is the same SP that was having an issue with a null return. You'll need to return a string value using something like return 'Success'; or something to avoid getting an error for the null return. That's because of the returns varchar not null in the definition.
I am copying data from a Snowflake table into an S3 external stage:
COPY INTO '#my_stage/my_folder/my_file.csv.gz' FROM (
SELECT *
FROM my_table
)
However this code runs daily and I don't want to overwrite my_file.csv.gz but rather keep all the historical versions. However I haven't found a way to create dynamic paths:
SET stage_name=CONCAT('#my_stage/my_folder/my_file', '_date.csv.gz');
COPY INTO $stage_name FROM (
SELECT *
FROM my_table
);
COPY INTO IDENTIFIER($stage_name) FROM (
SELECT *
FROM my_table
);
None of the later 2 queries work!
My question: How can I create dynamic Stage paths in Snowflake? Thanks
Here's a stored procedure you can use and modify. Note that the line with the comment to modify your copy into statement uses backticks instead of single or double quotes. In JavaScript, that allows use of single or double quotes in the string, multi-line constants, and replacement tokens in the form ${variable_name}
create or replace procedure COPY_TO_STAGE(PATH string)
returns variant
language javascript
as
$$
class Query{
constructor(statement){
this.statement = statement;
}
}
// Start of main function
var out = {};
// Change your copy into statement here.
var q = getQuery(`copy into '${PATH}' from (select * from my_table);`);
if (q.resultSet.next()) {
out["rows_unloaded"] = q.resultSet.getColumnValue("rows_unloaded");
out["input_bytes"] = q.resultSet.getColumnValue("input_bytes");
out["output_bytes"] = q.resultSet.getColumnValue("output_bytes");
} else {
out["Error"] = "Unknown error";
}
return out;
// End of main function
function getQuery(sql){
cmd1 = {sqlText: sql};
var query = new Query(snowflake.createStatement(cmd1));
query.resultSet = query.statement.execute();
return query;
}
$$;
Once you define it, you can use SQL variables as the input if you want:
SET stage_name=CONCAT('#my_stage/my_folder/my_file', '_date.csv.gz');
call copy_to_stage($stage_name);
This won't work. Unfortunately using variables for identifiers does not work for stages. You might need to create a Stored procedure with Dynamic SQL:
https://docs.snowflake.com/en/sql-reference/stored-procedures-usage.html#label-example-of-dynamic-sql-in-stored-procedure
So you can just call this procedure every day or generating a SP with several parameters for the path (Stage), the query which will be executed and the target filename.
I've been going through the test files of https://github.com/DATA-DOG/go-sqlmock to figure out how to create a stored procedure for mocking purposes. I have:
_, err = db.Exec(`
CREATE OR REPLACE FUNCTION val() RETURNS INT AS
$$ SELECT 1; $$
LANGUAGE sql;
`)
if err != nil {
t.Fatal(err)
}
I get:
all expectations were already fulfilled, call to exec 'CREATE OR REPLACE FUNCTION val() RETURNS INT AS $$ SELECT 1; $$ LANGUAGE sql;' query with args [] was not expected
If, instead, I try it with
mock.ExpectExec(`
CREATE OR REPLACE FUNCTION val() RETURNS INT AS
$$ SELECT 1; $$
LANGUAGE sql;
`,
).WillReturnResult(sqlmock.NewResult(0, 0))
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatal(err)
}
I get:
there is a remaining expectation which was not matched: ExpectedExec => expecting Exec which:
- matches sql: 'CREATE OR REPLACE FUNCTION val() RETURNS INT AS $$ SELECT 1; $$ LANGUAGE sql;'
- is without arguments
- should return Result having:
LastInsertId: 0
RowsAffected: 0
I am really confused on how to setup a basic stored procedure.
The sqlmock library works pretty well for this.
But please note that the ExpectExec receives a regular expression in order to match:
// ExpectExec expects Exec() to be called with sql query
// which match sqlRegexStr given regexp.
// the *ExpectedExec allows to mock database response
ExpectExec(sqlRegexStr string) *ExpectedExec
You are sending that function the exact string you expect to receive without any escaping.
To escape the string, add this:
import (
"regexp"
)
And then when adding the expectation, escape your string (note the regexp.QuoteMeta):
mock.ExpectExec(regexp.QuoteMeta(`
CREATE OR REPLACE FUNCTION val() RETURNS INT AS
$$
SELECT 1;
$$
LANGUAGE sql;
`),
).WillReturnResult(sqlmock.NewResult(0, 0))
That way, the escaped regexp will match your exec command.
I'm writing a sql script, and I'd like to use Management Studio to develop the query, and a C# program to run it in production.
My query contains parameters, like so;
SELECT * FROM TABLE
WHERE id = #id
I can feed in a value for #id in the C# program, and that works nicely. However, I also want to declare default values for testing in Management Studio. So I really want to write something like this pseudocode;
if not declared #id
declare #id int
set #id=43
end if
SELECT * FROM TABLE
WHERE id = #id
Is there any way to check to see if a variable name has already been taken?
You can't do exactly what you're after. I'd suggest either:
1) wrap the script up as a sproc and give defaults for the params
2) include a comment block at the top of the script that you can then uncomment when running in SSMS:
/*
-- Default variables for testing
DECLARE #Id INTEGER
SET #Id = 43
*/
SELECT * FROM Table WHERE id = #Id
I've managed to make some progress by marking out the default variables in the script, like so;
/** DEFAULTS **/
declare #id int
set #id = 43
/** END DEFAULTS **/
Then preprocessing the script in my C# program, like so;
script = RemoveBlock(script, "DEFAULTS");
And implementing the function like so;
public static string RemoveBlock(string script, string blockName)
{
if (script == null) { return null; }
var startTag = string.Format("/** {0} **/", blockName);
var endTag = string.Format("/** END {0} **/", blockName);
var startTagIdx = script.IndexOf(startTag);
if (startTagIdx == -1) { return script; }
var endTagIdx = script.IndexOf(endTag, startTagIdx + startTag.Length);
if (endTagIdx == -1) { return script; }
var endOfEndTag = endTagIdx + endTag.Length;
var beforeBlock = script.Substring(0, startTagIdx);
var afterBlock = script.Substring(endOfEndTag);
return beforeBlock + afterBlock;
}
So the C# program runs a version without the variables but with parameters.