asynchronous stored procedure in snowflake - snowflake-cloud-data-platform

I need to have stored procedure in snowflake that has to be executed in asynchronous mode .so if call is made is to the stored procedure ,it should immediately return queryid and close the session. The actual execution should happen independently. I tried implementing the wrapper stored procedure that calls asynchronous stored procedure but doesn't seem to work.

You can execute a stored procedure asynchronous from your Java app:
https://docs.snowflake.com/en/user-guide/jdbc-using.html#examples-of-asynchronous-queries
I created a simple stored procedure which waits 5 seconds and then return "Success". Here is a sample Java code to call the stored procedure, and get the query ID (while the stored procedure is still working):
public static void main(String[] args) throws SQLException, InterruptedException {
try (Connection con = getConnection()) {
try (var statement = con.createStatement()) {
var sql_command = "CALL TEST_SP()";
var resultSet = statement.unwrap(SnowflakeStatement.class).executeAsyncQuery(sql_command);
var queryID = resultSet.unwrap(SnowflakeResultSet.class).getQueryID();
System.out.printf("Calling TEST_SP() - Query ID is %s%n", queryID);
var queryStatus = resultSet.unwrap(SnowflakeResultSet.class).getStatus();
while (queryStatus == QueryStatus.RUNNING || queryStatus == QueryStatus.RESUMING_WAREHOUSE) {
Thread.sleep(1000); // 1000 milliseconds.
System.out.printf( "Waiting for 1 second...\n");
queryStatus = resultSet.unwrap(SnowflakeResultSet.class).getStatus();
}
if (resultSet.next()) System.out.println(resultSet.getString(1));
}
}
}
Output:
Calling TEST_SP() - Query ID is 019b08ac-32c3-3514-0000-b4550004911a
Waiting for 1 second...
Waiting for 1 second...
Waiting for 1 second...
Waiting for 1 second...
Waiting for 1 second...
Success
Of course, you don't need to wait until the end of the stored procedure, this code just shows that how the stored procedure is executed asynchronously.

Related

How to return result from a stored procedure to TASK RETURN_VALUE in snowflake?

I would like to return logging and status messages from a stored procedure to the TASK that calls it.
create or replace procedure status_return()
returns string not null
language javascript
as
$$
var result_status = 'The return status and debug information in string format';
return result_status; // Statement returned for info/debug purposes
$$;
I would like to pass the result from stored procedure call status_return() back to the task
-- Create a task that calls the stored procedure every hour
create or replace task call_SP
warehouse = SMALL
schedule = '1 minute'
as
call status_return();
When I execute TASK_HISTORY to view RETURN_VALUE is always empty.
select *
from table(information_schema.task_history(SCHEDULED_TIME_RANGE_START => dateadd(hours, -5, current_timestamp()) ,
TASK_NAME => 'call_sp'));
How can I view the result of a stored procedure in task_history for SUCCESS, FAILURE, or ERRORS?
I have tried creating a task in the following way, but I was unsuccessful and it return with errors.
create or replace task call_SP
warehouse = EDS_SMALL
schedule = '1 minute'
as
call system$set_return_value(call status_return());
Can I use Javascript in Tasks? To store the result of a stored procedure call into a variable and return it back to the TASK result
In order to be able to get a RETURN_VALUE in your TASK_HISTORY you have to set the return_value in your stored procedure using call system$set_return_value().
Examples can be found in snowflake documentation.
This is what it should looks like if you want the return_value field of the task_history to return your result status var when your task is launched :
create or replace procedure status_return()
returns string not null
language javascript
as
$$
var result_status = 'The return status and debug information in string format';
var rv_stmt = snowflake.createStatement({sqlText:`call system$set_return_value('` + result_status + `');`});
var rv_res = rv_stmt .execute(); // Set return_value
return result_status; // Statement returned for info/debug purposes
$$;

How to schedule a daily sql script in snowflake db

How to schedule a sql script in the snowflake database to run every day, and set the output file name to include the current date. E.g. if the code ran today then the file name should be 20200906*****.csv.gz, similary for tomorrow 20200907******.csv.gz.
You could use Snowflake TASKS in order to schedule execution of SQL statements.
Task can execute a single SQL statement, including a call to a stored procedure.
Tasks run according to a specified execution configuration, using any combination of a set interval and/or a flexible schedule using a subset of familiar cron utility syntax.
For your goal I would create a Stored Procedure (so that you could use variables for managing changing filename and for any more complex things).
SF Doc: https://docs.snowflake.com/en/sql-reference/sql/create-task.html
--create a new task that executes a single SQL statement based on CRON definition
CREATE TASK mytask_hour
WAREHOUSE = mywh
SCHEDULE = 'USING CRON 0 9-17 * * SUN America/Los_Angeles'
TIMESTAMP_INPUT_FORMAT = 'YYYY-MM-DD HH24'
AS
INSERT INTO mytable(ts) VALUES(CURRENT_TIMESTAMP);
--create a new task that executes a Stored Procedure every hour
create task my_copy_task
warehouse = mywh
schedule = '60 minute'
as
call my_unload_sp();
After creating a task, you must execute ALTER TASK … RESUME in order to enable it.
Use SHOW TASKS to check your task's definition/configuration and then query TASK_HISTORY in order to check executions.
Your Snowflake JS Stored Procedure could be something like this:
create or replace procedure SP_TASK_EXPORT()
RETURNS VARCHAR(256) NOT NULL
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
as $$
function getToday_yyyymmdd()
{
var v_out_Today;
rs = snowflake.execute ( { sqlText: `SELECT to_char(current_date,'yyyymmdd');` } );
if( rs.next())
{
v_out_Today = rs.getColumnValue(1); // get current date as yyyymmdd
}
return v_out_Today;
}
var result = new String('Successfully Executed');
var v_Today = getToday_yyyymmdd();
try {
var sql_command = `copy into #unload_gcs/LH_TBL_FIRST` + v_Today + `.csv.gz from ........`;
var stmt = snowflake.createStatement({sqlText: sql_command});
var res = stmt.execute();
}
catch (err) {
result = "Failed: Code: " + err.code + " | State: " + err.state;
result += "\n Message: " + err.message;
result += "\nStack Trace:\n" + err.stackTraceTxt;
}
return result;
$$;
Before creating your task and schedule it, test your Stored Procedure invoking it:
call SP_TASK_EXPORT();

Call stored procedure specifying only parameters with a value

In an instance of SQL Server 2016 I have a stored procedure with dozens of parameters. For example:
CREATE PROCEDURE spName (
#par1 INT = NULL,
#par2 VARCHAR(10) = NULL,
....
....
#par98 INT = NULL,
#par99 INT = NULL,
) AS
BEGIN
....
....
END
I have a client written in C# that calls the stored procedure specifying only the parameters with a value. Ex:
SqlCommand cmd = new SqlCommand();
cmd.CommandText = "spName";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Connection = dbConn;
cmd.Parameters.Add(new SqlParameter("par1", "val1"));
cmd.Parameters.Add(new SqlParameter("par47", "val47"));
...
cmd.ExecuteNonQuery();
It works perfectly! So, the procedure is executed and only the 2 parameters (par1 and par47) have a value. Other parameters maintain the default value (NULL).
I would do the same from a Java client using Microsoft JDBC driver 6.2.
I specify the parameters with List<Map<String, Object>>, so a list of couple parameterName-->parameterValue. The following method builds the PreparedStatement object:
private CallableStatement prepareStatement(String spName, Map<String, ?> parameters) throws SQLException {
setupConnection();
CallableStatement stmt = null;
try {
stmt = conn.prepareCall(getSpCallString(spName, parameters));
if (parameters != null) {
for (String parName : parameters.keySet())
stmt.setObject(parName, parameters.get(parName));
}
} catch (SQLException e) {
ApplicationLogging.severe("Cannot prepare callable statement", e);
throw e;
}
return stmt;
}
The method getSpCallString() generates a string of the type { call spName ?,?, ... , ? } with a number of ? as the number of parameters with a value passed to the procedure, so not all 99 parameters. If I have 2 parameter it generates the string { call spName ?,? }.
By passing for example par15=val15 and par47=val47 it raises the following exception:
com.microsoft.sqlserver.jdbc.SQLServerException: The index 2 is out of range.
I could resolve this putting in the call command the same number of ? as the number of parameter of the stored procedure but... I don't know the number of parameters for each stored procedure (and their position)!
In C# this is simply resolved because the parameters are assigned only with their name, so the number and the order of parameters can be really a black box.
Can I do this in some way in Java?
This is a confirmed deficiency in the current implementation of named parameter support for CallableStatement in the mssql-jdbc driver. Despite section 13.3.2 of the JDBC 4.2 specification stating ...
Named parameters can be used to specify only the values that have no default value.
... we seem to be required to provide a parameter placeholder for every possible parameter, and there doesn't appear to be a way to specify DEFAULT for the parameters we might otherwise simply omit.
As a workaround we could use code like this
public static ResultSet executeStoredProcedureQuery(
Connection conn, String spName, Map<String, Object> paramItems)
throws SQLException {
StringBuffer sqlBuf = new StringBuffer("EXEC ");
sqlBuf.append(spName);
int paramCount = 1;
for (String paramName : paramItems.keySet()) {
sqlBuf.append(
(paramCount++ > 1 ? ", " : " ") +
(paramName.startsWith("#") ? "" : "#") + paramName + "=?");
}
String sql = sqlBuf.toString();
myLogger.log(Level.INFO, sql);
// e.g., EXEC dbo.BreakfastSP #helpings=?, #person=?, #food=?
PreparedStatement ps = conn.prepareStatement(sql);
paramCount = 1;
for (String paramName : paramItems.keySet()) {
ps.setObject(paramCount++, paramItems.get(paramName));
}
return ps.executeQuery();
}
which we could call like this
// test data
Map<String, Object> paramItems = new HashMap<>();
paramItems.put("#person", "Gord");
paramItems.put("#food", "bacon");
paramItems.put("#helpings", 3);
//
ResultSet rs = executeStoredProcedureQuery(conn, "dbo.BreakfastSP", paramItems);
If using a third party library to facilitate calling such procedures is an option for you, then jOOQ certainly helps via its code generator for stored procedures, which generates stubs for each of your procedures, making such calls type safe. It includes support for:
Table valued functions
Table valued parameters
Defaulted parameters
In / Out parameters
Optional return value of procedures
Fetching undeclared update counts and result sets
Much more
In your case, you could write:
Spname sp = new Spname();
sp.setPar1("val1");
sp.setPar47("val47");
sp.execute(configuration); // The object containing your JDBC connection
sp.getResults(); // The result set(s) and update counts, if any
Behind the scenes, a JDBC CallableStatement is created, just like you would do manually:
try (CallableStatement s = c.prepareCall(
"{ ? = call [dbo].[spName] (#par1 = ?, #par47 = ?) }"
)) {
// Get the optional procedure return value that all procedures might return
s.registerOutParameter(1, Types.INTEGER);
s.setString(2, "val1");
s.setString(3, "val47");
s.execute();
// Lengthy procedure to fetch update counts and result set(s)
}
See this article if you want to generically fetch update counts and result set(s) with JDBC.
Disclaimer: I work for the company behind jOOQ.

JDBC SQL Server raised error handling in multi statement stored procedures

I have a multi-statement stored procedure that first performs a select and then raises an error if certain conditions are met.
The raise error in the stored procedure doesn't cause a JDBC SQLException like I expect however.
If I remove the SELECT, then it works fine. The same type of behavior occurs with the print statement.
I have multiple other ways to handle this, but for future reference I was wondering if there was a way to check if raised errors do exist.
The way the SQL server protocol works, you first need to process the result set produced by the select, and then move to the next result to get the exception.
To process all results (result sets, update counts and exceptions), you need do something like:
CallableStatement csmt = ...;
boolean isResultSet = cstmt.execute();
do {
if (isResultSet) {
// process result set
try (ResultSet rs = csmst.getResultSet()) {
while(rs.next()) {
// ...
}
}
} else {
int updateCount = rs.getUpdateCount();
if (updateCount == -1) {
// -1 when isResultSet == false means: No more results
break;
} else {
// Do something with update count
}
}
isResultSet = cstmt.getMoreResults();
} while(true);
When the execution of the stored procedure reaches the exception, this will also report the exception to your java application (iirc from getMoreResults()).

Multiple result sets and return value handling while calling sql server stored procedure from Groovy

I have MS SQL Server stored procedure (SQL Server 2012) which returns:
return value describing procedure execution result in general (successfull or not) with return #RetCode statement (RetCode is int type)
one result set (several records with 5 fields each)
another result set (several records with 3 fields each)
I calling this procedure from my Groovy (and Java) code using Java's CallableStatement object and I cannot find right way to handle all three outputs.
My last attempt is
CallableStatement proc = connection.prepareCall("{ ? = call Procedure_Name($param_1, $param_2)}")
proc.registerOutParameter(1, Types.INTEGER)
boolean result = proc.execute()
int returnValue = proc.getInt(1)
println(returnValue)
while(result){
ResultSet rs = proc.getResultSet()
println("rs")
result = proc.getMoreResults()
}
And now I get exception:
Output parameters have not yet been processed. Call getMoreResults()
I tried several approaches for some hours but didn't find correct one. Some others produced another exceptions.
Could you please help me with the issue?
Thanks In Advance!
Update (for Tim):
I see rc while I launched code:
Connection connection = dbGetter().connection
CallableStatement proc = connection.prepareCall("{ ? = call Procedure_Name($param_1, $param_2)}")
boolean result = proc.execute()
while(result){
ResultSet rs = proc.getResultSet()
println(rs)
result = proc.getMoreResults()
}
I see rc as object: net.sourceforge.jtds.jdbc.JtdsResultSet#1937bc8

Resources