I'm trying to call a Snowflake procedure directly out of an ETL process in Talend.
I'm using the tSnowflakeRow component since there is no dedicated component for calling Snowflake procedures as we would do for other DB types in Talend.
In the screenshot below, you can see that calling the procedure directly in Snowflake's preview app works well…
Nevertheless, calling it from Talend throws the following exception:
[WARN ]: org.talend.components.snowflake.runtime.SnowflakeRowReader - Query execution failed. Record was not processed.
net.snowflake.client.jdbc.SnowflakeSQLException: Stored procedure execution error: Scoped transaction started in stored procedure is incomplete and it was rolled back.
Row 1: Stored procedure execution error: Scoped transaction started in stored procedure is incomplete and it was rolled back.
The interesting line being
Row 1: Stored procedure execution error: Scoped transaction started in stored procedure is incomplete and it was rolled back.
Here's the DDL of my procedure:
CREATE OR REPLACE PROCEDURE "CALCULATEDURATIONFORDESIGNAABACUS"()
RETURNS VARCHAR(16777216)
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS '
var query1 =
`
INSERT INTO durations (id, odb_created_at, event_id_arrival, event_id_departure,
event_time_arrival, event_time_departure,
card_nr, ticket_type, duration, manufacturer, carpark_id)
WITH cte AS (
SELECT e.id, e.card_nr, e.event_time, e.ticket_type, e.manufacturer, e.carpark_id, e.device_type,
ROW_NUMBER() OVER (ORDER BY e.card_nr, e.carpark_id, e.event_time, e.device_type) AS rn
FROM events e
LEFT JOIN durations d ON d.event_id_arrival = e.id OR d.event_id_departure = e.id
WHERE e.event_time >= (SELECT PROP_VALUE::timestamp FROM properties WHERE prop_key = ''DURATION.LIMIT.DATE'')
AND e.device_type IN (1, 2)
AND event_type = 2
AND e.manufacturer LIKE ''DESIGNA_ABACUS%''
AND d.id IS NULL
)
SELECT
durationseq.nextval,
current_timestamp(),
arrived_entry.id,
departed_entry.id,
arrived_entry.event_time,
departed_entry.event_time,
arrived_entry.card_nr,
arrived_entry.ticket_type,
timestampdiff(second, arrived_entry.event_time, departed_entry.event_time),
arrived_entry.manufacturer,
arrived_entry.carpark_id
FROM (SELECT * FROM cte WHERE cte.device_type = 1) AS arrived_entry
INNER JOIN (SELECT * FROM cte WHERE cte.device_type = 2) AS departed_entry
ON arrived_entry.card_nr = departed_entry.card_nr
AND arrived_entry.carpark_id = departed_entry.carpark_id
AND arrived_entry.rn + 1 = departed_entry.rn
`;
snowflake.execute({ sqlText: query1 });
var query2 = "SELECT PROP_VALUE FROM properties WHERE prop_key = ''DURATION.LIMIT.DAYS''";
var stmt = snowflake.createStatement({ sqlText: query2 });
var resultSet = stmt.execute();
resultSet.next();
var prop_value = resultSet.getColumnValue(1);
var query3 =
`
UPDATE properties
SET PROP_VALUE = (
SELECT dateadd(day, -1 * ${prop_value}, MAX(event_time)) FROM events
WHERE event_time >= (
SELECT PROP_VALUE::timestamp FROM properties WHERE prop_key = ''DURATION.LIMIT.DATE''
)
)
WHERE PROP_KEY =''DURATION.LIMIT.DATE'';
`
stmt = snowflake.createStatement({ sqlText: query3 });
stmt.execute();
return ''true'';
';
Thank you!
As you do not start a transaction explicitly inside the stored procedure, the issue should be related to AUTOCOMMIT. I suggest you check the samples of the scoped transactions:
https://docs.snowflake.com/en/sql-reference/transactions.html#scoped-transactions
Maybe you disable AUTOCOMMIT, or Talend does it.
When AUTOCOMMIT is off, be especially careful combining implicit
transactions and stored procedures. If you accidentally leave a
transaction active at the end of a stored procedure, the transaction
is rolled back.
For example, the following pseudo-code example causes an implicit
ROLLBACK at the end of the stored procedure:
create procedure p1() ...
$$
insert into parent_table ...;
insert into child_table ...;
$$;
alter session set autocommit = false;
call p1; ----- this one will be rolled back
commit work;
Better late than never :)
Add an explicit transaction start / commit / rollback into the sproc for it to execute nicely when autocommit is disabled, like this:
create procedure p1() ...
LANGUAGE JAVASCRIPT
$$
snowflake.createStatement({sqlText: "BEGIN TRANSACTION"}).execute();
-- insert into or whatever...
snowflake.createStatement({sqlText: "COMMIT"}).execute();
return <what you need to return>;
$$;
Related
Hi I have one doubt in snowflake
how to use variable result in the next sselect session.
I want stored one statement result in variable and that variable result need to call another statment in snowflake server .
use schema test.public;
create or replace procedure empresult()
returns table ()
language sql
as
$$
set empresult=(select deptid from TEST.PUBLIC.DEPT where deptid=10)
declare res resultset(
select * from emp where deptno in ($empresult)
)
begin
return table(res);
end;
$$;
above statment getting error.
Syntax error: unexpected 'declare'. (line 4)
please tell me how to write stored procedure query to call the one statment result into another session select statment task.
The sample code is missing colons and the correct syntax for variables.
One possible fix that works:
create or replace procedure empresult()
returns table ()
language sql
as
$$
declare
empresult integer;
begin
select deptid into :empresult from DEPT where deptid=10;
let res resultset := (select * from emp where deptno = :empresult);
return table(res);
end;
$$;
call empresult();
Staging data:
create temp table dept as select 10 deptid;
create temp table emp as select 'a' a, 'b' b, 10 deptno;
how to run simple select query and get reusult in snowflakes procedure?
CREATE OR REPLACE PROCEDURE COARSDB0P_DB.COUNT_MISMATCH()
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
AS
$$
var command ="SELECT b.TABLE_CATALOG,b.TABLE_SCHEMA,b.TABLE_NAME,b.TABLE_TYPE,b.ROW_COUNT as Dev_COUNT,a.ROW_COUNT as UAT_COUNT FROM TABLE A ,information_schema.tables B where b.TABLE_NAME=a.TABLE_NAMES and b.TABLE_SCHEMA=a.TABLE_SCHEMA and b.TABLE_TYPE=a.TABLE_TYPE and TRY_CAST(b.ROW_COUNT as NUMBER) != TRY_CAST(a.ROW_COUNT as NUMBER);"
var cmd1_dict = {sqlText: command};
var stmt = snowflake.createStatement(cmd1_dict);
var rs = stmt.execute();
rs.next();
var output = rs.getColumnValue();
return output;
$$;
I need to return the actualy output of mentioned SLECT query.
See Returning tabular data from a stored procedure. Note this is for a SQL stored procedure, not Javascript. You will want something like:
create procedure ...
returns table (catalog varchar, schema varchar, ...)
declare
query varchar;
res resultset;
begin
-- build the dynamic SQL query
query := 'select ...'
res := (execute immediate :query);
return table (res);
end;
You can use Tabular SQL UDFs to return the result:
https://docs.snowflake.com/en/developer-guide/udf/sql/udf-sql-tabular-functions.html#calling-a-sql-udtf
I'm trying to use a temp table to update a database table in a stored procedure.
My client was throwing an error about an illegal null value, so I tried testing within SSMS and it told me that the name of the temp table was invalid (regardless of what I named it).
If I run the beginning and change the INSERT INTO SERVICE SELECT S.* to simply SELECT S.* and run the code after Declaring and Defining #OldServicesString (so I can leave out the ALTER/CREATE PROCEDURE line) it runs exactly as expected.
Here's the beginning of the SP:
ALTER PROCEDURE [dbo].[app_CreateNewServicesOldFSD] (#OldServicesStr nvarchar(max)) AS
SET NOCOUNT ON
DECLARE #User char(10) = (SELECT TOP 1 CREATED_BY_USER FROM BATCHLOG WHERE BPROCESS_ID = 3 ORDER BY ID DESC);
DECLARE #LastOldService int = (SELECT MAX(ID) FROM SERVICE);
SELECT TOP 0 * INTO #Service FROM SERVICE;
ALTER TABLE #Service
DROP COLUMN RECNUM;
INSERT INTO #Service exec dbo.app_NewServicesOldFSD
;WITH cteOldFSDsToCreate AS (
SELECT JobID.Item JobID, CONVERT(date,ServDate.Item,103) ServDate
FROM dbo.SplitStringToTable(#OldServicesStr,',',2) JobID
INNER JOIN dbo.SplitStringToTable(#OldServicesStr,',',2) ServDate ON JobID.Rw = ServDate.Rw AND JobID.Cl = 0 AND ServDate.Cl = 1
)
INSERT INTO SERVICE SELECT S.*
FROM #Service S
INNER JOIN cteOldFSDsToCreate N ON N.JobID = S.JOB_ID AND N.ServDate = S.DATE
DROP TABLE #Service
A useable and likely #OldServicesStr could be '11428,23/07/2019,11429,23/07/2019,15186,5/10/2019'
To test it in SSMS I opened a new Query and typed in
exec app_CreateNewServicesOldFSD '11428,23/07/2019,11429,23/07/2019,15186,5/10/2019'
And got the following error:
Warning: Null value is eliminated by an aggregate or other SET operation.
Msg 208, Level 16, State 0, Procedure app_CreateNewServicesOldFSD, Line 65 [Batch Start Line 7]
Invalid object name '#Service'.
Completion time: 2020-11-20T20:36:57.1356921+11:00
I know that is not Your case but in the sake of not forget, there is also a limitation using Biztalk WCF-Custom adapter with SQL Bindings and temp tables.
According to this site :
http://thoughtsofmarcus.blogspot.com/2010/11/calling-stored-procedures-from-biztalk.html
You need to SET FMTONLY OFF before creating temp tables and inserting into them and
SET FMTONLY ON before selecting from them as in example from below.
ALTER PROCEDURE [dbo].[FetchTestData]
(#a4 varchar(4))
AS
DECLARE #FmtOnlyIsSet bit = 0
IF (1=0) BEGIN SET #FmtOnlyIsSet = 1 END
IF #FmtOnlyIsSet = 1
SET FMTONLY OFF
SELECT t1, t2, t3 INTO #temp01 FROM Table_1
IF #FmtOnlyIsSet IS NULL
SET FMTONLY ON
SELECT t1, t2, t3 FROM #temp01
RETURN
I have a problem with a JOB execution, the case is the next:
I have a job with category (Uncategorized Local) than have 1 step with "Transact SQL script" type.
The transact code is:
DECLARE #fecha_inicio DATETIME,
#fecha_fin DATETIME,
#fecha_inicio_p VARCHAR(10),
#fecha_fin_p VARCHAR(10),
#vc_mensaje_error AS VARCHAR(255)
BEGIN TRY
SELECT #fecha_inicio_p = par_valor1
FROM parametro
WHERE par_codigo_dominio = '000213' and par_codigo = 'FI'
SELECT #fecha_fin_p = par_valor1
FROM parametro
WHERE par_codigo_dominio = '000213' and par_codigo = 'FF'
SELECT #fecha_inicio = CONVERT(DATETIME, #fecha_inicio_p, 103)
SELECT #fecha_fin = DATEADD(D,+1,CONVERT(DATETIME, #fecha_fin_p, 103))
EXEC usp_p_broad_maestra_docvar #fecha_inicio,#fecha_fin,'E'
END TRY
BEGIN CATCH
select #vc_mensaje_error = ERROR_MESSAGE();
raiserror(#vc_mensaje_error,16,1)
END CATCH
The stored procedure does:
Truncate a table
Insert Into Select... UNION SELECT.... UNION SELECT...
call a job to use ssis to generate a file
Write Log
The problem is than when I execute the job, it never terminates and I have to cancel execution.
But when I don't use a stored procedure and the code put into JOB the job terminates ok.
Please your help maybe there are differences in the execution a stored procedure into job versus execute stored procedure code into job.
Thanks
I am learning SQL Server stored procedures by way of writing a C# application for holiday bookings.
One of the stored procedures updates records within a table based if that record exists in a select query which has relationships with other tables and the table I am trying to update. The stored procedure executes fine and returns a status of 0 which implies nothing went wrong but the records are not being updated with the new status field value.
If I take the select query out of the stored procedure and run this by it's self it does return the records I expected to be updated with the new status column value. I am a bit lost, is it something to do with locking in that I cannot update a table where I am also selecting from the same table?
The stored procedure code is as follows:
SET ANSI_NULLS ON
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[spDeletetblBalancesUsingTypeURNAndRoleURN]
#ABSENCE_TYPE_URN uniqueidentifier,
#ROLE_URN uniqueidentifier,
#LAST_USER_SAVED uniqueidentifier
AS
BEGIN
BEGIN TRY
BEGIN TRANSACTION
UPDATE [dbo].[tblBalances]
SET
[STATUS] = 0
WHERE
[tblBalances].[URN] IN
(
SELECT
[tblBalances].[URN]
FROM
[tblRoles] LEFT JOIN [tblGroupsAssignedToAbsenceTypes] ON [tblRoles].[URN] = [tblGroupsAssignedToAbsenceTypes].[ROLE_URN]
LEFT JOIN [tblUsersAssignedToRoles] ON [tblGroupsAssignedToAbsenceTypes].[ROLE_URN] = [tblUsersAssignedToRoles].[ROLE_URN]
LEFT JOIN [tblBalances] ON [tblUsersAssignedToRoles].[USER_URN] = [tblBalances].[USER_URN]
WHERE
[tblRoles].[URN] = #ROLE_URN
AND
[tblGroupsAssignedToAbsenceTypes].[ABSENCE_TYPE_URN] = #ABSENCE_TYPE_URN
AND
[tblUsersAssignedToRoles].[STATUS] = 1
AND
[tblGroupsAssignedToAbsenceTypes].[STATUS] = 1
AND
[tblBalances].[ABSENCE_TYPE_URN] = #ABSENCE_TYPE_URN
AND
[tblBalances].[STATUS] = 1
AND
ISNULL([tblBalances].[EXPIRY_DATE], '1900-01-01 00:00:00.000') <> '1900-01-01 00:00:00.000'
)
COMMIT TRANSACTION
BEGIN TRANSACTION
INSERT INTO [dbo].[tblAudit]
([TRANSACTION_USER], [TABLE_NAME], [CURRENT_VALUES], [NEW_VALUES])
VALUES
(#LAST_USER_SAVED, 'tblBalances', 'ABSENCE_TYPE_URN : ' + CONVERT(nvarchar(36), #ABSENCE_TYPE_URN) + CHAR(13) + CHAR(10)
+ 'ROLE_URN : ' + CONVERT(nvarchar(36), #ROLE_URN)
, 'Delete Record')
COMMIT TRANSACTION
SELECT 0
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION
SELECT 1
END CATCH
END