All,
I am trying to create an generic copy command into snowflake using a JavaScript procedure
var my_sql_command = "select * from vw_etl_src order by 2 asc";
-- sql will return something to this effect
-- copy into etl_db.eod_data from #myint_stage/cm01jun2022bhav.csv.gz;
var statement1 = snowflake.createStatement( {sqlText: my_sql_command} );
-- get the result into a variable and
while (result_set1.next()) {
var res = result_set1.getColumnValue(1);
-- I was hoping to executing the copy command
snowflake.createStatement( { sqlText: res
} ).execute();
}
I am calling this and although it runs no data is loaded . I did truncate the table manually.
I have set EXECUTE AS OWNER.
Any pointers
Thanks
There is no reason for this block not to work. All I see is, that result_set1 is not defined, but it's probably because you didn't share the whole script.
var result_set1 = statement1.execute();
You also didn't share any errors, so I assume that JS procedure is completed without any error. In this case, check the query history to see how your COPY command was executed:
https://docs.snowflake.com/en/user-guide/ui-history.html#overview-of-features
https://docs.snowflake.com/en/sql-reference/functions/query_history.html
You may also want to check COPY_HISTORY:
https://docs.snowflake.com/en/sql-reference/functions/copy_history.html
Related
I am trying to create a stored procedure to copy from the external stage (s3 bucket) and use a pattern for the file name. The pattern is based on the concatenated current date but I need to set a variable to use as a pattern. Is it possible to do something like this?
CREATE OR REPLACE PROCEDURE test_copy()
RETURNS STRING
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
SET my_Date=(select concat('.*', regexp_replace(current_date(),'-',''), '.*.parquet' );
var sql_command = '
COPY INTO table1
FROM '#s3bucket'
(file_format => PARQUET, pattern=>$my_Date)
);
'
snowflake.execute(
{
sqlText: sql_command
});
return "Successfully executed.";
$$;
As you can generate the current date in javascript, why not create my_Date purely in javascript?
You then need to create sql_command by concatenating the required strings and variables together
I am creating a snowflake stored procedure where I need to store the output if the SQL query into an array , whose values will be later used to create SQL Command which will be the output. Please suggest how do I store the output in an array.
Please refer to the doc on how to setup SP in Snowflake:
https://docs.snowflake.com/en/sql-reference/stored-procedures-api.html
https://docs.snowflake.com/en/sql-reference/stored-procedures-usage.html
There are lots of examples over there.
You just need to know how to query Snowflake and get results, and the rest are just standard Javascript.
Example as below:
create or replace procedure test_sp()
returns string
language javascript
as
$$
var my_data = [];
var stmt = snowflake.createStatement({sqlText: "SELECT * FROM table_name" });
var rs = stmt.execute();
while (rs.next()) {
my_data.push(rs.getColumnValue(1));
my_data.push(rs.getColumnValue(2));
}
// then you can use my_data array in later part of your code
return '';
$$;
I am trying to create an UDTF where I am planning to pass the dynamic where clause as an argument to the UDTF
CREATE OR REPLACE FUNCTION FUNCTION_NAME(where_clause VARCHAR)
RETURNS TABLE ()
LANGUAGE JAVASCRIPT
AS
$$ var sql_command=SELECT COL1, COL2 FROM TABLE_1 JOIN TABLE_2 ON ....
+where_clause+ GROUP BY ...;
var stmt = snowflake.createStatement( {sqlText: sql_command} );
var resultSet = stmt.execute(); resultSet.next();
$$
Is this something possible and also I want to be able to handle quotes in the arguments during calling of the function
for example if the call is:
SELECT * FROM TABLE(FUNCTION_NAME('WHERE COL_1='value''));
How do I handle quotes during calling of the function. Thanks for the help in advance
I wanted to underline the answer of Felipe. JavaScript UDTFs do not support "Stored Procedures API".
https://docs.snowflake.com/en/sql-reference/stored-procedures-api.html
So you can not create and run a dynamic SQL inside of a JavaScript UDTF. It must have "processRow" callback function to process the values coming as parameters and return them as rows.
https://docs.snowflake.com/en/sql-reference/udf-js-table-functions.html#basic-hello-world-examples
You can create a stored procedure to run a dynamic SQL and return the result as an array/json, but it won't be effective.
For your original question, you just need to escape the quote by repeating it or using slash:
call FUNCTION_NAME('WHERE i=''1''');
call FUNCTION_NAME('WHERE i=\'1\'');
If have a
DROP VIEW IF EXISTS mydatabase.myschema.myname;
CREATE OR REPLACE TABLE mydatabase.myschema.myname AS ...
that fails with error code 2203 SQL compilation error: Object found is of type 'TABLE', not specified type 'VIEW'..
My intention was to create a script to "convert" a set of existing views into tables (updated periodically via tasks). I wanted the script to be repeteable, so I thought I could DROP VIEW IF EXISTS xxx to drop the view if it exists but it seems that this will fail if there is already a table of the same name. So first time the script runs ok, it drops the view and creates the table but if I run the script again it will fail because now there is table with that same name.
So is there any way to ignore the error in the DROP VIEW IF EXISTS xxx or just to run the command if there is a VIEW with that name?
You have a number of options.
You can have your script read from the INFORMATION_SCHEMA to get a list of views and to delete. This SQL gets a list of all views except in the INFORMATION_SCHEMA.
select * from INFORMATION_SCHEMA.VIEWS where TABLE_SCHEMA <> 'INFORMATION_SCHEMA';
If you just want to drop the view names and avoid running into errors, here's a stored procedure you can call to try dropping a view without generating an error:
create or replace procedure DropView(viewName string)
returns string
language JavaScript
execute as OWNER
as
$$
var sql_command =
'drop view ' + VIEWNAME;
try {
var stmt = snowflake.createStatement( {sqlText: sql_command} );
var resultSet = stmt.execute();
while (resultSet.next()) {
outString = resultSet.getColumnValue('status');
}
}
catch (err) {
outString = err; // Return a success/error indicator.
}
return outString;
$$;
If you want to loop through every database and schema in the entire account, I wrote a stored procedure to do that. It's designed for dependency checking on all views, but could be modified to delete them too.
https://snowflake.pavlik.us/index.php/2019/10/14/object-dependency-checking-in-snowflake
My suggestion would be to create a stored procedure that loops through all of your views and creates tables from them. In that stored procedure, you could check to see if the object exists already as a table and skip that object.
Using SQL Server 2008 R2 I am getting deadlocks when the same update statement (with different parameters) is running concurrently. Here is the deadlock graph (sorry cannot post images on here yet):
http://i.stack.imgur.com/E6JBK.png
And here is the actual execution plan:
http://i.stack.imgur.com/emm9i.png
The update is like this:
exec sp_executesql N'UPDATE mapping.IssuerAlternateName
SET
UseCount = UseCount + 1,
MostRecentlyAppeared = GETDATE(),
MostRecentlyAppearedUnderlyingAssetName = #p1
WHERE ID = #p0
',N'#p0 int,#p1 nvarchar(4000)',#p0=1234,#p1=N'blah blah blah'
If I have understood things correctly we are trying to read and write from the same index (PK_IssuerAlternateName_1).
Is there any way to resolve this? I was wondering if adding an additional index to the primary key and using WITH INDEX might fix it by stopping the read of PK_IssuerAlternateName_1 (sorry the full name is truncated in the execution plan screenshot).
Or is the best option just to live with this and retry the transaction, which is how the error is currently handled in .NET client. It is certainly successful on retry, but it would be good to avoid the deadlock if possible.
Thanks
In situations similar to this, I have used the UPDLOCK hint to let the database know I intend to update this row. It is not implied by the UPDATE statement. Without the lock hint, it will first obtain a "shared" lock, and then try to escalate. However, this causes deadlocks in certain scenarios.
You will need to do this within your own TransactionScope to ensure everything works correctly.
var sql = #"UPDATE mapping.IssuerAlternateName with (UPDLOCK)
SET
UseCount = UseCount + 1,
MostRecentlyAppeared = GETDATE(),
MostRecentlyAppearedUnderlyingAssetName = #p1
WHERE ID = #p0";
var options = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted // don't use Serializable!
};
using (var scope = new TransactionScope(TransactionScopeOption.RequiresNew, options))
{
using (var context = new YourDbContext())
{
// execute your command here
}
}