I would like to create a table in snowflake, appending the date to the end of its name. What is the best way to do that?
original table = "DB"."SCHEMA"."CLONEME"
desired new table = "DB"."SCHEMA"."CLONEME_20200812BKP"
Tried setting the date variables, but it didn't work.
First attempt:
set var1= (SELECT TO_CHAR(DATE_TRUNC('DAY',CONVERT_TIMEZONE('UTC', CURRENT_DATE())),'YYYYMMDD'));
set var2 = concat('DB.SCHEMA.CLONEME_',$var1);
create table $var2 clone DB.SCHEMA.CLONEME;
-- and got the following error:
-- SQL compilation error: syntax error line 1 at position 13 unexpected '$var2'.
I'd recommend using the IDENTIFIER function:
https://docs.snowflake.com/en/sql-reference/identifier-literal.html
Example:
CREATE OR REPLACE TABLE CLONEME(
src_string VARCHAR(20));
INSERT INTO CLONEME
VALUES('JKNHJYGHTFGRTYGHJ'), ('ABC123'), (null), ('0123456789');
set var1= (SELECT TO_CHAR(DATE_TRUNC('DAY',CONVERT_TIMEZONE('UTC', CURRENT_DATE())),'YYYYMMDD'));
set var2 = concat('CLONEME_',$var1);
SELECT getvariable('VAR1'), getvariable('VAR2');
--20200812 CLONEME_20200812
create table identifier($var2) clone CLONEME;
--Table CLONEME_20200812 successfully created
Related
I've been trying to learn how to use sqlite3 for python 3.10 and I can't find any explanation of how I'm supposed to grab saved data From a database and insert it into a variable.
I'm attempting to do that myself in this code but It just prints out
<sqlite3.Cursor object at 0x0000018E3C017AC0>
Anyone know the solution to this?
My code is below
import sqlite3
con = sqlite3.connect('main.db')
cur = con.cursor()
#Create a table called "Datatable" if it does not exist
cur.execute('''CREATE TABLE IF NOT EXISTS datatable
(Name PRIMARY KEY, age, pronouns) ''')
# The key "PRIMARY KEY" after Name disallow's information to be inserted
# Into the table twice in a row.
name = 'TestName'#input("What is your name? : ")
age = 'TestAge'#input("What is your age? : ")
def data_entry():
cur.execute("INSERT INTO datatable (name, age)")
con.commit
name = cur.execute('select name from datatable')
print(name)
Expected result from Print(name) : TestName
Actual result : <sqlite3.Cursor object at 0x00000256A58B7AC0>
The execute statement fills the cur object with data, but then you need to get the data out:
rows = cur.fetchall()
for row in rows:
print(row)
You can read more here: https://www.sqlitetutorial.net/sqlite-python/sqlite-python-select/
I have a CSV file with the following data:
eno | phonelist | shots
"1" | "['1112223333','6195551234']" | "[[11,12]]"
The DDL statement I have used to create table in snowflake is as follows:
CREATE TABLE ArrayTable (eno INTEGER, phonelist array,shots array);
I need to insert the data from the CSV into the Snowflake table and the method I have used is:
create or replace stage ArrayTable_stage file_format = (TYPE=CSV)
put file://ArrayTable #ArrayTable_stage auto_compress=true
copy into ArrayTable from #ArrayTable_stage/ArrayTable.gz
file_format = (TYPE=CSV FIELD_DELIMITER='|' FIELD_OPTIONALLY_ENCLOSED_BY='\"\')
But when I try to run the code, I get the error:
Copy to table failed: 100069 (22P02): Error parsing JSON:
('1112223333','6195551234')
How to resolve this?
FIELD_OPTIONALLY_ENCLOSED_BY='\"\' base on the row you have that should just be '\"'
select parse_json('[\'1112223333\',\'6195551234\']');
works (the back slashes are to get around the sql parser)
but your output has parens (, ) which is different.
SELECT column2, TRY_PARSE_JSON(column2) as j
FROM #ArrayTable_stage/ArrayTable.gz
file_format = (TYPE=CSV FIELD_DELIMITER='|' FIELD_OPTIONALLY_ENCLOSED_BY='\"\')
WHERE j is null;
will show which values are failing to parse..
failing that you might want to use to_array to parse column2 and thus insert into you table the SELECTED/transformed data, that is failing to auto transform
I'm setting up a masking policy that can be bypassed if the user's current role inherits from a specified role. This can be easily done with the function IS_ROLE_IN_SESSION. The challenge is I want to be able to change the specified role without having to modify the masking policy.
These examples assume the user is using a role other than ACCOUNTADMIN.
I got it to work with a session variable, but this is not secure since I can't control access to session variables:
create or replace table tab as select * from values('personal value') d (data);
set unmask_role = 'PUBLIC';
alter table tab modify column data unset masking policy;
create or replace masking policy hide as (d varchar) returns varchar ->
iff(is_role_in_session($unmask_role),d,replace(d,'personal value','hidden'));
alter table tab modify column data set masking policy hide;
set unmask_role = 'PUBLIC';
select * from tab;
-- Works as expected: shows personal value
set unmask_role = 'ACCOUNTADMIN';
select * from tab;
-- Works as expected: shows hidden
Ideally I would provide the role in a table since I can control access to the contents of a table but I can't get past these errors:
create or replace table unmask_role_tab as select 'PUBLIC' role;
alter table tab modify column data unset masking policy;
create or replace masking policy hide as (d varchar) returns varchar ->
iff(is_role_in_session((select role from unmask_role_tab)),d,replace(d,'personal value','hidden'));
alter table tab modify column data set masking policy hide;
select * from tab;
-- Fails with error:
-- SQL compilation error: error line Check Arg at position 0 invalid argument for function [IS_ROLE_IN_SESSION] unexpected argument [(SELECT UNMASK_ROLE_TAB.ROLE AS "ROLE" FROM UNMASK_ROLE_TAB AS UNMASK_ROLE_TAB)] at position 0,
alter table tab modify column data unset masking policy;
create or replace masking policy hide as (d varchar) returns varchar ->
(select iff(is_role_in_session(role),d,replace(d,'personal value','hidden')) from unmask_role_tab);
alter table tab modify column data set masking policy hide;
select * from tab;
-- Fails with error:
-- SQL compilation error: error line Check Arg at position 0 invalid argument for function [IS_ROLE_IN_SESSION] unexpected argument [UNMASK_ROLE_TAB.ROLE] at position 0,
It is an interesting question as it boils down to how to pass a "non-static" value to function that requires string_literal
IS_ROLE_IN_SESSION
is_role_in_session( '<string_literal>' )
Using view instead of table(if new entries has to be added then view defintion has to be updated, without changing masking policy definition):
create or replace table tab as select * from values('personal value') d (data);
CREATE OR REPLACE VIEW unmask_role_view
AS
SELECT 1 AS col WHERE IS_ROLE_IN_SESSION('PUBLIC')
-- UNION SELECT 1 AS col WHERE IS_ROLE_IN_SESSION('...') -- more entries
;
create or replace masking policy hide as (d varchar) returns varchar ->
case when exists(SELECT 1 FROM unmask_role_view) then d
else replace(d,'personal value','hidden')
end;
alter table tab modify column data set masking policy hide;
select * from tab;
A solution that requires defining all roles that should have access to data. It has one advantage though the roles are listed explicitly. One of the drawbacks is maintenance of this table.
create or replace table tab as select * from values('personal value') d (data);
create or replace table unmask_role_tab as select 'PUBLIC' role;
-- here we compare against CURRENT_ROLE
-- so we need all roles that have access to masked data
create or replace masking policy hide as (d varchar) returns varchar ->
case when exists(SELECT 1 FROM unmask_role_tab u WHERE u.role = CURRENT_ROLE()) then d
else replace(d,'personal value','hidden')
end;
alter table tab modify column data set masking policy hide;
select * from tab;
CREATE MASKING POLICY
CREATE [ OR REPLACE ] MASKING POLICY [ IF NOT EXISTS ] <name> AS
(VAL <data_type>) RETURNS <data_type> -> <expression_ON_VAL>
You can use:
Conditional Expression Functions
Context Functions,
and UDFs to write the SQL expression.
Attempt 1: Standard call
SELECT IS_ROLE_IN_SESSION(u.role) FROM unmask_role_tab u;
-- SQL compilation error: error line Check Arg at position 0 invalid argument
-- for function [IS_ROLE_IN_SESSION] unexpected argument [U.ROLE] at position 0
SELECT IS_ROLE_IN_SESSION(u.role::STRING) FROM unmask_role_tab u;
-- SQL compilation error: error line Check Arg at position 0 invalid argument
-- for function [IS_ROLE_IN_SESSION] unexpected argument [U.ROLE] at position 0
Attempt 2: Create UDF(executiing build SQL is not available)
CREATE OR REPLACE FUNCTION role_check(role_name STRING)
RETURNS boolean
LANGUAGE JAVASCRIPT
AS
$$
var res = snowflake.createStatement({sqlText: 'SELECT IS_ROLE_IN_SESSION(:1)'
, binds:[ROLE_NAME]}).execute()
res.next();
return res.getColumnValue(1);
$$;
SELECT role_check(u.role) FROM unmask_role_tab u;
-- JavaScript execution error: Uncaught ReferenceError:
-- snowflake is not defined in ROLE_CHECK
Attempt 3 SQL UDF(same error like with direct call
CREATE OR REPLACE FUNCTION role_check(role_name STRING)
RETURNS BOOLEAN
LANGUAGE SQL
AS $$
IS_ROLE_IN_SESSION(ROLE_NAME)
$$;
SELECT *, role_check(role) FROM unmask_role_tab;
-- SQL compilation error: error line Check Arg at position 0 invalid argument
-- for function [IS_ROLE_IN_SESSION] unexpected argument [UNMASK_ROLE_TAB.ROLE]
Attempt 4 User-Defined stored procedure:
CREATE OR REPLACE PROCEDURE role_check_proc(role_name STRING)
RETURNS boolean
LANGUAGE JAVASCRIPT
AS
$$
var res = snowflake.createStatement({sqlText: 'SELECT IS_ROLE_IN_SESSION(:1)'
,binds:[ROLE_NAME]}).execute()
res.next();
return res.getColumnValue(1);
$$;
CALL role_check_proc((SELECT role FROM unmask_role_tab));
-- TRUE
-- Works only if table contains single entry
It returns result but stored procedure call cannot be used in masking policy/SQL query call.
Wrapping them with function will not work as it is not possible to call SP from function.
CREATE OR REPLACE FUNCTION role_check(role_name STRING)
RETURNS BOOLEAN
LANGUAGE SQL
AS $$
CALL role_check_proc(ROLE_NAME::STRING)
$$;
I have the following SQL XML in several rows of a table (table is tbldatafeed column in configuration_xml). All of the UserName="" and Password="" is different each time for each row and does not repeat so I can not find/replace off of that. I am trying to write a query that finds all of those and replaces them with Username/Passwords I choose.
<DataFeed xmlns="http://www.tech.com/datafeed/dfx/2010/04" xmlns:plugin="pluginExtensions" Type="TODO" Guid="TODO" UserAccount="DF_LEAN_PopulateCommentsSubForm" Locale="en-US" DateFormat="" ThousandSeparator="" NegativeSymbol="" DecimalSymbol="" SendingNotifications="false" SendJobStatusNotifications="false" RecipientUserIds="" RecipientGroupIds="" RecipientEmailAddresses="" Name="CI_C11.01_Lean-Lean_Reject Comments_A2A" >
<Transporter>
<transporters:ArcherWebServiceTransportActivity xmlns:transporters="clr-namespace:ArcherTech.DataFeed.Activities.Transporters;assembly=ArcherTech.DataFeed" xmlns:out="clr-namespace:ArcherTech.DataFeed;assembly=ArcherTech.DataFeed" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:compModel="clr-namespace:ArcherTech.DataFeed.ComponentModel;assembly=ArcherTech.DataFeed" xmlns:channel="clr-namespace:ArcherTech.DataFeed.Engine.Channel;assembly=ArcherTech.DataFeed" xmlns:engine="clr-namespace:ArcherTech.DataFeed.Engine;assembly=ArcherTech.DataFeed" xmlns:kernel="clr-namespace:ArcherTech.Kernel.Channel;assembly=ArcherTech.Kernel" xmlns="clr-namespace:ArcherTech.DataFeed;assembly=ArcherTech.DataFeed" xmlns:schema="clr-namespace:System.Xml.Schema;assembly=System.Xml" xmlns:xmlLinq="clr-namespace:System.Xml.Linq;assembly=System.Xml" xmlns:domain="clr-namespace:ArcherTech.Common.Domain;assembly=ArcherTech.Common" xmlns:s="clr-namespace:System;assembly=mscorlib" x:Key="transportActivity" SearchType="ReportId" Uri="https://arcs-d" RecordsPerFile="100" ReportID="EC514865-88D5-49CE-A200-7769EC1C2A88" UseWindowsAuth="false" IsWindowsAuthSpecific="false" WindowsAuthUserName="i9XzCczAQ7J2rHwkg6wG9QF8+O9NCYJZP6y5Kzw4be0+cdvUaGu/9+rHuLstU736pnQrRcwmnSIhd6oPKIvnLA==" WindowsAuthPassword="+y0tCAKysxEMSGv1unpHxfg6WjH5XWylgP45P5MLRdQ6+zAdOLSVy7s3KJa3+9j2i83qn8I8K7+1+QBlCJT1E7sLQHWRFOCEdJgXaIr1gWfUEO+7kjuJnZcIEKZJa2wHyqc2Z08J2SKfdCLh7HoLtg==" WindowsAuthDomain="" ProxyName="" ProxyPort="8080" ProxyUsername="" ProxyPassword="" ProxyDomain="" IsProxyActive="False" ProxyOption="None" InstanceName="ARCS-D" TempFileOnSuccessAction="DoNothing" TempFileOnSuccessRenameString="" TempFileOnErrorAction="DoNothing" TempFileOnErrorRenameString="" Transform="{engine:DataFeedBinding Path=Transform}" SessionContext="{engine:DataFeedBinding Path=Session}">
<transporters:ArcherWebServiceTransportActivity.Credentials>
<NetworkCredentialWrapper UserName="TeSZmI1SqO0eJ0G2nDVU+glFg/9eZfeMppYQnPfbeg8=" Password="Slt4VHqjkYscWyCwZK40QJ7KOQroG9OTKr+RGt9bQjE=" />
</transporters:ArcherWebServiceTransportActivity.Credentials>
</transporters:ArcherWebServiceTransportActivity>
</Transporter>
</DataFeed>
I need to be able to set a value and replace it with a query
I have written the following
select #config_xml=configuration_xml from bldatafeed where datafeed_name = 'REMOVED'
update tbldatafeed set configuration_xml.modify(//*:NetworkCredentialWrapper/#UserName)[1] with "abc" ')
where datafeed_name = 'REMOVED'
This does the trick but it only works if I set the "abc" password each time in each area and in some cases I am running this against 50+ rows.
I also tried:
Declare #server nvarchar(max) = 'abc'
Declare #config_xml xml
select #config_xml=configuration_xml from bldatafeed where datafeed_name = 'REMOVED'
update tbldatafeed set configuration_xml.modify(//*:NetworkCredentialWrapper/#UserName)[1] with #server ')
where datafeed_name = 'REMOVED'
The error from this is that: XQuery [tbldatafeed.configuration_xml.modify()]: Top-level attribute nodes are not supported
What I would like to be able to do is set my variable and utilize that as I will be setting this up for multiple rows and unfortunately this error is making this a very difficult problem to solve.
Thanks for any help, this has kept me confused for a bit.
Use the function sql:variable() to use a variable in the XQuery expression.
declare #T table(X xml);
insert into #T values('<X UserName=""/>');
declare #UserName nvarchar(max) = 'abc'
update #T set
X.modify('replace value of (/X/#UserName)[1]
with sql:variable("#UserName")');
I have two databases in MSSQL and by a program inserting values into first table and i want this procedure to insert the same values into second table.
ALTER PROCEDURE [dbo].[ylk_banka_ekle]
#banka_kod varchar(25)
AS
BEGIN
INSERT INTO MikroDB_V15_TEST2.dbo.BANKALAR
(ban_RECid_DBCno, ban_RECid_RECno, ban_SpecRECNo,
ban_iptal, ban_fileid, ban_hidden, ban_kilitli, ban_degisti, ban_CheckSum,
ban_create_user, ban_create_date, ban_lastup_user, ban_lastup_date,
ban_special1, ban_special2, ban_special3, ban_kod)
SELECT
ban_RECid_DBCno, ban_RECid_RECno, ban_SpecRECNo,
ban_iptal, ban_fileid, ban_hidden, ban_kilitli, ban_degisti, ban_CheckSum,
ban_create_user, ban_create_date, ban_lastup_user, ban_lastup_date,
ban_special1, ban_special2, ban_special3, ban_kod
FROM BANKALAR
WHERE (ban_kod = #banka_kod)
UPDATE BANKALAR SET ban_RECid_RECno = ban_RECno WHERE ban_RECno !=
ban_RECid_RECno
END
GO
In sql when i execute this procedure it says only 'Command is successful' but not adding any rows into second database.
Anybody can give an idea where my mistake is?