DBT : For loop for 2 variable simultaneously - snowflake-cloud-data-platform

My Sample code is :
{%- set VARIABLE = ["123","1234"] -%}
{%- set VARIABLE1 = ["abc","def"] -%}
with FINAL as (
{% for (col,col1) in (VARIABLE,VARIABLE1) %}
Select
current_date as date,
{{ col }} as test1,
'{{ col1 }}' as test2
UNION ALL
{% endfor %}
select current_date as date1,
'Temp' as test1,
'Temp' as test2
) select * from FINAL
Output I am getting :
create or replace view Test.CRS_Foundation.test_for
as (
with FINAL as (
Select
current_date as date,
123 as test1,
'1234' as test2
UNION ALL
Select
current_date as date,
abc as test1,
'def' as test2
UNION ALL
select current_date as date1,
'Temp' as test1,
'Temp' as test2
) select * from FINAL
);
Expected Output :
create or replace view Test.CRS_Foundation.test_for
as (
with FINAL as (
Select
current_date as date,
123 as test1,
'abc' as test2
UNION ALL
Select
current_date as date,
1234 as test1,
'def' as test2
UNION ALL
select current_date as date1,
'Temp' as test1,
'Temp' as test2
) select * from FINAL
);
Is there anything i missed in code, code is running fine but output is not as expected
Is there anything i missed in code, code is running fine but output is not as expected
Is there anything i missed in code, code is running fine but output is not as expected

You want to use zip, which is a function from the Python standard library that takes two sequences and returns a sequence of tuples, with one element from each input sequence. zip is available in the dbt jinja context (docs).
With zip, your code becomes:
{%- set VARIABLE = ["123","1234"] -%}
{%- set VARIABLE1 = ["abc","def"] -%}
with FINAL as (
{% for (col,col1) in zip(VARIABLE,VARIABLE1) %}
Select
current_date as date,
{{ col }} as test1,
'{{ col1 }}' as test2
UNION ALL
{% endfor %}
select current_date as date1,
'Temp' as test1,
'Temp' as test2
) select * from FINAL

Related

How to replace loop results inside snowflake.execute statement?

I am looking for a way to iterate over result set that has columns id1, id2 and pass the resultset column values to be used in inner query as shown below.
Variables of resultset
column1
column2
Eg:
select * from months cross join (select column1 AS "id1",column2 AS "id2")
Select "GEOGRAPHY" from mytable WHERE id1 = column1 and id2 = column2
However when i execute the stored procedure , i encounter following error. Any pointers much appreciated . Thank you
Error:
Execution error in stored procedure TEST_PROC_STMT: SQL compilation error: error line 8 at position 81 invalid identifier 'COLUMN2' At Snowflake.execute, line 14 position 10
Full Query
create or replace procedure TEST_PROC_STMT()
returns varchar not null
language javascript
EXECUTE AS CALLER
as
$$
var distinct_sql_command = "select distinct id1, id2 from mytable ";
var statement1 = snowflake.createStatement( {sqlText: distinct_sql_command} );
var result_set1 = statement1.execute();
// Loop through the results, processing one row at a time...
while (result_set1.next()) {
var column1 = result_set1.getColumnValueAsString(1);
var column2 = result_set1.getColumnValueAsString(2);
snowflake.execute( {sqlText: `
INSERT INTO mytable("GEOGRAPHY")(
Select * from (
with months as (
select dateadd(month, seq4(), '2020-02-01') "REPORTING MONTH" from table (generator(rowcount => 12))
), months_ids as (
select * from months cross join (select column1 AS "id1",column2 AS "id2")
) ,
event_months as (
Select * from (Select *,ROW_NUMBER() OVER (PARTITION BY id1,id2 ORDER BY "REPORTING MONTH") As rn FROM mytable) Where rn =1
) ,
final as (
select "REPORTING MONTH",id1,id2
, (select array_agg("GEOGRAPHY") within group (order by "REPORTING MONTH" desc) from mytable where a."REPORTING MONTH">="REPORTING MONTH" and a.id1=id1
from months_ids a order by "REPORTING MONTH"
)
Select a."GEOGRAPHY" from final a left join event_months b on a.id1=b.id1 and a.id2 = b.id2 where a."REPORTING MONTH" > b."REPORTING MONTH"
Except
Select "GEOGRAPHY" from mytable WHERE id1 = column1 and id2 = column2
)
)
`});
}
return "success";
$$
;
CALL TEST_PROC_STMT();
There are at least two problems with the code. First, "var" defines a variable, which is a one-time operation. In this case it needs to be moved out of the while loop:
// Loop through the results, processing one row at a time...
while (result_set1.next()) {
var column1 = result_set1.getColumnValueAsString(1);
var column2 = result_set1.getColumnValueAsString(2);
This is an easy fix:
// Loop through the results, processing one row at a time...
var column1;
var column2;
while (result_set1.next()) {
column1 = result_set1.getColumnValueAsString(1);
column2 = result_set1.getColumnValueAsString(2);
The other problem is the code specifies "column2" as a literal column name in the SQL, not a variable. You can tell this because the error message uppercased the variable name, so Snowflake is looking for "COLUMN2" and can't find it. There's also a use of "column1" that appears it should be a replacement variable.
You can fix that by making it a replacement variable. Change these two lines:
select * from months cross join (select column1 AS "id1",column2 AS "id2")
... and this one ...
Select "GEOGRAPHY" from mytable WHERE id1 = column1 and id2 = column2
To this:
select * from months cross join (select ${column1} AS "id1", ${column2} AS "id2")
... and this ...
Select "GEOGRAPHY" from mytable WHERE id1 = column1 and id2 = ${column2}
Note that the ${replacement_variable} syntax only works in JavaScript when you define a string using backticks. It will not work when using single or double quotes to terminate a string.
Getting past those two may expose others, but could make it just run.

Renaming a JSON column for a UNION

Remark: my example is overly simplified. In reality, I am dealing with a huge query. But to illustrate the issue/errors, let us resort to apples and oranges.
My original query looked like this:
SELECT 'FruitsCount' AS "Type", (SELECT count(id) as Counter, [Name] FROM Fruits group by name FOR JSON PATH) AS "Value"
Which would result in something like. Let's refer to this as Format A
|---------------------|------------------------------------------------------------------------------|
| Type | Value |
|---------------------|------------------------------------------------------------------------------|
| FruitCount | [{"Counter":2, "Name":"Apple"},{"Counter":3, "Name":"Orange"}] |
|---------------------|------------------------------------------------------------------------------|
However, now I want to create a union of Fruit and Vegetable counts. My query now looks like this
(SELECT count(id) as Counter, [Name] FROM Fruits group by name
UNION
SELECT count(id) as Counter, [Name] FROM Vegetables group by name)
FOR JSON PATH
|---------------------|------------------------------------------------------------------------------|
| JSON_F52E2B61-18A1-11d1-B105-00805F49916B |
|---------------------|------------------------------------------------------------------------------|
| [{"Counter":2, "Name":"Apple"},{"Counter":3, "Name":"Orange"},{"Counter":7, "Name":"Tomato"}] |
|---------------------|------------------------------------------------------------------------------|
However, I want it in the format as before, where I have a Type and Value columns (Format A).
I tried doing the following:
SELECT 'FruitsCount' AS "Type", ((SELECT count(id) as Counter, [Name] FROM Fruits group by name
UNION
SELECT count(id) as Counter, [Name] FROM Vegetables group by name) FOR JSON PATH) as "Value"
However, I am presented with Error 156: Incorrect syntax near the keyword 'FOR'.
Then I tried the following:
SELECT 'FruitsAndVegCount' AS "Type", (SELECT count(id) as Counter, [Name] FROM Fruits group by name
UNION
SELECT count(id) as Counter, [Name] FROM Vegetables group by name FOR JSON PATH) as "Value"
However, I am presented with Error 1086: The FOR XML and FOR JSON clauses are invalid in views, inline functions, derived tables, and subqueries when they contain a set operator.
I'm stuck in trying to get my "union-ized" query to be in Format A.
Update 1: Here is the desired output
|---------------------|------------------------------------------------------------------------------------------------|
| Type | Value |
|---------------------|------------------------------------------------------------------------------------------------|
| FruitAndVegCount | [{"Counter":2, "Name":"Apple"},{"Counter":3, "Name":"Orange"},{"Counter":7, "Name":"Tomato"}] |
|---------------------|------------------------------------------------------------------------------------------------|
The goal is to only have a single row, with 2 columns (Type, Value) where Type is whatever I specify (i.e. FruitAndVegCount) and Value is a JSON of the ResultSet that is created by the union query.
If I understand the question correctly, the following statement is an option:
SELECT
[Type] = 'FruitAndVegCount',
[Value] = (
SELECT Counter, Name
FROM (
SELECT count(id) as Counter, [Name] FROM Fruits group by name
UNION ALL
SELECT count(id) as Counter, [Name] FROM Vegetables group by name
) t
FOR JSON PATH
)
You could do it with two columns, Type and Value, as follows. Something like this
select 'FruitAndVegCount' as [Type],
(select [Counter], [Name]
from (select count(id) as Counter, [Name] from #Fruits group by [name]
union all
select count(id) as Counter, [Name] from #Vegetables group by [name]) u
for json path) [Value];
Output
Type Value
FruitAndVegCount [{"Counter":2,"Name":"apple"},{"Counter":1,"Name":"pear"},{"Counter":2,"Name":"carrot"},{"Counter":1,"Name":"kale"},{"Counter":2,"Name":"lettuce"}]

Select default value if column does not exist in oracle

I am trying to select a default value to a column if the column does not exist in the table. Following code seems doesn't give my expected output.
SELECT CustomerID, (select case when exists (select *
from INFORMATION_SCHEMA.COLUMNS
where table_name = 'Customers' and
column_name = 'Age'
)
then Age
else 0 as Age
end)
FROM (select * from Customers);
Since Age column doesn't exist in the table result should be given as:-
CustomerID | Age
-----------|----
Cust01 | 0
Cust02 | 0
Can someone suggest me a solution or the error in above code snippet.
As mentioned in the comments by experts, you need to use a Oracle PLSQL block and achieve your requirement using dynamic sql. Please see below the same code which is written keeping your requirement in mind. You can try implementing your code with this:
declare
var number;
var2 varchar2(4000);
type abc is record
( col1 number,
col2 number);
type var3 is table of abc index by pls_integer;
var4 var3;
begin
select count(1)
into var
from employee;
var2:= 'select A, '||case when var is null then 1 else 2 end || ' from test where rownum < 10';
dbms_output.put_line(var2);
execute immediate var2 bulk collect into var4;
for rec in 1..var4.last
loop
dbms_output.put_line(var4(rec).col1 ||',' ||var4(rec).col2);
end loop;
end;
Output:
select A, 2 from test where rownum < 10
1444,2
1445,2
1446,2
1447,2
1448,2
1449,2
1450,2
1451,2
1452,2

How to replace SELECT statement inside IF statement for it to work [duplicate]

This question already has answers here:
Oracle: how to INSERT if a row doesn't exist
(9 answers)
Closed 9 years ago.
I have a simple question - for examples sake let's have the table
CITY(ID,Name).
An idea would be that when I want to add new city I first make sure it's not already in the table CITY.
Code example would be:
IF cityName NOT IN (SELECT name FROM city) THEN
INSERT INTO City(ID, NAME) VALUES(100, cityName);
ELSE
Raise namingError;
END IF;
However I can't have that subquery inside if statement so what should I replace it with? Any kind of list or variable or trick that I could use?
IF NOT EXISTS(SELECT 1 FROM CITY WHERE NAME = <CITYNAME>)
INSERT INTO City(ID, NAME) VALUES(100, cityName);
OR
INSERT INTO City
SELECT 100,'cityName'
FROM dual
WHERE NOT EXISTS (SELECT 1
FROM CITY
WHERE name = cityname
)
I read the second query here
I don't have a database to try this out, but this should work
You could use a merge command to perform the insert into the table. While the merge command is used to perform an insert if the data is not present or an update if the data is present in this case since you just have two fields it will just preform the insert for you.
This is useful if you want to take data from one or more tables and combine them into one.
MERGE INTO city c
USING (SELECT * FROM city_import ) h
ON (c.id = h.id and c.city = h.city)
WHEN MATCHED THEN
WHEN NOT MATCHED THEN
INSERT (id, city)
VALUES (h.id, h.city);
http://www.oracle-base.com/articles/9i/merge-statement.php
If it was me I'd probably do something like
DECLARE
rowCity CITY%ROWTYPE;
BEGIN
SELECT * INTO rowCity FROM CITY c WHERE c.NAME = cityName;
-- If we get here it means the city already exists; thus, we raise an exception
RAISE namingError;
EXCEPTION
WHEN NO_DATA_FOUND THEN
-- cityName not found in CITY; therefore we insert the necessary row
INSERT INTO City(ID, NAME) VALUES(100, cityName);
END;
Share and enjoy.
Two options:
One using INSERT INTO ... SELECT with a LEFT OUTER JOIN; and
The other using MERGE
SQL Fiddle
Oracle 11g R2 Schema Setup:
CREATE TABLE city (
ID NUMBER(2) PRIMARY KEY,
NAME VARCHAR2(20)
);
INSERT INTO city
SELECT 1, 'City Name' FROM DUAL;
CREATE TABLE city_errors (
ID NUMBER(2),
NAME VARCHAR2(20),
TS TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
ERROR VARCHAR2(20)
);
Query 1:
DECLARE
city_id CITY.ID%TYPE := 2;
city_name CITY.NAME%TYPE := 'City Name';
namingError EXCEPTION;
PRAGMA EXCEPTION_INIT( namingError, -20001 );
BEGIN
INSERT INTO city ( id, name )
SELECT city_id,
city_name
FROM DUAL d
LEFT OUTER JOIN
city c
ON ( c.name = city_name )
WHERE c.id IS NULL;
IF SQL%ROWCOUNT = 0 THEN
RAISE namingError;
END IF;
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
-- Do something when duplicate ID found.
INSERT INTO city_errors ( ID, NAME, ERROR ) VALUES ( city_id, city_name, 'Duplicate ID' );
WHEN namingError THEN
-- Do something when duplicate Name found.
INSERT INTO city_errors ( ID, NAME, ERROR ) VALUES ( city_id, city_name, 'Duplicate Name' );
END;
Results:
Query 2:
DECLARE
city_id CITY.ID%TYPE := 3;
city_name CITY.NAME%TYPE := 'City Name';
namingError EXCEPTION;
PRAGMA EXCEPTION_INIT( namingError, -20001 );
BEGIN
MERGE INTO city c
USING ( SELECT city_id AS id,
city_name AS name
FROM DUAL ) d
ON ( c.Name = d.Name )
WHEN NOT MATCHED THEN
INSERT VALUES ( d.id, d.name );
IF SQL%ROWCOUNT = 0 THEN
RAISE namingError;
END IF;
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
-- Do something when duplicate ID found.
INSERT INTO city_errors ( ID, NAME, ERROR ) VALUES ( city_id, city_name, 'Duplicate ID' );
WHEN namingError THEN
-- Do something when duplicate Name found.
INSERT INTO city_errors ( ID, NAME, ERROR ) VALUES ( city_id, city_name, 'Duplicate Name' );
END;
Results:
Query 3:
SELECT * FROM City
Results:
| ID | NAME |
|----|-----------|
| 1 | City Name |
Query 4:
SELECT * FROM City_Errors
Results:
| ID | NAME | TS | ERROR |
|----|-----------|--------------------------------|----------------|
| 2 | City Name | January, 02 2014 20:01:49+0000 | Duplicate Name |
| 3 | City Name | January, 02 2014 20:01:49+0000 | Duplicate Name |

mssql - select from another table if no results from the first table

I have to run a query to find a price. I have multiple price lists, each in their own table. customer_price, default_price and minimum_price.
not all price lists have all the products and at times a product might not appear on any price list and then the price needs to return 0 for the price.
so I want to:
select price from customer_price where customer='walmart' and product='whitebread'
if this returns a result then all is well. if the result is 0,NULL or the query returns no rows I want this to run:
select price from default_price where customer='walmart' and product='whitebread'
if this returns a result then all is well. if the result is 0,NULL or the query returns no rows I want to simply return 0.
I am not sure how to proceed. I tried a case statement but the case statement fails if no row is found in the results. how can I do this or say if 0 results then
thanks in advance, as always.
select top 1 price from (
select 1 as qorder, price from customer_price where price > 0 and customer='walmart' and product='whitebread'
union
select 2 as qorder, price from default_price where price > 0 and customer='walmart' and product='whitebread'
union
select 3 as qorder, 0
) as sq
order by qOrder
case when
(select count(price) from customer_price where customer='walmart' and product='whitebread' and price is not null and price > 0) > 0
then
select price from customer_price where customer='walmart' and product='whitebread'
else
select price from default_price where customer='walmart' and product='whitebread'
end
I'm writing a two way solution which includes for both Empty Select Queries or Tables. You can replace your query
if OBJECT_ID('tempdb..#temp1') is not null
drop table #temp1
select * into #temp1 from (select col1, col2 from First_Table) as firstTableVariable
if (##ROWCOUNT > 0)
begin
print 'data is displayed from the first table/query'
select * from #temp1
end
else
begin
print 'Data displayed from second table/query '
select col1, col2 from second_Table
end
end

Resources