I want to select data as per below criteria in Netezza.
can someone help me to write the sql.
Case 1: Unique ID has 2 "."s
Deal ID = Parse from UNIQ_ID. Pos 1 to first "."
E.g.
Unique ID = 0000149844.FXFWD.COIBI_I
Deal ID = 0000149844
Case 2: Unique ID has 1 "."s
Deal ID = Parse from UNIQ_ID. First "." to end
E.g
Unique ID = 25808.1234140AT`enter code here`
Deal ID = 1234140AT
Use "position" function of Netezza to determine the position of ".", Make use of this output in "substr" function to extract required fields.
For Case 1 :
select substr('0000149844.FXFWD.COIBI_I',1,(position('.' in '0000149844.FXFWD.COIBI_I') - 1));
For Case 2 :
select substr('25808.1234140AT',(position('.' in '25808.1234140AT') + 1));
Related
Stackoverflow supports table markdown. For example, to display a table like this:
N_NATIONKEY
N_NAME
N_REGIONKEY
0
ALGERIA
0
1
ARGENTINA
1
2
BRAZIL
1
3
CANADA
1
4
EGYPT
4
You can write code like this:
|N_NATIONKEY|N_NAME|N_REGIONKEY|
|---:|:---|---:|
|0|ALGERIA|0|
|1|ARGENTINA|1|
|2|BRAZIL|1|
|3|CANADA|1|
|4|EGYPT|4|
It would save a lot of time to generate the Stackoverflow table markdown automatically when running Snowflake queries.
The following stored procedure accepts either a query string or a query ID (it will auto-detect which it is) and returns the table results as Stackoverflow table markdown. It will automatically align numbers and dates to the right, strings, arrays, and objects to the left, and other types default to centered. It supports any query you can pass to it. It may be a good idea to use $$ to terminate the string passed into the procedure in case the SQL contains single quotes. You can create the procedure and test it using this script:
create or replace procedure MARKDOWN("queryOrQueryId" string)
returns string
language javascript
execute as caller
as
$$
const MAX_ROWS = 50; // Set the maximum row count to fetch. Tables in markdown larger than this become hard to read.
var [rs, i, c, row, props] = [null, 0, 0, 0, {}];
if (!queryOrQueryId || queryOrQueryId == 0){
queryOrQueryId = `select * from table(result_scan(last_query_id())) limit ${MAX_ROWS}`;
}
queryOrQueryId = queryOrQueryId.trim();
if (isUUID(queryOrQueryId)){
rs = snowflake.execute({sqlText:`select * from table(result_scan('${queryOrQueryId}')) limit ${MAX_ROWS}`});
} else {
rs = snowflake.execute({sqlText:`${queryOrQueryId}`});
}
props.columnCount = rs.getColumnCount();
for(i = 1; i <= props.columnCount; i++){
props["col" + i + "Name"] = rs.getColumnName(i);
props["col" + i + "Type"] = rs.getColumnType(i);
}
var table = getHeader(props);
while(rs.next()){
row = "|";
for(c = 1; c <= props.columnCount; c++){
row += escapeMarkup(rs.getColumnValueAsString(c)) + "|";
}
table += "\n" + row;
}
return table;
//------ End main function. Start of helper functions.
function escapeMarkup(s){
s = s.replace(/\\/g, "\\\\");
s = s.replaceAll('|', '\\|');
s = s.replace(/\s+/g, " ");
return s;
}
function getHeader(props){
s = "|";
for (var i = 1; i <= props.columnCount; i++){
s += props["col" + i + "Name"] + "|";
}
s += "\n";
for (var i = 1; i <= props.columnCount; i++){
switch(props["col" + i + "Type"]) {
case 'number':
s += '|---:';
break;
case 'string':
s += '|:---';
break;
case 'date':
s += '|---:';
break;
case 'json':
s += '|:---';
break;
default:
s += '|:---:';
}
}
return s + "|";
}
function isUUID(str){
const regexExp = /^[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$/gi;
return regexExp.test(str);
}
$$;
-- Usage type 1, a simple query:
call stackoverflow_table($$ select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.NATION limit 5 $$);
-- Usage type 2, a query ID:
select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.NATION limit 5;
set quid = (select last_query_id());
call stackoverflow_table($quid);
Edit: Based on Fieldy's helpful feedback, I modified the procedure code to allow passing null or 0 or a blank string '' as the parameter. This will use the last query ID and is a helpful shortcut. It also adds a constant to the code that will limit the returns to a set number of rows. This limit will be applied when using query IDs (or sending null, '', or 0, which uses the last query ID). The limit is not applied when the input parameter is the text of a query to run to avoid syntax errors if there's already a limit applied, etc.
Greg Pavlik's Javascript Stored Procedure solution made me wonder if this would be any easier with the new Python language support in Stored Procedures. This is currently a public-preview feature.
The Python Snowpark API supports returning a result as a Pandas dataframe, and Pandas supports returning a dataframe in Markdown format, via the tabulate package. Here's the stored procedure.
CREATE OR REPLACE PROCEDURE markdown_table(query_id VARCHAR)
RETURNS VARCHAR
LANGUAGE PYTHON
RUNTIME_VERSION = '3.8'
PACKAGES = ('snowflake-snowpark-python','pandas','tabulate', 'regex')
HANDLER = 'markdown_table'
EXECUTE AS CALLER
AS $$
import pandas as pd
import tabulate
import regex
def markdown_table(session, queryOrQueryId = None):
# Validate UUID
if(queryOrQueryId is None):
pandas_result = session.sql("""Select * from table(result_scan(last_query_id()))""").to_pandas()
elif(bool(regex.match("^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", queryOrQueryId))):
pandas_result = session.sql(f"""select * from table(result_scan('{queryOrQueryId}'))""").to_pandas()
else:
pandas_result = session.sql(queryOrQueryId).to_pandas()
return pandas_result.to_markdown()
$$;
Which you can use as follows:
-- Usage type 1, use the result from the query ran immediately proceeding the Store-Procedure Call
select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.NATION limit 5;
call markdown_table(NULL);
-- Usage type 2, pass in a query_id
select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.NATION limit 5;
set quid = (select last_query_id());
select $quid;
call markdown_table($quid);
-- Usage type 3, provide a Query string to the Store-Procedure Call
call markdown_table('select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.NATION limit 5');
The table can also be
N_NATIONKEY|N_NAME|N_REGIONKEY
--|--|--
0|ALGERIA|0
1|ARGENTINA|1
2|BRAZIL|1
3|CANADA|1
4|EGYPT|4
giving, so it can be a simpler solution
N_NATIONKEY
N_NAME
N_REGIONKEY
0
ALGERIA
0
1
ARGENTINA
1
2
BRAZIL
1
3
CANADA
1
4
EGYPT
4
I grab the result table and use notepad++ and replace tab \t with pipe space | and then insert by hand the header marker line. I sometime replace the empty null results with the text null to make the results make more sense. the form you use with the start/end pipes gets around the need for that.
DBeaver IDE supports "data export as markdown" and "advanced copy as markdown" out-of-the-box:
Output:
|R_REGIONKEY|R_NAME|R_COMMENT|
|-----------|------|---------|
|0|AFRICA|lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to |
|1|AMERICA|hs use ironic, even requests. s|
|2|ASIA|ges. thinly even pinto beans ca|
|3|EUROPE|ly final courts cajole furiously final excuse|
|4|MIDDLE EAST|uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl|
It is rendered as:
R_REGIONKEY
R_NAME
R_COMMENT
0
AFRICA
lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to
1
AMERICA
hs use ironic, even requests. s
2
ASIA
ges. thinly even pinto beans ca
3
EUROPE
ly final courts cajole furiously final excuse
4
MIDDLE EAST
uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl
I have a column (text) in my Postgres DB (v.10) with a JSON format.
As far as i now it's has an array format.
Here is an fiddle example: Fiddle
If table1 = persons and change_type = create then i only want to return the name and firstname concatenated as one field and clear the rest of the text.
Output should be like this:
id table1 did execution_date change_type attr context_data
1 Persons 1 2021-01-01 Create Name [["+","name","Leon Bill"]]
1 Persons 2 2021-01-01 Update Firt_name [["+","cur_nr","12345"],["+","art_cd","1"],["+","name","Leon"],["+","versand_art",null],["+","email",null],["+","firstname","Bill"],["+","code_cd",null]]
1 Users 3 2021-01-01 Create Street [["+","cur_nr","12345"],["+","art_cd","1"],["+","name","Leon"],["+","versand_art",null],["+","email",null],["+","firstname","Bill"],["+","code_cd",null]]
Disassemble json array into SETOF using json_array_elements function, then assemble it back into structure you want.
select m.*
, case
when m.table1 = 'Persons' and m.change_type = 'Create'
then (
select '[["+","name",' || to_json(string_agg(a.value->>2,' ' order by a.value->>1 desc))::text || ']]'
from json_array_elements(m.context_data::json) a
where a.value->>1 in ('name','firstname')
)
else m.context_data
end as context_data
from mutations m
modified fiddle
(Note:
utilization of alphabetical ordering of names of required fields is little bit dirty, explicit order by case could improve readability
resulting json is assembled from string literals as much as possible since you didn't specified if "+" should be taken from any of original array elements
the to_json()::text is just for safety against injection
)
Using SSIS I am bringing in raw text files that contain this in the output:
I use this data later to report on. The Key columns get pivoted. However, I don't want to show all those columns individually, I only want to show the total.
To accomplish this my idea was calculate the Sum on insert using a trigger, and then insert the sum as a new row into the data.
The output would look something like:
Is what I'm trying to do possible? Is there a better way to do this dynamically on pivot? To be clear I'm not just pivoting these rows for a report, there are other ones that don't need the sum calculated.
Using derived column and Script Component
You can achieve this by following these steps:
Add a derived column (name: intValue) with the following expression:
(DT_I4)(RIGHT([Value],2) == "GB" ? SUBSTRING([Value],1,FINDSTRING( [Value], " ", 1)) : "0")
So if the value ends with GB then the number is taken else the result is 0.
After that add a script component, in the Input and Output Properties, click on the Output and set the SynchronousInput property to None
Add 2 Output Columns outKey , outValue
In the Script Editor write the following script (VB.NET)
Private SumValues As Integer = 0
Public Overrides Sub PostExecute()
MyBase.PostExecute()
Output0Buffer.AddRow()
Output0Buffer.outKey = ""
Output0Buffer.outValue = SumValues.ToString & " GB"
End Sub
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Output0Buffer.AddRow()
Output0Buffer.outKey = Row.Key
Output0Buffer.outValue = Row.Value
SumValues += Row.intValue
End Sub
I am going to show you a way but I don't recommend adding total to the end of the detail data. If you are going to report on it show it as a total.
After source add a data transformation:
C#
Add two columns to your data flow: Size int and type string
Select Value as readonly
Here is the code:
string[] splits = Row.value.ToString().Split(' '); //Make sure single quote for char
int goodValue;
if(Int32.TryParse(splits[0], out goodValue))
{
Row.Size = goodValue;
Row.Type = "GB";
}
else
{
Row.Size = 0;
Row.Type="None";
}
Now you have the data with the proper data types to do arithmatic in your table.
If you really want the data in your format. Add a multicast and an aggregate and SUM(Size) and then merge back into your original flow.
I was able to solve my problem in another way using a trigger.
I used this code:
INSERT INTO [Table] (
[Filename]
, [Type]
, [DeviceSN]
, [Property]
, [Value]
)
SELECT ms.[Filename],
ms.[Type],
ms.[DeviceSN],
'Memory Device.Total' AS [Key],
CAST(SUM(CAST(left(ms.[Value], 2) as INT)) AS VARCHAR) + ' GB' as 'Value'
FROM [Table] ms
JOIN inserted i ON i.Row# = ms.Row#
WHERE ms.[Value] like '%GB'
GROUP BY ms.[filename],
ms.[type],
ms.[devicesn]
I'm building an application that needs to allow the user to filter a data table according to different filters. So, the user will have three different filter posibilites but he might use only one, or two or the three of them at the same tame.
So, let's say I have the following columns on the table:
ID (int) PK
Sede (int)
Programa (int)
Estado (int)
All of those columns will store numbers, integers. The "ID" column is the primary key, "Sede" stores 1 or 2, "Programa" is any number between 1 and 15, and "Estado" will store numbers between 1 and 13.
The user may filter the data stored in the table using any of those filters (Sede, Programa or Estado). But the might, as well, use two filters, or the three of them at the same time.
The idea is that this application works like the data filters on Excel. I created a simulated table on excel to show what I want to achieve:
This first image shows the whole table, without applying any filter.
Here, the user selected a filter for "Sede" and "Programa" but leaved the "Estado" filter empty. So the query returns the values that are equal to the filter, but leaves the "Estado" filter open, and brings all the records, filering only by "Sede" (1) and "Programa" (6).
In this image, the user only selected the "Estado" filter (5), so it brings all the records that match this criteria, it doesn't matter if "Sede" or "Programa" are empty.
If I use a SELECT clasuse with a WHERE on it, it will work, but only if the three filters have a value:
DECLARE #sede int
DECLARE #programa int
DECLARE #estado int
SET #sede = '1'
SET #programa = '5'
SET #estado = '12'
SELECT * FROM [dbo].[Inscripciones]
WHERE
([dbo].[Inscripciones].[Sede] = #sede)
AND
([dbo].[Inscripciones].[Programa] = #programa)
AND
([dbo].[Inscripciones].[Estado] = #estado)
I also tryed changing the "AND" for a "OR", but I can't get the desired result.
Any help will be highly appreciated!! Thanks!
common problem: try using coalesce on the variable and for the 2nd value use the field name you're comparing to. Be careful though; Ensure it's NULL and not empty string being passed!
What this does is take the first non-null value of the variable passed in or the value you're comparing to.. Thus if the value passed in is null the comparison will always return true.
WHERE
[dbo].[Inscripciones].[Sede] = coalesce(#sede, [dbo].[Inscripciones].[Sede])
AND
[dbo].[Inscripciones].[Programa] = coalesce(#programa, [dbo].[Inscripciones].[Programa])
AND
[dbo].[Inscripciones].[Estado] = coalesce(#estado, [dbo].[Inscripciones].[Estado])
If sede is null and programa and estado are populated the compare would look like...
?=? (or 1=1)
?=programa variable passed in
?=Estado variable passed in
Boa Sorte!
Thank you all for your anwers. After reading the article posted in the comments by #SeanLange I was finally able to achieve what was needed. Using a CASE clause in the WHERE statement solves the deal. Here's the code:
SELECT
*
FROM [dbo].[Inscripciones]
WHERE
([dbo].[Inscripciones].[Sede] = (CASE WHEN #sede = '' THEN [dbo].[Inscripciones].[Sede] ELSE #sede END))
AND
([dbo].[Inscripciones].[Programa] = (CASE WHEN #programa = '' THEN [dbo].[Inscripciones].[Programa] ELSE #programa END))
AND
([dbo].[Inscripciones].[Estado] = (CASE WHEN #estado = '' THEN [dbo].[Inscripciones].[Estado] ELSE #estado END))
AND
([dbo].[Inscripciones].[TipoIngreso] = (CASE WHEN #tipoingreso = '' THEN [dbo].[Inscripciones].[TipoIngreso] ELSE #tipoingreso END))
Thanks again!!
I have a table like
City POSTAGE PRICE**
HOUSTON DEFAULT 20
DEFAULT AIR 14
DEFAULT GROUND 30
DEFAULT DEFAULT 40
Now i want to query for price on this table with a combination like 'CHICAGO,GROUND'
which should check if the perfect combination exists, else i should substitute DEFAULT and search for the value..
example,
HOUSTON,AIR should return 14
HOUSTON,GROUND should return 20
HOUSTON,FEDEX should return 20
CHICAGO,FEDEX should return 40
Is there a way to achieve this instead of writing multiple queries ..
thank you!
This uses the SQL*Plus syntax for passing parameters, you may need to change that to suit:
select price
from your_table
where ( city = '&p_city' or city = 'DEFAULT')
and ( postage = '&p_postage' or postage = 'DEFAULT')
order by case when city = '&p_city' then 1 else 2 end
, case when postage = '&p_postage' then 1 else 2 end
This will return multiple rows but presumably you want only the one PRICE. The ORDER BY clause prioritises matches on CITY over matches on POSTAGE. You can then select the first row.