Extract into multiple columns from JSON with PostgreSQL - arrays

I have a column item_id that contains data in JSON (like?) structure.
+----------+---------------------------------------------------------------------------------------------------------------------------------------+
| id | item_id |
+----------+---------------------------------------------------------------------------------------------------------------------------------------+
| 56711 | {itemID":["0530#2#1974","0538\/2#2#1974","0538\/3#2#1974","0538\/18#2#1974","0539#2#1974"]}" |
| 56712 | {itemID":["0138528#2#4221","0138529#2#4221","0138530#2#4221","0138539#2#4221","0118623\/2#2#4220"]}" |
| 56721 | {itemID":["2704\/1#1#1356"]}" |
| 56722 | {itemID":["0825\/2#2#3349","0840#2#3349","0844\/10#2#3349","0844\/11#2#3349","0844\/13#2#3349","0844\/14#2#3349","0844\/15#2#3349"]}" |
| 57638 | {itemID":["0161\/1#2#3364","0162\/1#2#3364","0163\/2#2#3364"]}" |
| 57638 | {itemID":["109#1#3364","110\/1#1#3364"]}" |
+----------+---------------------------------------------------------------------------------------------------------------------------------------+
I need the last four digits before every comma (if there is) and the last 4 digits distincted and separated into individual colums.
The distinct should happen across id as well, so only one result row with id: 57638 is permitted.
Here is a fiddle with a code draft that is not giving the right answer.
The desired result should look like this:
+----------+-----------+-----------+
| id | item_id_1 | item_id_2 |
+----------+-----------+-----------+
| 56711 | 1974 | |
| 56712 | 4220 | 4221 |
| 56721 | 1356 | |
| 56722 | 3349 | |
| 57638 | 3364 | 3365 |
+----------+-----------+-----------+
There can be quite a lot 'item_id_%' column in the results.

with the_table (id, item_id) as (
values
(56711, '{"itemID":["0530#2#1974","0538\/2#2#1974","0538\/3#2#1974","0538\/18#2#1974","0539#2#1974"]}'),
(56712, '{"itemID":["0138528#2#4221","0138529#2#4221","0138530#2#4221","0138539#2#4221","0118623\/2#2#4220"]}'),
(56721, '{"itemID":["2704\/1#1#1356"]}'),
(56722, '{"itemID":["0825\/2#2#3349","0840#2#3349","0844\/10#2#3349","0844\/11#2#3349","0844\/13#2#3349","0844\/14#2#3349","0844\/15#2#3349"]}'),
(57638, '{"itemID":["0161\/1#2#3364","0162\/1#2#3364","0163\/2#2#3364"]}'),
(57638, '{"itemID":["109#1#3365","110\/1#1#3365"]}')
)
select id
,(array_agg(itemid)) [1] itemid_1
,(array_agg(itemid)) [2] itemid_2
from (
select distinct id
,split_part(replace(json_array_elements(item_id::json -> 'itemID')::text, '"', ''), '#', 3)::int itemid
from the_table
order by 1
,2
) t
group by id
DEMO

You can unnest the json array, get the last 4 characters of each element as a number, then do conditional aggregation:
select
id,
max(val) filter(where rn = 1) item_id_1,
max(val) filter(where rn = 2) item_id_2
from (
select
id,
right(val, 4)::int val,
dense_rank() over(partition by id order by right(val, 4)::int) rn
from mytable t
cross join lateral jsonb_array_elements_text(t.item_id -> 'itemID') as x(val)
) t
group by id
You can add more conditional max()s to the outer query to handle more possible values.
Demo on DB Fiddle:
id | item_id_1 | item_id_1
----: | --------: | --------:
56711 | 1974 | null
56712 | 4220 | 4221
56721 | 1356 | null
56722 | 3349 | null
57638 | 3364 | 3365

Related

How to identify valid records based on column values in snowflake

I have a table as below
I want output like below
This means I have few predefined pairs, example
if one employee is coming from both HR_INTERNAL and HR_EXTERNAL, take only that record which is from HR_INTERNAL
if one employee is coming from both SALES_INTERNAL and SALES_EXTERNAL, take only that record which is from SALES_INTERNAL
etc.
Is there a way to achieve this?
I used ROW_NUMBER to rank
ROW_NUMBER() OVER(PARTITION BY "EMPID" ORDER BY SOURCESYSTEM ASC) AS RANK_GID
I just put them on a table like this:
create or replace table predefined_pairs ( pairs ARRAY );
insert into predefined_pairs select [ 'HR_INTERNAL', 'HR_EXTERNAL' ] ;
insert into predefined_pairs select [ 'SALES_INTERNAL', 'SALES_EXTERNAL' ] ;
Then I use the following query to produce the output you wanted:
select s.sourcesystem, s.empid,
CASE WHEN COUNT(1) OVER(PARTITION BY EMPID) = 1 THEN 'ValidRecord'
WHEN p.pairs[0] IS NULL THEN 'ValidRecord'
WHEN p.pairs[0] = s.sourcesystem THEN 'ValidRecord'
ELSE 'InvalidRecord'
END RecordValidity
from source s
left join predefined_pairs p on array_contains( s.sourcesystem::VARIANT, p.pairs ) ;
+-------------------+--------+----------------+
| SOURCESYSTEM | EMPID | RECORDVALIDITY |
+-------------------+--------+----------------+
| HR_INTERNAL | EMP001 | ValidRecord |
| HR_EXTERNAL | EMP001 | InvalidRecord |
| SALES_INTERNAL | EMP002 | ValidRecord |
| SALES_EXTERNAL | EMP002 | InvalidRecord |
| HR_EXTERNAL | EMP004 | ValidRecord |
| SALES_INTERNAL | EMP005 | ValidRecord |
| PURCHASE_INTERNAL | EMP003 | ValidRecord |
+-------------------+--------+----------------+

Group By same or similar string sql

1) Suppose i have a table like this:-
| id | color_code | fruit |
|:------|--------------|----------------:|
| 1 | 000001 | apple |
| 2 | 000001 | apple |
| 3 | 000001 | apple |
| 4 | 000002 | lemon |
| 5 | 000002 | lemon |
| 6 | 000003 | grapes |
| 7 | 000003 | grapes |
How can i group by the fruit column according to the color_code column in sql server?
like this i suppose:-
| id | color_code | fruit | group_concat(id) |
|:------|--------------|-----------------|---------------------|
| 1 | 000001 | apple | 1,2,3 |
| 4 | 000002 | lemon | 2,5 |
| 6 | 000003 | grapes | 6,7 |
2) What if i have 3 tables (like shown below) which require join, how can i achieve this?
series_no table:
| id | desc_seriesno |
|:------|----------------:|
| 7040 | AU1011 |
| 7041 | AU1022 |
| 7042 | AU1033 |
| 7043 | AU1044 |
| 7044 | AU1055 |
| 7045 | AU1066 |
brand table:
| id | desc_brand |
|:------|----------------:|
| 1020 | Audi |
| 1021 | Bentley |
| 1022 | Ford |
| 1023 | BMW |
| 1024 | Mazda |
| 1025 | Toyota |
car_info table:
| seriesno_id | brand_id | color |
|:---------------|------------|--------:|
| 7040 | 1020 | white |
| 7040 | 1020 | black |
| 7040 | 1020 | pink |
| 7041 | 1021 | yellow |
| 7041 | 1021 | brown |
| 7042 | 1022 | purple |
| 7042 | 1022 | black |
| 7042 | 1022 | green |
| 7043 | 1023 | blue |
| 7044 | 1024 | red |
| 7045 | 1025 | maroon |
| 7045 | 1025 | white |
this is my current query with sql server 2014:-
SELECT SN.id AS seriesid, B.id AS brandid, B.desc_brand
FROM [db1].[dbo].[series_no] SN
LEFT JOIN [db1].[dbo].[car_info] CI
ON CI.seriesno_id = SN.id
RIGHT JOIN [db1].[dbo].[brand] B
ON B.id = CI.brand_id
GROUP BY SN.id, B.id
ORDER BY SN.id ASC
but unfortunately it gave me an error since i cannot group by similar string this way.
i want it to be like this:-
| seriesid | brandid | desc_brand | count |
|:-----------|------------|---------------|-------|
| 7040 | 1020 | Audi | 3 |
| 7041 | 1021 | Bentley | 2 |
| 7042 | 1022 | Ford | 3 |
| 7043 | 1023 | BMW | 1 |
| 7044 | 1024 | Mazda | 1 |
| 7045 | 1025 | Toyota | 2 |
1 Fruit Color
Assuming the table name is FruitColor, you can get the desired output by the following query -
SELECT MIN(id) AS id
, color_code
, fruit
, group_concat_id = STUFF((SELECT ',' + CAST(id AS VARCHAR)
FROM FruitColor AS fci
WHERE fci.fruit = fc.fruit AND fci.color_code = fc.color_code
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
FROM FruitColor AS fc
GROUP BY color_code, fruit
ORDER BY id;
The MIN() selects the first id of the group.
Since there is no default GROUP_CONCAT function like in MySql in SQL Server, you have to use the STUFF function and FOR XML PATH. To learn more about group concat you can visit this link https://sqlperformance.com/2014/08/t-sql-queries/sql-server-grouped-concatenation
You can customize the WHERE clause to match only by color_code.
2. You can have several options for this -
Option (a): Show counts for series with brands
SELECT seriesno_id AS seriesid, ci.brand_id AS bandid, desc_brand, COUNT(*) AS [count]
FROM db1.dbo.car_info AS ci
LEFT JOIN db1.dbo.brand AS b ON (b.id = ci.brand_id)
GROUP BY seriesno_id, ci.brand_id, desc_brand;
Here you don't need to use the series table if you want to show counts for cars having brand(s).
You may not need to use the RIGHT JOIN on the brand table because if brand table contains a record which
is not in car_info table, then seriesno_id would be null.
Option (b): Show counts for all the series with or without a brand
SELECT sn.id AS seriesid, ci.brand_id AS bandid, desc_brand, COUNT(*) AS [count]
FROM db1.dbo.series_no AS sn
LEFT JOIN db1.dbo.car_info AS ci ON (ci.seriesno_id = sn.id)
LEFT JOIN db1.dbo.brand AS b ON (b.id = ci.brand_id)
GROUP BY sn.id, ci.brand_id, desc_brand;
Option (c): The work around for selecting a column which is not in a GROUP BY
SELECT seriesno_id AS seriesid, ci.brand_id AS bandid, MAX(desc_brand) AS desc_brand, COUNT(*) AS [count]
FROM db1.dbo.car_info AS ci
LEFT JOIN db1.dbo.brand AS b ON (b.id = ci.brand_id)
GROUP BY seriesno_id, ci.brand_id;
Here, if we are certain that each brand contains only one desc_brand, we can use an aggregate on it.
This is bcause applying aggregate only one value returns that value. I used MAX here.
Personally I would go with option (a) as it makes more sense.
Update on GROUP BY exception for desc_brand being NTEXT...
Cast desc_brand to NVARCHAR to avoid the exception.
CAST(desc_brand AS NVARCHAR(200))
Also I highly recommend using VARCHAR / NVARCHAR instead of any TEXT, CHAR etc. because they usually occupy more memory.
SELECT
id = SUBSTRING(group_concat,1,1),
color_code,
fruit,
group_concat
FROM(
SELECT distinct
m.color_code,
m.fruit,
group_concat = STUFF((SELECT ',' + CONVERT(varchar(10),md.id)
FROM [Test_1].[dbo].[Stuff] md
WHERE m.fruit = md.fruit
AND m.color_code = md.color_code
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
FROM [Test_1].[dbo].[Stuff] m)x
use below code ..
SELECT distinct
m.color_code
, m.fruit
, group_concat = STUFF((
SELECT ',' + CONVERT(varchar(10),md.id)
FROM dbo.tablename md
WHERE m.fruit = md.fruit and m.color_code = md.color_code
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
FROM dbo.tablename m
for second :
SELECT SN.id AS seriesid, B.id AS brandid, B.desc_brand ,count(*)
FROM [db1].[dbo].[series_no] SN
LEFT JOIN [db1].[dbo].[car_info] CI
ON CI.seriesno_id = SN.id
RIGHT JOIN [db1].[dbo].[brand] B
ON B.id = CI.brand_id
GROUP BY SN.id, B.id ,B.desc_brand
ORDER BY 4 ASC

Selecting grouped rows after first two rows SQL Server

This is a bit of a tricky question/situation and my search fu failed me.
Lets say i have the following data
| UID | SharedID | Type | Date |
|-----|----------|------|-----------|
| 1 | 1 | foo | 2/4/2016 |
| 2 | 1 | foo | 2/5/2016 |
| 3 | 1 | foo | 2/8/2016 |
| 4 | 1 | foo | 2/11/2016 |
| 5 | 2 | bar | 1/11/2016 |
| 6 | 2 | bar | 2/11/2016 |
| 7 | 3 | baz | 2/1/2016 |
| 8 | 3 | baz | 2/3/2016 |
| 9 | 3 | baz | 2/11/2016 |
And I would like to ommit a variable number of leading rows (most recent date in this case) and lets say that number is 2 in this example. The resulting table would be something like this:
| UID | SharedID | Type | Date |
|-----|----------|------|-----------|
| 1 | 1 | foo | 2/4/2016 |
| 2 | 1 | foo | 2/5/2016 |
| 7 | 3 | baz | 2/1/2016 |
Is this possible in SQL? Essentially I want to filter on an unknown number of rows which uses the date column as the order by. The goal is to get the oldest types and get a list of UID's in the process.
Sure, it's possible. Use a ROW_NUMBER function to assign a value to each row, partitioning by the SharedID column so that the count restarts every time that ID changes, and select those rows with a value greater than your limit.
WITH cteNumberedRows AS (
SELECT UID, SharedID, Type, Date,
ROW_NUMBER() OVER(PARTITION BY SharedID ORDER BY Date DESC) AS RowNum
FROM YourTable
)
SELECT UID, SharedID, Type, Date
FROM cteNumberedRows
WHERE RowNum > 2;
Not sure if I understand what you mean but something like this?
SELECT * FROM MyTable t1 JOIN MyTable T2 ON t2.id NOT IN (
SELECT TOP 2 UID FROM myTable
WHERE SharedID = t1.sharedID
ORDER BY [Date] DESC
)

SQL Server : update sequence number across multiple groups

I would like to update a table:
| id | type_id | created_at | sequence |
|----|---------|------------|----------|
| 1 | 1 | 2010-04-26 | NULL |
| 2 | 1 | 2010-04-27 | NULL |
| 3 | 2 | 2010-04-28 | NULL |
| 4 | 3 | 2010-04-28 | NULL |
To this (note that created_at is used for ordering, and sequence is "grouped" by type_id):
| id | type_id | created_at | sequence |
|----|---------|------------|----------|
| 1 | 1 | 2010-04-26 | 1 |
| 2 | 1 | 2010-04-27 | 2 |
| 3 | 2 | 2010-04-28 | 1 |
| 4 | 3 | 2010-04-28 | 1 |
Same question has been raised but for SQL Server.
Link
Thanks.
You can use ROW_NUMBER() to get sequence number per type_id slice. Use a CTE to make UPDATE operation simpler:
;WITH ToUpdate AS (
SELECT id, type_id, created_at, sequence,
ROW_NUMBER() OVER (PARTITION BY type_id ORDER BY created_at) AS newSeq
FROM mytable
)
UPDATE ToUpdate
SET sequence = newSeq
Demo here

Twice Inner Join on same table with Aggregate function

being a novice sql user:
I have a simple table storing some records over night daily. table:
Table: T1
+----+-----+----+-----------+------------+
| Id | A | AB | Value | Date |
+----+-----+----+-----------+------------+
| 1 | abc | I | -48936.08 | 2013-06-24 |
| 2 | def | A | 431266.19 | 2013-06-24 |
| 3 | xyz | I | -13523.90 | 2013-06-24 |
| 4 | abc | A | 13523.90 | 2013-06-23 |
| 5 | xyz | I | -13523.90 | 2013-06-23 |
| 6 | def | A | 13523.90 | 2013-06-22 |
| 7 | def | I | -13523.90 | 2013-06-22 |
+----+-----+----+-----------+------------+
I would like to get all values of columns A,AB, Value for the latest Date on Column A filtered on AB = I
basically the result should look like:
+----+-----+----+-----------+------------+
| Id | A | AB | Value | Date |
+----+-----+----+-----------+------------+
| 1 | abc | I | -48936.08 | 2013-06-24 |
| 3 | xyz | I | -13523.90 | 2013-06-24 |
| 7 | def | I | -13523.90 | 2013-06-22 |
+----+-----+----+-----------+------------+
I have tried to use inner join twice on the same table but failed to come up with correct result.
any help would be appreciated.
thanks :)
This will work with sqlserver 2005+
;WITH a as
(
SELECT id, A,AB, Value, Date
, row_number() over (partition by A order by Date desc) rn
FROM t1
WHERE AB = 'I'
)
SELECT id, A,AB, Value, Date
FROM a WHERE rn = 1
; WITH x AS (
SELECT id
, a
, ab
, "value"
, "date"
, Row_Number() OVER (PARTITION BY a ORDER BY "date" DESC) As row_num
FROM your_table
WHERE ab = 'I'
)
SELECT *
FROM x
WHERE row_num = 1

Resources