I would like to know if is possible to 'covnert' a json object to a json array to iterate over a mixed set of data.
I have two rows that look like
{Data:{BASE:{B1:0,color:green}}}
{Data:{BASE:[{B1:1,color:red},{B1:0,color:blue}]}}
I would like to extract the B1 val from all this rows, but I am a bit blocked :)
My first try was a json_extract_array, but it fails on the 1st row (not an array).
Then my second try was a json_array_length with a case, but that fails at the 1st row (not array)
Can I handle this situation in any way?
Basically I need to extract all the rows where B1 > 0 in one of the json array (or object) and maybe return the node that contains B1 > 0.
Your main problem is you mix the data types under the json -> 'Data' -> 'BASE' path, which cannot be handled easily. I could come up with a solution, but you should fix your schema, f.ex. to only contain arrays at that path.
with v(j) as (
values (json '{"Data":{"BASE":{"B1":0,"color":"green"}}}'),
('{"Data":{"BASE":[{"B1":1,"color":"red"},{"B1":0,"color":"blue"}]}}')
)
select j, node
from v,
lateral (select j #> '{Data,BASE}') b(b),
lateral (select substring(trim(leading E'\x20\x09\x0A\x0D' from b::text) from 1 for 1)) l(l),
lateral (select case
when l = '{' and (b #>> '{B1}')::numeric > 0 then b
when l = '[' then (select e from json_array_elements(b) e where (e #>> '{B1}')::numeric > 0 limit 1)
else null
end) node(node)
where node is not null
SQLFiddle
To return the rows where at least one object has B1 > 0
select *
from t
where true in (
select (j ->> 'B1')::int > 0
from json_array_elements (json_column -> 'Data' -> 'BASE') s (j)
)
With a little help from the other answers here, I did this:
with v(j) as (
values (json '{"Data":{"BASE":{"B1":0,"color":"green"}}}'),
('{"Data":{"BASE":[{"B1":1,"color":"red"},{"B1":0,"color":"blue"}]}}')
)
select j->'Data'->'BASE'->>'B1' as "B1"
from v
where j->'Data'->'BASE'->>'B1' is not null
union all
select json_array_elements(j->'Data'->'BASE')->>'B1'
from v
where j->'Data'->'BASE'->>'B1' is null
Divided it into two queries, one which just fetches a single value if there is only one, and one that unwraps the array if there are multiple, utilizing that PostgreSQL returns null if you ask for the text of something that is an array. Then, I just unioned the result from the two queries, which resulted in:
------
| B1 |
------
| 0 |
| 1 |
| 0 |
Related
I am not sure if my database design is good for this tricky case and I also ask for help how the query for this could look like.
I plan a query with the following table:
search_array | value | id
-----------------------+-------+----
{XYa,YZb,WQb} | b | 1
{XYa,YZb,WQb,RSc,QZa} | a | 2
{XYc,YZa} | c | 3
{XYb} | a | 4
{RSa} | c | 5
There are 5 main elements in the search_array: XY, YZ, WQ, RS, QZ and 3 Values: a, b, c that are concardinated to each element.
Each row has also one value: a, b or c.
My aim is to find all rows that fit to a specific row in this sense: At first it should be checked if they have any same main elements in their search_arrays (yellow marked in the example).
As example:
Row id 4 an row id 5 wouldnt match because XY != RS.
Row id 1, 2 and 3 would match two times because they have all XY and YZ.
Row id 1 and 2 would even match three times because they have also WQ in common.
And second: if there is a Main Element match it should be 'crosschecked' if the lowercase letters after the Main Elements fit to the value of the other row.
As example: The only match for Row id 1 in the table would be Row id 4 because they both search for XY and the low letters after the elements match each value of the two rows.
Another match would be ROW id 2 and 5 with RS and search c to value c and search a to value a (green and orange marked).
My idea was to cut the search_array elements in the query in two parts with the RIGHT and LEFT command for strings. But I dont know how to combine the subqueries for this search.
Or would be a complete other solution faster? Like splitting the search array into another table with the columns 'foregin key' to the maintable, 'main element' and 'searched_value'. I am not sure if this is the best solution because the program would all the time switch to the main table to find two rows out of 3 million rows to compare their searched_values to the values?
Thank you very much for your answers and your time!
You'll have to represent the data in a normalized fashion. I'll do it in a WITH clause, but it would be better to store the data in this fashion to begin with.
WITH unravel AS (
SELECT t.id, t.value,
substr(u.val, 1, 2) AS arr_main,
substr(u.val, 3, 1) AS arr_val
FROM mytable AS t
CROSS JOIN LATERAL unnest(t.search_array) AS u(val)
)
SELECT a.id AS first_id,
a.value AS first_value,
b.id AS second_id,
b.value AS second_value,
a.arr_main AS main_element
FROM unravel AS a
JOIN unravel AS b
ON a.arr_main = b.arr_main
AND a.arr_val = b.value
AND b.arr_val = a.value;
I have a table that contains a json array. Here is a sample of the contents of the field from:
SELECT json_array FROM table LIMIT 5;
Result:
[{"key1":"value1"}, {"key1":"value2"}, ..., {"key2":"value3"}]
[]
[]
[]{"key1":"value1"}
[]
How can I retrieve all the values and count how many of each value was found?
I am using PostgreSQL 9.5.14, and I have tried the solutions here Querying a JSON array of objects in Postgres
and the ones suggested to me by another generous stackoverflow user in my last question: How can I parse JSON arrays in postgresql?
I tried:
SELECT
value -> 'key1'
FROM
table,
json_array_elements(json_array);
which sadly does not work for me due to receiving the error: cannot call json_array_elements on a scalar
This error happens when using a query that returns more than one row or more than one column as a scalar subquery.
Another solution I tried was:
SELECT json_array as json, (json_array->0),
coalesce(
case
when (json_array->0) IS NULL then null
else (json_array->0->>'key1')
end,
'No value') AS "Value"
FROM table;
which only returned null values for the "Value"
Referencing Querying a JSON array of objects in Postgres I attempted to use this solution as well:
WITH json_test (col) AS (
values (json_arrays)
)
SELECT
y.x->'key1' "key1"
FROM json_test jt,
LATERAL (SELECT json_array_elements(jt.col) x) y;
But I would need to be able to fit all the elements of the json_arrays into json_test
So far I have only attempted to list all the values in the all json arrays, but my ideal end-result for the query resembles this:
Value | Amount
---------------
value1 | 48
value2 | 112
value3 | 93
value4 | 0
Yet again I am grateful for any help with this, thank you in advance.
step-by-step demo:db<>fiddle
SELECT
each.value,
COUNT(*)
FROM
data,
json_array_elements(json_array) elems, -- 1
json_each_text(elems) each -- 2
GROUP BY each.value -- 3
Expand array into one row for each array element
split the key/value pairs into two columns
group by the new value column/count
I have an array in a hive table, and I want to extract the two last elements of each array, something like this:
["a", "b", "c"] -> ["b", "c"]
I tried a code like this:
SELECT
*,
array[size] AS term_n,
array[size - 1] AS term_n_1
FROM
(SELECT *, size(array) AS size FROM MyTable);
But it didn't work, someone has any idea please?
array is a reserved word and should be qualified.
An inner sub-query should be aliased.
Array index start with 0. If the array size is 5 then the last index is 4.
Demo
with MyTable as (select array('A','B','C','D','E') as `array`)
SELECT *
,`array`[size - 1] AS term_n
,`array`[size - 2] AS term_n_1
FROM (SELECT *
,size(`array`) AS size
FROM MyTable
) t
;
+-----------------------+--------+--------+----------+
| t.array | t.size | term_n | term_n_1 |
+-----------------------+--------+--------+----------+
| ["A","B","C","D","E"] | 5 | E | D |
+-----------------------+--------+--------+----------+
I don't know the error that you are getting, but it should be something like
select
yourarray[size(yourarray)],
yourarray[size(yourarray)-1]
from mytable
This is a solution to extract the last element of an array in the same query (notice it is not very optimal, and you can apply the same principle to extract n last elements of the array), the logic is to calculate the size of the last element (amount of letters minus the separator character) and then make a substring from 0 to the total size minus the calculated amount of characters to extract
Table of example:
col1 | col2
--------------
row1 | aaa-bbb-ccc-ddd
You want to get (extracting the last element, in this case "-ddd"):
row1 | aaa-bbb-ccc
the query you may need:
select col1, substr(col2,0,length(col2)-(length(reverse(split(reverse(col2),'-')[0]))+1)) as shorted_col2_1element from example_table
If you want to add more elements you have to keep adding the positions in the second part of the operation.
Example to extract the last 2 elements:
select col1, substr(col2,0,length(col2)-(length(reverse(split(reverse(col2),'-')[0]))+1) + length(reverse(split(reverse(col2),'-')[1]))+1)) as shorted_col2_2element from example_table
after executing this second command line you will have something like:
row1 | aaa-bbb
*As said previously this is a not optimal solution at all, but may help you
I have a hive table with the column "periode", the type of the column is string.
The column have values like the following:
[{periode:20160118-20160205,nb:1},{periode:20161130-20161130,nb:1},{periode:20161130-20161221,nb:1}]
[{periode:20161212-20161217,nb:0}]
I want to cast this column in array<struct<periode:string, nb:int>>.
The final goal is to have one raw by periode.
For this I want to use lateral view with explode on the column periode.
That's why I want to convert it to array<struct<string, int>>
Thanks for help.
Sidi
You don't need to "cast" anything, you just need to explode the array and then unpack the struct. I added an index to your data to make it more clear where things are ending up.
Data:
idx arr_of_structs
0 [{periode:20160118-20160205,nb:1},{periode:20161130-20161130,nb:1},{periode:20161130-20161221,nb:1}]
1 [{periode:20161212-20161217,nb:0}]
Query:
SELECT idx -- index
, my_struct.periode AS periode -- unpacks periode
, my_struct.nb AS nb -- unpacks nb
FROM database.table
LATERAL VIEW EXPLODE(arr_of_structs) exptbl AS my_struct
Output:
idx periode nb
0 20160118-20160205 1
0 20161130-20161130 1
0 20161130-20161221 1
1 20161212-20161217 0
It's a bit unclear from your question what the desired result is, but as soon as you update it I'll modify the query accordingly.
EDIT:
The above solution is incorrect, I didn't catch that your input is a STRING.
Query:
SELECT REGEXP_EXTRACT(tmp_arr[0], "([0-9]{8}-[0-9]{8})") AS periode
, REGEXP_EXTRACT(tmp_arr[1], ":([0-9]*)") AS nb
FROM (
SELECT idx
, pos
, COLLECT_SET(tmp_col) AS tmp_arr
FROM (
SELECT idx
, tmp_col
, CASE WHEN PMOD(pos, 2) = 0 THEN pos+1 ELSE pos END AS pos
FROM (
SELECT *
, ROW_NUMBER() OVER () AS idx
FROM database.table ) x
LATERAL VIEW POSEXPLODE(SPLIT(periode, ',')) exptbl AS pos, tmp_col ) y
GROUP BY idx, pos) z
Output:
periode nb
20160118-20160205 1
20161130-20161130 1
20161130-20161221 1
20161212-20161217 0
What about use the split function? you should be able to do something like
select nb, period from
(select split(periode, "-") as periods, nb from yourtable) t
LATERAL VIEW explode(periods) sss AS period;
I didnt tried but it should work :)
EDIT: the above should work if you have a column periodes following a pattern date-date-date.. and a column nb, but it looks like that it isn't the case here. The following query should work for you (verbose but work)
select period, nb from (
select
regexp_replace(split(split(tok1,",")[1],":")[1], "[\\]|}]", "") as nb,
split(split(split(tok1,",")[0],":")[1],"-") as periods
from
(select split(YOURSTRINGCOLUMN, "},") as s1 from YOURTABLE)
r1 LATERAL VIEW explode(s1) ss1 AS tok1
) r2 LATERAL VIEW explode(periods) ss2 AS period;
I realize this question is 1YO, but I ran into this same issue and tackled it by using the json_split brickhouse UDF.
SELECT EXPLODE(
json_split(
'[{"periode":"20160118-20160205","nb":1},{"periode":"20161130-20161130","nb":1},{"periode":"20161130-20161221","nb":1}]'
));
col
{"periode":"20160118-20160205","nb":1}
{"periode":"20161130-20161130","nb":1}
{"periode":"20161130-20161221","nb":1}
Sorry for the spaghetti code.
There's also a similar question here using JSON arrays instead of JSON strings. It's not the same case, but for anyone facing this kind of task it might be useful in a bigger context.
Datatype:
id: int4
keywords: text
objectivable_id: int4
Postgres version: PostgreSQL 9.5.3
Business_objectives table:
id keywords objectivable_id
1 keyword1a,keyword1b,keyword1c 6
2 keyword2a 6
3 testing 5
Currently the query I'm using is :
select array(select b.keywords from business_objectives b where b.objectivable_id = 6)
It selects the keywords of matched objectivable_id as:
{"keyword1a,keyword1b,keyword1c","keyword2a"}
Over here I wanted the result to be :
{"keyword1a","keyword1b","keyword1c","keyword2a"}
I tried using "string_agg(text, delimiter)", but it just combines all the keywords into one single pocket of an array.
You can simply (and cheaply!) use:
SELECT string_to_array(string_agg(keywords, ','), ',')
FROM business_objectives
WHERE objectivable_id = 6;
Concatenate your comma separate lists with string_agg(), and then convert the complete text to an array with string_to_array().
So something like this can give you expected result:
SELECT array_agg( j.keys )
FROM business_objectives b,
LATERAL ( SELECT k
FROM unnest ( string_to_array( b.keywords, ',' ) ) u( k )
) j( keys )
WHERE b.objectivable_id = 6;
array_agg
-------------------------------------------
{keyword1a,keyword1b,keyword1c,keyword2a}
(1 row)
With the LATERAL part, we look at the outer query to create a new view. Simply it does split of your keywords as set of rows which you can then feed into array_agg() function.
See more about LATERAL: https://www.postgresql.org/docs/9.6/static/queries-table-expressions.html#QUERIES-LATERAL