Listing Nested JSON Objects - snowflake-cloud-data-platform

Question

We have the following JSON structure from an TSheets API which has an actual timesheet 'id' as an object in the hierarchy. This means there's no fixed hierarchy structure and we need to dynamically find a way to loop through each timesheet.
We've stored this data in a variant column, but want to flatten it and have a row per timesheet. Is there a way to list all objects under results.timesheets to retrieve all ids in a single column (i.e. '13510958','13510960') so we can loop through these to obtain the lower level details. Seems like an odd way to construct an API response!
JSON can be found below:
{
"results": {
"timesheets": {
"13510958": {
"id": 13510958,
"user_id": 1360082,
"jobcode_id": 16297998,
"start": "",
"end": "",
"duration": 28800,
"date": "2021-03-29",
"tz": 1,
"tz_str": "Europe/London",
"type": "manual",
"location": "QuickBooks Time web",
"on_the_clock": false,
"locked": 0,
"notes": "",
"customfields": {
"802478": "",
"650642": "",
"650640": "Consulting Services:Services"
},
"last_modified": "2021-04-19T14:34:16+00:00",
"attached_files": [],
"created_by_user_id": 1360067
},
"13510960": {
"id": 13510960,
"user_id": 1360082,
"jobcode_id": 16297998,
"start": "",
"end": "",
"duration": 28800,
"date": "2021-03-30",
"tz": 1,
"tz_str": "Europe/London",
"type": "manual",
"location": "QuickBooks Time web",
"on_the_clock": false,
"locked": 0,
"notes": "",
"customfields": {
"802478": "",
"650642": "",
"650640": "Consulting Services:Services"
},
"last_modified": "2021-04-19T14:34:16+00:00",
"attached_files": [],
"created_by_user_id": 1360067
}}
} }

Answer 1

You can use LATERL FLATTEN, LISTAGG or ARRAY_AGG to get it:
with json_data as ( select parse_json('{
"results": {
"timesheets": {
"13510958": {
"id": 13510958,
"user_id": 1360082,
"jobcode_id": 16297998,
"start": "",
"end": "",
"duration": 28800,
"date": "2021-03-29",
"tz": 1,
"tz_str": "Europe/London",
"type": "manual",
"location": "QuickBooks Time web",
"on_the_clock": false,
"locked": 0,
"notes": "",
"customfields": {
"802478": "",
"650642": "",
"650640": "Consulting Services:Services"
},
"last_modified": "2021-04-19T14:34:16+00:00",
"attached_files": [],
"created_by_user_id": 1360067
},
"13510960": {
"id": 13510960,
"user_id": 1360082,
"jobcode_id": 16297998,
"start": "",
"end": "",
"duration": 28800,
"date": "2021-03-30",
"tz": 1,
"tz_str": "Europe/London",
"type": "manual",
"location": "QuickBooks Time web",
"on_the_clock": false,
"locked": 0,
"notes": "",
"customfields": {
"802478": "",
"650642": "",
"650640": "Consulting Services:Services"
},
"last_modified": "2021-04-19T14:34:16+00:00",
"attached_files": [],
"created_by_user_id": 1360067
}}
}}') raw )
select listagg( v.key , ',' ), array_agg( v.key)
from json_data,
lateral flatten( raw:results.timesheets ) v;

Answer 2

When you want to obtain the lower level details without looping through them, you can also access them directly. For example the timesheet, user_id and duration:
with json_data as (
select parse_json('{
"results": {
"timesheets": {
"13510958": {
"id": 13510958,
"user_id": 1360082,
"jobcode_id": 16297998,
"start": "",
"end": "",
"duration": 28800,
"date": "2021-03-29",
"tz": 1,
"tz_str": "Europe/London",
"type": "manual",
"location": "QuickBooks Time web",
"on_the_clock": false,
"locked": 0,
"notes": "",
"customfields": {
"802478": "",
"650642": "",
"650640": "Consulting Services:Services"
},
"last_modified": "2021-04-19T14:34:16+00:00",
"attached_files": [],
"created_by_user_id": 1360067
},
"13510960": {
"id": 13510960,
"user_id": 1360082,
"jobcode_id": 16297998,
"start": "",
"end": "",
"duration": 28800,
"date": "2021-03-30",
"tz": 1,
"tz_str": "Europe/London",
"type": "manual",
"location": "QuickBooks Time web",
"on_the_clock": false,
"locked": 0,
"notes": "",
"customfields": {
"802478": "",
"650642": "",
"650640": "Consulting Services:Services"
},
"last_modified": "2021-04-19T14:34:16+00:00",
"attached_files": [],
"created_by_user_id": 1360067
}}
}}') raw )
select key, value:user_id, value:duration
from json_data,
lateral flatten(input=>raw:results.timesheets)

Answer 3

I was able to get this to work.
SELECT VALUE AS TIMESHEET_JSON, TIMESHEET_JSON:id AS ID
FROM TABLE(FLATTEN(INPUT=> PARSE_JSON('{Your JSON Here}'):results.timesheets));
You may need to do a little more work to get your JSON into the flatten. A lateral flatten may be useful. This worked once I put your JSON into a one column cte.
SELECT VALUE AS TIMESHEET_JSON, TIMESHEET_JSON:id AS ID
FROM cte
,LATERAL FLATTEN(INPUT=> PARSE_JSON(JSON):results.timesheets);
You can then use typical JSON parsing syntax against the VALUE column for getting at your individual attributes per record.