Stream Analytics GetArrayElements as String - arrays

I have a Stream analytics job that gets the data from an external source (I do not have a say on how the data is being formatted). I am trying to import the data into my data lake, storing as a JSON. This works fine, but I also want to get the output in a CSV, this is where I am having trouble.
As the input data has an array as one of the column, when importing in JSON it recognizes it and provides the right data i.e. places them in brackets [A, B, C], but when I use it in CSV I get the column represented as the word "Array". I thought I would convert it to XML, use STUFF and get them in one line, but it does not like using a SELECT statement in a CROSS APPLY.
Has anyone worked with Stream Analytics importing data into CSV, that has array column? If so, how did you manage to import the array values?
Sample data:
[
{"GID":"10","UID":1,"SID":"5400.0","PG:["75aef","e5f8e"]},
{"GID":"10","UID":2,"SID":"4400.0","PG:["75aef","e5f8e","6d793"]}
]
PG is the column I am trying to extract, so the output CSV should look something like.
GID|UID|SID|PG
10|1|5400.0|75aef,e5f8e
10|2|4400.0|75aef,e5f8e,6d793
This is the query I am using,
SELECT
D.GID ,
D.UID ,
D.SID ,
A.ArrayValue
FROM
dummy AS D
CROSS APPLY GetArrayElements(D.PG) AS A
As you could imagine, this gives me results in this format.
GID|UID|SID|PG
10|1|5400.0|75aef
10|1|5400.0|e5f8e
10|2|4400.0|75aef
10|2|4400.0|e5f8e
10|2|4400.0|6d793


As Pete M said, you could try to create a JavaScript user-defined function to convert an array to a string, and then you could call this User-defined function in your query.
JavaScript user-defined function:
function main(inputobj) {
var outstring = inputobj.toString();
return outstring;
}
Call UDF in query:
SELECT
TI.GID,TI.UID,TI.SID,udf.extractdatafromarray(TI.PG)
FROM
[TEST-SA-DEMO-BLOB-Input] as TI
Result:

Related

Big query nested json strings to arrays then new tables

Bigquery Database
I've got a webhook that's pushing to my big query table. The problem is it has lots of nested json strings which are brought in as strings. I ultimately want to make each column with these json strings into their own tables but I'm getting stuck because I can't figure out how to get them unnested and into an array.
[{"id":"63bddc8cfe21ec002d26b7f4","description":"General Admission", "src_currency":"USD","src_price":50.0,"src_fee":0.0,"src_commission":1.79,"src_discount":0.0,"applicable_pass_id":null,"seats_label":null,"seats_section_label":null,"seats_parent_type":null,"seats_parent_label":null,"seats_self_type":null,
"seats_self_label":null,"rate_type":"Rate","option_name":null,"price_level_id":null,"src_discount_price":50.0,"rate_id":"636d6d5cea8c6000222c640d","cost_item_id":"63bddc8cfe21ec002d26b7f4"}]
Here's the sample return from the original source and below is a screenshot of what I'm working with.
[Current Database
I've tried a number of things but the multiple nestings and string to array issue are really hampering everything I've tried.
I'm honestly not sure exactly what output/structure is best for this data set. I assume that each of the json returns probably just needs to be its own table and I can reference or join them based off that first "id" value in the json strings but I'm wide open to suggestions.

You can use a combination of JSON functions, and array functions to manipulate this kind of data.
JSON_EXTRACT_ARRAY can convert the JSON formatted string into an array, UNNEST then can make each entry into rows, and finally JSON_EXTRACT_SCALAR can pull out individual columns.
So here's an example of what I think you're trying to accomplish:
with sampledata as (
select """[{"id":"63bddc8cfe21ec002d26b7f4","description":"General Admission", "src_currency":"USD","src_price":50.0,"src_fee":0.0,"src_commission":1.79,"src_discount":0.0,"applicable_pass_id":null,"seats_label":null,"seats_section_label":null,"seats_parent_type":null,"seats_parent_label":null,"seats_self_type":null,"seats_self_label":null,"rate_type":"Rate","option_name":null,"price_level_id":null,"src_discount_price":50.0,"rate_id":"636d6d5cea8c6000222c640d","cost_item_id":"63bddc8cfe21ec002d26b7f4"},{"id":"63bddc8cfe21ec002d26b7f4","description":"General Admission", "src_currency":"USD","src_price":50.0,"src_fee":0.0,"src_commission":1.79,"src_discount":0.0,"applicable_pass_id":null,"seats_label":null,"seats_section_label":null,"seats_parent_type":null,"seats_parent_label":null,"seats_self_type":null,"seats_self_label":null,"rate_type":"Rate","option_name":null,"price_level_id":null,"src_discount_price":50.0,"rate_id":"636d6d5cea8c6000222c640d","cost_item_id":"63bddc8cfe21ec002d26b7f4"}]""" as my_json_string
)
select JSON_EXTRACT_SCALAR(f,'$.id') as id, JSON_EXTRACT_SCALAR(f,'$.rate_type') as rate_type, JSON_EXTRACT_SCALAR(f,'$.cost_item_id') as cost_item_id
from sampledata, UNNEST(JSON_EXTRACT_ARRAY(my_json_string)) as f
Which creates rows with specific columns from that data, like this:

Convert PostgreSQL nested JSON to numeric array in Tableau

I have a PostgreSQL database containing a table test_table with individual records. First column is a simple store_id, second column meausurement is a nested json.
store_id | measurement
----------------------
0 | {...}
The format of the measurement column is as follows:
{
'file_info': 'xxxx',
'data': {
'contour_data': {
'X': [-97.0, -97.0, -97.0, -97.0, -97.0, -97.0],
'Y': [-43.0, -41.0, -39.0, -39.0, -38.0, -36.0]
}
}
}
I would like to plot Y vs. X in a scatter plot in Tableau. Therefore I connected the database successfully with the PostgreSQL connector of Tableau. From this page I learned, that I have to use Custom SQL queries to extract data from the json object, since Tableau doesn't directly support the json datatype of Postgres. I tried already the following Custom SQL Query in Tableau:
select
store_id as store_id,
measurement#>>'{data, contour_data, X}' as contour_points_x,
measurement#>>'{data, contour_data, Y}' as contour_points_y
from test_table
which successfully extracts the two arrays to two new columns contour_points_x and contour_points_y. However both new columns are in Tableau of type string, so I cannot use them as data source for a plot.
How do I have to adjust the Custom SQL query to make the data arrays plottable in a Tableau scatter plot?

Looks like you need to split the columns. Check this https://help.tableau.com/current/pro/desktop/en-us/split.htm
EDIT - the linked approach works when you can reliably assume an upper bound for the number of points in each list. One way to split arbitrarily sized lists is described here https://apogeeintegration.com/blog/apogee-busts-out-multi-value-cells-using-tableau-prep-builder

The answer is a concatenation of several functions and/or syntax operations. One has to
use the #> operator to dig in the json and return it as json type (not as text type as >># does).
use json_array_elements_text() to expand the json to a set of text.
use type cast operator :: to convert text to float
/* custom SQL Query in Tableau */
select
store_id as store_id,
json_array_elements_text(measurement#>'{data, contour_data, X}')::float as contour_points_x,
json_array_elements_text(measurement#>'{data, contour_data, Y}')::float as contour_points_y,
from test_table
Both resulting columns appear in a Tableau Sheet now as discrete measures. Changing to discrete dimensions allows to plot contour_points_y vs. contour_points_x as desired.

Postgres: is there any row_to_json equivalent that returns values only?

In a project I'm working on, I need to stream potentially large data sets from a Postgres database to the client, for analytics purposes.
The application is built in Rails (irrelevant for this question) and after a bit of research I'm currently able to stream query results by using COPY in Postgres:
COPY (SELECT row_to_json(t) from (#{query}) t) TO STDOUT;
Sources (for who's interested):
https://shift.infinite.red/fast-csv-report-generation-with-postgres-in-rails-d444d9b915ab
https://github.com/brianhempel/stream_json_demo
This works, but it yields every row as a key-value pair, e.g.:
["{\"id\":403457,\"email\":\"email403457#example.com\",\"first_name\":\"Firstname403457\",\"last_name\":\"Lastname403457\",\"source\":\"adwords\",\"created_at\":\"2015-08-05T22:43:07.295796\",\"updated_at\":\"2017-01-19T04:48:29.464051\"}"]
In the spirit of minimising the size (in bytes) of the response and especially since this is getting served through the web, I want to return just an array of values for every row, i.e.:
["[403457, \"email403457#example.com\", \"Firstname403457\", \"Lastname403457\", \"adwords\", \"2015-08-05T22:43:07.295796\", \"2017-01-19T04:48:29.464051\"]"]
Is there a way to achieve this within Postgres, even by nesting functions, starting from the query above?

You could create a simple SQL function that converts a row into the desired format:
CREATE FUNCTION row2json(anyelement) RETURNS json
LANGUAGE sql STABLE AS
'SELECT json_agg(z.value) FROM json_each(row_to_json($1)) z';
Then you use that to transform the output:
SELECT row2json(mytab) FROM mytab;
If performance is more important than JSON output, just cast the result to a string:
SELECT CAST(mytab AS text) FROM mytab;

Export an array into a CSV-file in PL/pgSQL

I have a function, which RETURNS SETOF text[]. Sample result of this function:
{080213806381,"personal data1","question 1",answer1,"question 2",answer2,"question 3","answer 3"}
{080213806382,"personal data1","question 1",answer1,"question 2",answer2,"question 3","answer 3"}
I'm forming each row with a statement like:
resultRow := array_append(resultRow, fetchedRow.data::text);
and then:
RETURN NEXT resultRow;
And here's my COPY command:
COPY(
SELECT myFunction()
) TO 'D:\test_output.csv' WITH (FORMAT 'csv', DELIMITER E',', HEADER false)
And I have a couple of problems:
Regardless the fact that values are appended to the array in the same way, some of them are double-quoted and some of them are not. This somehow depends on a presence of space character in a value. Look, for instance, at the 1st element of the array or at the answer2 and "answer 3" in each row. I want some unified behavior.
After exporting in to CSV with COPY command I'm getting the same rows with all these curly braces at the beginning and the end. I dont want them in CSV.
What can I do to solve these issues?

You wish to export rows of varying numbers of columns. You're producing a set of arrays, but from there want to produce a CSV file.
The immediate issue - array literals aren't CSV
Your function returns text[] literals, i.e. PostgreSQL array literals.
These are not CSV as commonly recognised. They're comma-separated, yes, but they follow different syntax rules. You can't reliably treat an array literal as a CSV row or vice versa.
Don't attempt to just chop the delimiting {...} off and treat the array literal as a CSV row.
COPY won't work well or at all
COPY is not going to work well for you. It's designed to handle relations, i.e. uniform sets of structured rows where each column is of a well defined type and each row has the same number of columns.
You could redefine your function to return a setof record and pad your records with nulls to always be the same width, but it'll be pretty ugly and limited, plus the CSV will then incorporate the nulls.
What COPY will do is export a single column CSV containing array literals in a single CSV field. This certainly will not be what you want.
Solution 1: Export client-side
You might be better off doing this on the client side, via a script or program to generate the CSV. Have the program receive the set of arrays and then write it to CSV via a suitable library, like Python's csv module. Choose a client scripting language where the PostgreSQL driver understands arrays and can transform them to arrays in the language's format - again, like psycopg2 for Python.
e.g. given dummy function:
CREATE OR REPLACE FUNCTION get_rows() RETURNS setof text[] AS $$
VALUES
('{080213806381,"personal data1","question 1",answer1,"question 2",answer2,"question 3","answer 3"}'::text[]),
('{080213806382,"personal data1","question 1",answer1,"question 2",answer2,"question 3","answer 3","q4","a4"}'::text[])
$$ LANGUAGE SQL;
a client script could be as simple as:
#!/usr/bin/env python
import psycopg2
import csv
with psycopg2.connect('dbname=craig') as conn:
curs = conn.cursor()
with open("test.csv","w") as csvfile:
f = csv.writer(csvfile)
curs.execute("SELECT * FROM get_rows()")
for row in curs:
f.writerow(row[0])
Solution 2: Export CSV directly from a procedure
Alternately, if the CSV document isn't too big, you could produce the entire CSV in a single procedure, perhaps using plpythonu and the csv module, or a similar CSV library for your preferred procedural language. Because the whole CSV document must be accumulated in memory this won't scale to very very large documents.

Using text array as result format is wrong idea - a text array format is not simply convertible with CSV format. Return table instead
CREATE OR REPLACE FUNCTION foo()
RETURNS TABLE(c1 text, c2 text, c3 text, c4 text, c5 text, c6 text, c7 text, c8 text)
AS $$
VALUES('080213806381','personal data1','question 1','answer1','question 2','answer2','question 3','answer 3'),
('080213806382','personal data1','question 1','answer1','question 2','answer2','question 3','answer 3');
$$ LANGUAGE sql;
postgres=# COPY (SELECT * FROM foo()) TO stdout CSV;
080213806381,personal data1,question 1,answer1,question 2,answer2,question 3,answer 3
080213806382,personal data1,question 1,answer1,question 2,answer2,question 3,answer 3
Time: 1.228 ms

Importing jSON Array into Hive/Hadoop

I'm using the HortonWorks Hadoop Sandbox and I have imported a jSON string into a table, as per this guide however I run into trouble as my string contains an array which is not handled well by json_tuple(). I have tried the expolode() function, but that returns the following error:
"Error occurred executing hive query: OK FAILED: UDFArgumentException explode() takes an array or a map as a parameter"
What does that mean exactly and how can I fix it? Would the problem be with the format of the table? I just followed as per the guide above and created the table with:
CREATE EXTERNAL TABLE games
(
value STRING
)
LOCATION '/user/hue/games'
The value I am trying to explode is an array like this:
[{"gameId":49927894,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":100,"championId":53,"spell1":4,"spell2":14,"level":30,"ipEarned":228,"createDate":1404215396001,"fellowPlayers":[{"summonerId":305443,"teamId":200,"championId":62},{"summonerId":468080,"teamId":100,"championId":23},{"summonerId":356814,"teamId":100,"championId":64},{"summonerId":279828,"teamId":200,"championId":222},{"summonerId":284016,"teamId":100,"championId":112},{"summonerId":222776,"teamId":200,"championId":37},{"summonerId":380226,"teamId":100,"championId":104},{"summonerId":871165,"teamId":200,"championId":238},{"summonerId":443896,"teamId":200,"championId":102}],"stats":{"level":12,"goldEarned":7829,"numDeaths":2,"barracksKilled":1,"minionsKilled":15,"championsKilled":2,"goldSpent":6755,"totalDamageDealt":21181,"totalDamageTaken":10271,"team":100,"win":true,"largestMultiKill":1,"physicalDamageDealtPlayer":8258,"magicDamageDealtPlayer":12390,"physicalDamageTaken":7603,"magicDamageTaken":2215,"timePlayed":1560,"totalHeal":782,"totalUnitsHealed":1,"assists":7,"item0":3069,"item1":3024,"item2":2045,"item3":3117,"item4":3082,"item6":3341,"visionWardsBought":1,"magicDamageDealtToChampions":3193,"physicalDamageDealtToChampions":1144,"totalDamageDealtToChampions":4870,"trueDamageDealtPlayer":532,"trueDamageDealtToChampions":532,"trueDamageTaken":451,"wardKilled":2,"wardPlaced":12,"totalTimeCrowdControlDealt":36}},{"gameId":49921219,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":100,"championId":72,"spell1":4,"spell2":11,"level":30,"ipEarned":71,"createDate":1404210159007,"fellowPlayers":[{"summonerId":323330,"teamId":100,"championId":102},{"summonerId":430227,"teamId":200,"championId":134},{"summonerId":1403590,"teamId":200,"championId":114},{"summonerId":382370,"teamId":100,"championId":99},{"summonerId":445767,"teamId":100,"championId":51},{"summonerId":490295,"teamId":200,"championId":19},{"summonerId":313523,"teamId":100,"championId":4},{"summonerId":308769,"teamId":200,"championId":25},{"summonerId":1591549,"teamId":200,"championId":67}],"stats":{"level":16,"goldEarned":12337,"numDeaths":5,"minionsKilled":60,"championsKilled":4,"goldSpent":11955,"totalDamageDealt":128541,"totalDamageTaken":37539,"killingSprees":1,"largestKillingSpree":2,"team":100,"win":false,"neutralMinionsKilled":84,"largestMultiKill":1,"physicalDamageDealtPlayer":68670,"magicDamageDealtPlayer":40557,"physicalDamageTaken":25561,"magicDamageTaken":8767,"timePlayed":2354,"totalHeal":4024,"totalUnitsHealed":1,"assists":12,"item0":3143,"item1":3102,"item2":2045,"item3":3283,"item4":3207,"item6":3364,"visionWardsBought":2,"magicDamageDealtToChampions":3449,"physicalDamageDealtToChampions":3735,"totalDamageDealtToChampions":7690,"trueDamageDealtPlayer":19313,"trueDamageDealtToChampions":505,"trueDamageTaken":3209,"wardKilled":3,"wardPlaced":5,"neutralMinionsKilledEnemyJungle":7,"neutralMinionsKilledYourJungle":77,"totalTimeCrowdControlDealt":1290}},{"gameId":49846672,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":100,"championId":72,"spell1":4,"spell2":11,"level":30,"ipEarned":92,"createDate":1404137990304,"fellowPlayers":[{"summonerId":390284,"teamId":100,"championId":29},{"summonerId":333970,"teamId":200,"championId":236},{"summonerId":306968,"teamId":200,"championId":157},{"summonerId":275006,"teamId":200,"championId":84},{"summonerId":271369,"teamId":200,"championId":92},{"summonerId":942667,"teamId":100,"championId":68},{"summonerId":475293,"teamId":100,"championId":201},{"summonerId":560185,"teamId":100,"championId":115},{"summonerId":340177,"teamId":200,"championId":54}],"stats":{"level":18,"goldEarned":14683,"numDeaths":3,"barracksKilled":1,"turretsKilled":2,"minionsKilled":33,"championsKilled":10,"goldSpent":13153,"totalDamageDealt":146399,"totalDamageTaken":29156,"doubleKills":1,"killingSprees":1,"largestKillingSpree":9,"team":100,"win":true,"neutralMinionsKilled":86,"largestMultiKill":2,"physicalDamageDealtPlayer":85905,"magicDamageDealtPlayer":42140,"physicalDamageTaken":25170,"magicDamageTaken":2669,"largestCriticalStrike":349,"timePlayed":1932,"totalHeal":4997,"totalUnitsHealed":1,"assists":19,"item0":3143,"item1":3207,"item2":3211,"item3":3078,"item4":3009,"item5":1028,"item6":3340,"visionWardsBought":2,"magicDamageDealtToChampions":4955,"physicalDamageDealtToChampions":6923,"totalDamageDealtToChampions":12270,"trueDamageDealtPlayer":18353,"trueDamageDealtToChampions":391,"trueDamageTaken":1317,"wardKilled":3,"wardPlaced":7,"neutralMinionsKilledEnemyJungle":6,"neutralMinionsKilledYourJungle":80,"totalTimeCrowdControlDealt":879}},{"gameId":49842237,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":200,"championId":72,"spell1":4,"spell2":11,"level":30,"ipEarned":239,"createDate":1404132909924,"fellowPlayers":[{"summonerId":1201246,"teamId":100,"championId":18},{"summonerId":306670,"teamId":200,"championId":412},{"summonerId":1424602,"teamId":100,"championId":32},{"summonerId":331359,"teamId":100,"championId":134},{"summonerId":775098,"teamId":200,"championId":10},{"summonerId":309637,"teamId":200,"championId":67},{"summonerId":1111164,"teamId":100,"championId":161},{"summonerId":352500,"teamId":200,"championId":76},{"summonerId":627990,"teamId":100,"championId":13}],"stats":{"level":16,"goldEarned":11745,"numDeaths":1,"turretsKilled":3,"minionsKilled":20,"championsKilled":3,"goldSpent":9790,"totalDamageDealt":119585,"totalDamageTaken":24242,"killingSprees":1,"largestKillingSpree":3,"team":200,"win":true,"neutralMinionsKilled":79,"largestMultiKill":1,"physicalDamageDealtPlayer":64908,"magicDamageDealtPlayer":36298,"physicalDamageTaken":16213,"magicDamageTaken":6611,"timePlayed":1880,"totalHeal":4364,"totalUnitsHealed":1,"assists":12,"item0":3207,"item1":3102,"item2":3068,"item3":3009,"item4":1029,"item5":1027,"item6":3341,"visionWardsBought":2,"magicDamageDealtToChampions":2429,"physicalDamageDealtToChampions":2137,"totalDamageDealtToChampions":4760,"trueDamageDealtPlayer":18378,"trueDamageDealtToChampions":194,"trueDamageTaken":1416,"wardKilled":3,"wardPlaced":9,"neutralMinionsKilledEnemyJungle":9,"neutralMinionsKilledYourJungle":70,"totalTimeCrowdControlDealt":801}},{"gameId":49694129,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":200,"championId":19,"spell1":4,"spell2":11,"level":30,"ipEarned":54,"createDate":1404050008933,"fellowPlayers":[{"summonerId":791058,"teamId":100,"championId":104},{"summonerId":441236,"teamId":200,"championId":115},{"summonerId":469417,"teamId":100,"championId":89},{"summonerId":347644,"teamId":200,"championId":102},{"summonerId":1874556,"teamId":100,"championId":75},{"summonerId":342971,"teamId":200,"championId":25},{"summonerId":359199,"teamId":100,"championId":254},{"summonerId":342453,"teamId":200,"championId":81},{"summonerId":381059,"teamId":100,"championId":38}],"stats":{"level":13,"goldEarned":7193,"numDeaths":5,"minionsKilled":19,"championsKilled":1,"goldSpent":6335,"totalDamageDealt":98517,"totalDamageTaken":26415,"team":200,"win":false,"neutralMinionsKilled":75,"largestMultiKill":1,"physicalDamageDealtPlayer":45035,"magicDamageDealtPlayer":41134,"physicalDamageTaken":21640,"magicDamageTaken":4166,"timePlayed":1622,"totalHeal":12341,"totalUnitsHealed":1,"assists":6,"item0":3153,"item1":3111,"item2":3160,"item6":3340,"visionWardsBought":1,"magicDamageDealtToChampions":4864,"physicalDamageDealtToChampions":1146,"totalDamageDealtToChampions":6355,"trueDamageDealtPlayer":12348,"trueDamageDealtToChampions":344,"trueDamageTaken":608,"wardPlaced":11,"neutralMinionsKilledEnemyJungle":6,"neutralMinionsKilledYourJungle":69,"totalTimeCrowdControlDealt":478}},{"gameId":49679487,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":100,"championId":19,"spell1":4,"spell2":11,"level":30,"ipEarned":103,"createDate":1404045076287,"fellowPlayers":[{"summonerId":368843,"teamId":100,"championId":53},{"summonerId":1850211,"teamId":200,"championId":59},{"summonerId":364524,"teamId":100,"championId":51},{"summonerId":919576,"teamId":200,"championId":89},{"summonerId":280244,"teamId":100,"championId":238},{"summonerId":448125,"teamId":200,"championId":23},{"summonerId":990233,"teamId":200,"championId":63},{"summonerId":551468,"teamId":100,"championId":82},{"summonerId":620390,"teamId":200,"championId":81}],"stats":{"level":18,"goldEarned":16567,"numDeaths":4,"barracksKilled":1,"turretsKilled":2,"minionsKilled":51,"championsKilled":10,"goldSpent":14820,"totalDamageDealt":251883,"totalDamageTaken":46042,"doubleKills":1,"killingSprees":3,"largestKillingSpree":4,"team":100,"win":true,"neutralMinionsKilled":139,"largestMultiKill":2,"physicalDamageDealtPlayer":115795,"magicDamageDealtPlayer":117678,"physicalDamageTaken":33706,"magicDamageTaken":10046,"timePlayed":2273,"totalHeal":25918,"totalUnitsHealed":1,"assists":10,"item0":3153,"item1":3143,"item2":3160,"item3":3284,"item4":3091,"item5":3075,"item6":3364,"visionWardsBought":1,"magicDamageDealtToChampions":16479,"physicalDamageDealtToChampions":9121,"totalDamageDealtToChampions":26052,"trueDamageDealtPlayer":18409,"trueDamageDealtToChampions":451,"trueDamageTaken":2289,"wardKilled":2,"wardPlaced":7,"neutralMinionsKilledEnemyJungle":18,"neutralMinionsKilledYourJungle":121,"totalTimeCrowdControlDealt":1008}},{"gameId":49643142,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":100,"championId":19,"spell1":4,"spell2":11,"level":30,"ipEarned":253,"createDate":1404024693523,"fellowPlayers":[{"summonerId":460616,"teamId":100,"championId":81},{"summonerId":559939,"teamId":100,"championId":79},{"summonerId":1030746,"teamId":200,"championId":236},{"summonerId":421450,"teamId":200,"championId":62},{"summonerId":321477,"teamId":100,"championId":7},{"summonerId":276152,"teamId":100,"championId":92},{"summonerId":1292269,"teamId":200,"championId":63},{"summonerId":561631,"teamId":200,"championId":29},{"summonerId":465045,"teamId":200,"championId":59}],"stats":{"level":18,"goldEarned":16790,"numDeaths":5,"barracksKilled":3,"turretsKilled":3,"minionsKilled":105,"championsKilled":12,"goldSpent":18050,"totalDamageDealt":250227,"totalDamageTaken":54929,"killingSprees":2,"largestKillingSpree":8,"team":100,"win":true,"neutralMinionsKilled":99,"largestMultiKill":1,"physicalDamageDealtPlayer":117959,"magicDamageDealtPlayer":113778,"physicalDamageTaken":45163,"magicDamageTaken":7668,"timePlayed":2261,"totalHeal":34014,"totalUnitsHealed":1,"assists":10,"item0":3153,"item1":3091,"item2":3160,"item3":3274,"item4":3124,"item5":3065,"item6":3340,"magicDamageDealtToChampions":13747,"physicalDamageDealtToChampions":5524,"totalDamageDealtToChampions":19995,"trueDamageDealtPlayer":18488,"trueDamageDealtToChampions":724,"trueDamageTaken":2097,"wardPlaced":5,"neutralMinionsKilledEnemyJungle":26,"neutralMinionsKilledYourJungle":73,"totalTimeCrowdControlDealt":1183}},{"gameId":49459840,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":200,"championId":72,"spell1":4,"spell2":11,"level":30,"ipEarned":113,"createDate":1403929077578,"fellowPlayers":[{"summonerId":674416,"teamId":100,"championId":67},{"summonerId":1600182,"teamId":100,"championId":105},{"summonerId":481511,"teamId":200,"championId":13},{"summonerId":790397,"teamId":100,"championId":412},{"summonerId":1020148,"teamId":200,"championId":53},{"summonerId":651548,"teamId":100,"championId":76},{"summonerId":401190,"teamId":100,"championId":62},{"summonerId":553639,"teamId":200,"championId":51},{"summonerId":1142091,"teamId":200,"championId":115}],"stats":{"level":18,"goldEarned":15974,"numDeaths":4,"turretsKilled":2,"minionsKilled":70,"championsKilled":9,"goldSpent":14048,"totalDamageDealt":165162,"totalDamageTaken":36304,"doubleKills":1,"killingSprees":1,"largestKillingSpree":8,"team":200,"win":true,"neutralMinionsKilled":88,"largestMultiKill":2,"physicalDamageDealtPlayer":94125,"magicDamageDealtPlayer":48323,"physicalDamageTaken":26136,"magicDamageTaken":7798,"largestCriticalStrike":613,"timePlayed":2497,"totalHeal":4294,"totalUnitsHealed":1,"assists":15,"item0":3283,"item1":3102,"item2":3075,"item3":3207,"item4":3078,"item5":1057,"item6":3364,"visionWardsBought":2,"magicDamageDealtToChampions":7822,"physicalDamageDealtToChampions":12309,"totalDamageDealtToChampions":21568,"trueDamageDealtPlayer":22712,"trueDamageDealtToChampions":1436,"trueDamageTaken":2369,"wardKilled":5,"wardPlaced":5,"neutralMinionsKilledEnemyJungle":5,"neutralMinionsKilledYourJungle":83,"totalTimeCrowdControlDealt":899}},{"gameId":49460256,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":100,"championId":72,"spell1":4,"spell2":11,"level":30,"ipEarned":244,"createDate":1403925869628,"fellowPlayers":[{"summonerId":563206,"teamId":200,"championId":11},{"summonerId":582721,"teamId":200,"championId":13},{"summonerId":279571,"teamId":100,"championId":61},{"summonerId":449166,"teamId":200,"championId":4},{"summonerId":276810,"teamId":200,"championId":67},{"summonerId":628613,"teamId":100,"championId":412},{"summonerId":403086,"teamId":100,"championId":23},{"summonerId":860284,"teamId":100,"championId":222},{"summonerId":458717,"teamId":200,"championId":201}],"stats":{"level":16,"goldEarned":10708,"numDeaths":4,"minionsKilled":44,"championsKilled":4,"goldSpent":9738,"totalDamageDealt":113355,"totalDamageTaken":25334,"killingSprees":1,"largestKillingSpree":2,"team":100,"win":true,"neutralMinionsKilled":65,"largestMultiKill":1,"physicalDamageDealtPlayer":62145,"magicDamageDealtPlayer":36007,"physicalDamageTaken":15132,"magicDamageTaken":9118,"largestCriticalStrike":262,"timePlayed":1984,"totalHeal":3256,"totalUnitsHealed":1,"assists":10,"item0":3207,"item1":3283,"item2":1011,"item3":3078,"item4":3082,"item6":3340,"sightWardsBought":1,"visionWardsBought":1,"magicDamageDealtToChampions":5033,"physicalDamageDealtToChampions":6708,"totalDamageDealtToChampions":12367,"trueDamageDealtPlayer":15202,"trueDamageDealtToChampions":626,"trueDamageTaken":1084,"wardKilled":1,"wardPlaced":6,"neutralMinionsKilledEnemyJungle":4,"neutralMinionsKilledYourJungle":61,"totalTimeCrowdControlDealt":678}},{"gameId":49453199,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"RANKED_SOLO_5x5","mapId":1,"teamId":200,"championId":58,"spell1":4,"spell2":14,"level":30,"ipEarned":71,"createDate":1403919131778,"fellowPlayers":[{"summonerId":592234,"teamId":100,"championId":80},{"summonerId":394103,"teamId":200,"championId":119},{"summonerId":790397,"teamId":100,"championId":412},{"summonerId":651548,"teamId":100,"championId":236},{"summonerId":707290,"teamId":200,"championId":40},{"summonerId":468391,"teamId":200,"championId":59},{"summonerId":347220,"teamId":200,"championId":84},{"summonerId":272606,"teamId":100,"championId":105},{"summonerId":509766,"teamId":100,"championId":77}],"stats":{"level":18,"goldEarned":12506,"numDeaths":9,"turretsKilled":1,"minionsKilled":234,"championsKilled":5,"goldSpent":12185,"totalDamageDealt":152192,"totalDamageTaken":42303,"team":200,"win":false,"neutralMinionsKilled":11,"largestMultiKill":1,"physicalDamageDealtPlayer":142022,"magicDamageDealtPlayer":8822,"physicalDamageTaken":30981,"magicDamageTaken":10296,"timePlayed":2340,"totalHeal":3935,"totalUnitsHealed":1,"assists":7,"item0":3074,"item1":3143,"item2":3102,"item3":1055,"item4":1054,"item5":3047,"item6":3361,"sightWardsBought":3,"visionWardsBought":1,"magicDamageDealtToChampions":3116,"physicalDamageDealtToChampions":17951,"totalDamageDealtToChampions":22415,"trueDamageDealtPlayer":1348,"trueDamageDealtToChampions":1348,"trueDamageTaken":1026,"wardKilled":1,"wardPlaced":9,"neutralMinionsKilledEnemyJungle":2,"neutralMinionsKilledYourJungle":9,"totalTimeCrowdControlDealt":200}}]
Any help very much appreciated!

The explode() hive function takes a hive array or map, and you gave it a String value. json_tuple() worked in the guide because it made your string into a map.
You'll want to convert your json array into a format that hive can accept, or use one of the JSON SerDes or something of that nature in order to query the way you want.
JSON SerDe for Hive that supports JSON arrays

Resources