So I have a hive external table with schema looks like this :
{
.
.
`x` string,
`y` ARRAY<struct<age:string,cId:string,dmt:string>>,
`z` string
}
So basically I need to query a column(column "y") which is array of nested json,
I can see data of column "y" from hive, but data in that column seems invisible to presto, even though presto knows schema of this field, like this:
array(row(age varchar,cid varchar,dmt varchar))
As you can see presto already knows this field is array of row.
Notes:
1.The table is a hive external table.
2.I get schema of field "y" by using ODBC driver, but data is just all empty, however I can see something like this in hive :
[{"age":"12","cId":"bx21hdg","dmt":"120"}]
3.Presto queries hivemetastore for schema.
4.Table was stored as parquet format.
So how can I see my data in field "y" please?
Please try the below. This should work in Presto.
"If the array element is a row data type, the result is a table with one column for each row field in the element data type. The result table column data types match the corresponding array element row field data types"
select
y,age,cid,dmt
from
table
cross join UNNEST(y) AS nested_data(age,cid,dmt)
Reference: https://www.ibm.com/support/knowledgecenter/en/SSEPGG_10.5.0/com.ibm.db2.luw.sql.ref.doc/doc/r0055064.html
Related
Have a table where two columns of type image. These contain XML formatted messages. Some turn out to be invalid XML. I would like to find all records where either of the 2 columns contain '&' that isn't part of '&'. Is this possible?
For example:
select *
from maxinterrormsg
where msgdata like '%&CAP%'
or intmsgdata like '%&CAP%'
Except of course this does not work.
I have a PostgreSQL database containing a table test_table with individual records. First column is a simple store_id, second column meausurement is a nested json.
store_id | measurement
----------------------
0 | {...}
The format of the measurement column is as follows:
{
'file_info': 'xxxx',
'data': {
'contour_data': {
'X': [-97.0, -97.0, -97.0, -97.0, -97.0, -97.0],
'Y': [-43.0, -41.0, -39.0, -39.0, -38.0, -36.0]
}
}
}
I would like to plot Y vs. X in a scatter plot in Tableau. Therefore I connected the database successfully with the PostgreSQL connector of Tableau. From this page I learned, that I have to use Custom SQL queries to extract data from the json object, since Tableau doesn't directly support the json datatype of Postgres. I tried already the following Custom SQL Query in Tableau:
select
store_id as store_id,
measurement#>>'{data, contour_data, X}' as contour_points_x,
measurement#>>'{data, contour_data, Y}' as contour_points_y
from test_table
which successfully extracts the two arrays to two new columns contour_points_x and contour_points_y. However both new columns are in Tableau of type string, so I cannot use them as data source for a plot.
How do I have to adjust the Custom SQL query to make the data arrays plottable in a Tableau scatter plot?
Looks like you need to split the columns. Check this https://help.tableau.com/current/pro/desktop/en-us/split.htm
EDIT - the linked approach works when you can reliably assume an upper bound for the number of points in each list. One way to split arbitrarily sized lists is described here https://apogeeintegration.com/blog/apogee-busts-out-multi-value-cells-using-tableau-prep-builder
The answer is a concatenation of several functions and/or syntax operations. One has to
use the #> operator to dig in the json and return it as json type (not as text type as >># does).
use json_array_elements_text() to expand the json to a set of text.
use type cast operator :: to convert text to float
/* custom SQL Query in Tableau */
select
store_id as store_id,
json_array_elements_text(measurement#>'{data, contour_data, X}')::float as contour_points_x,
json_array_elements_text(measurement#>'{data, contour_data, Y}')::float as contour_points_y,
from test_table
Both resulting columns appear in a Tableau Sheet now as discrete measures. Changing to discrete dimensions allows to plot contour_points_y vs. contour_points_x as desired.
I need to your help to change the dimension attribute data type in Excel sheet when connecting cube via excel and I have defined a column as "FLOAT" in Data warehouse and "DOUBLE" in Cube,but that attribute is always displaying as string in Excel sheet when I connect the cube via Excel sheet.
and I didn't define the name column property as well because name column will always support string datatype.
Please find the below example from Adventure works cube to understand my scenario.
I have taken "Weight" attribute from Product dimension and the key column and Value column of this attribute has "Double" datatype and Name column has "Wchar" datatype in the adventure works.
Because of Name column property , Weight attribute is showing as string in Excel sheet.
So I have removed the Name column definition to "Weight" attribute and I kept the Value column and Key column property as "Double" , but still in excel "Weight" attribute is displaying as "String".
Can anyone please help to make this dimension attribute as "double" in excel sheet as well?
Please find the below screenshots for reference.
Thank you in advance for you help.
Bhavani R
You can use MemberValue to get the value of weight as numeric.
CREATE MEMBER your_new_member AS
your_old_member.currentmember.MemberValue
Expanding #GregGalloway comments as an answer:
Create a Named Calculation, for our sample, Weight Numeric
Then edit the Dimension and add an Attribute Relationship between Weight Numeric and Weight
Now, this column is available as a property in Excel Pivot table
Voila, this new column in Excel is now really a Numeric column. The formatting options in Excel will really format this column.
Related SO Query:
SSAS, dimension numeric value filtering
I have a flat file and used conditional split to filter the record into a single row. For example, RecordType == "2" retrieves single row with record having multiple columns say A,B,C,D and E. I want to pass the result of Column C value to a variable. And then to use it to update the table like:
Update tablename
Set A = that variable
Where A is null
Could you please help me in find out the solution.
I would not use the variable but use a Ole DB Command object.
You set the connection.
Then add your SQL from above:
Update tablename Set A = ? Where A is null
The map to Col C.
However, what I might guess you are trying to do is add a column to your other record set that has the detail but no key.
I would use a script component to do this:
Similar to this example:
Importing Grouped Report Data to Database
I'm new to BIRT and I was wondering how i can access only one data of a MongoDB array. For now i can only specify a field and the field i want is in an array, so it gives me all the values for this field in an array. e.g.: [val1, val2, val3]
The problem is I only want one value to show because I want other fields form the same array.
i.e.:
COLUMN1 COLUMN2 COLUMN3
element[0].field1 element[0].field2 element[0].field3
element[1].field1 element[1].field2 element[1].field3
element[2].field1 element[2].field2 element[2].field3
and now it's giving me
COLUMN1
[element[0].field1, element[1].field1, element[2].field1]
and so on for the other columns.
Thanks!
I think I found it. It was in the Advanced settings from the Query section in the Data sets section..