Full Text Search in OrientDB JSON Data - database

I have following data in OrientDB 3.0.27 where some of the values are in JSON Array and some are string
{
"#type": "d",
"#rid": "#57:0",
"#version": 2,
"#class": "abc_class",
"user_name": [
"7/1 LIBOR Product"
],
"user_Accountability": [],
"user_Rollout_32_date": [],
"user_Brands": [
"AppNet"
],
"user_lastModificationTime": [
"2019-11-27 06:40:35"
],
"user_columnPercentage": [
"0.00"
],
"user_systemId": [
"06114a87-a099-0c30c60b49c4"
],
"user_lastModificationUser": [
"system"
],
"system_type": "Product",
"user_createDate": [
"2017-10-27 09:58:42"
],
"system_modelId": "bian_model",
"user_parent": [
"a12a41bd-af6f-0ca028af480d"
],
"user_Strategic_32_value": [],
"system_oeId": "06114a87-a099-0c30c60b49c4",
"user_description": [],
"#fieldTypes": "user_name=e,user_Accountability=e,user_Rollout_32_date=e,user_Brands=e,user_lastModificationTime=e,user_columnPercentage=e,user_systemId=e,user_lastModificationUser=e,user_createDate=e,user_parent=e,user_Strategic_32_value=e,user_description=e"
}
I have tried following queries:
select * from `abc_class ` where any() = ["AppNet"] limit 2;
select * from `abc_class ` where any() like '%a099%' limit 2;
Both of the above queries work since they are respecting the datatype of the field.
I want to run a contains query which will search in ANY field with ANY data type (like String, number, JSON Array, etc) more of like a - full text search.
select * from `abc_class ` where any() like '%AppNet%' limit 2;
The above query doesn't work since the real value is inside JSON Array. Tried almost all the things from filtering section documentation
How can I achieve full-text search like functionality with the existing data?
EDIT # 1
After doing more research now I'm able to atleast convert the array value into string and then run like operator on it, like below;
select * from `abc_class` where user_name.asString() like '%LIBOR%'
However, using any().asString() doesn't result any result
select * from `abc_class` where any().asString() like '%LIBOR%'
If the above query can be enhanced somehow to query any column as string, then the problem can be resolved.

If all the column values needs to be searched then we can create a JSON object of the full row data and convert it into String.
Then query the string with like keyword, as follows:
select * from `abc_class` where #this.toJSON().asString() like '%LIBOR%'
If we will be converting to #this.asString() directly then we'll be getting the count of array elements instead of the real data inside the array elements like below:
abc_class#57:4{system_modelId:model,system_oeId:14f4b593-a57d-4d37ad070a10,system_type:Product,user_lastModificationUser:[1],user_name:[1],user_description:[0],user_Accountability:[0],user_lastModificationTime:[1],user_Rollout_32_date:[0],user_Strategic_32_value:[0],user_createDate:[1],user_Brands:[0],user_parent:[1],user_systemId:[1],user_columnCompletenessPercentage:[1]} v2
Therefore, we need to first convert into JSON and then into String to query the full record using #this.toJSON().asString()
References:
https://orientdb.com/docs/last/sql/SQL-Methods.html
https://orientdb.com/docs/last/sql/SQL-Where.html
https://orientdb.com/docs/last/sql/SQL-Syntax.html

Related

Trying to parse json data without having access to OPEN_JSON

We have a production database that has compatibility level of 100, this cannot change and open_json requires a level of 130. So I'm trying to figure out a little work around
Here is an example json
SET #json = '{
"sessionid": "XXXXX",
"userid": "XXXX",
"scorm": [{
"name": "variable_name",
"value": "variable_value"
},
{
"name": "variable_name",
"value": "variable_value"
},
... Continues on like this
]
}';
So I am able to get values out of this JSON by doing the following
SELECT JSON(#json,'$.sessionid') as SESSIONID
SELECT JSON_VALUE(#json, '$.scorm[0].value') as FirstValue
But I have no idea how many will be inside the scorm: [{}] array, but we need all of them to save into a table. MAX there ever will be is probably 25 different values.
Soo I figured I could do something like:
//PSUEDO CODE
while(value != null)
DO
//Get values
select value = JSON_VALUE(#json, '$.scorm[0].value') as VALUE
select name = JSON_VALUE(#json, '$.scorm[0].name') as NAME
//Erase values from json
SET #json = JSON_VALUE(#json, '$.scorm[0], NULL)
SET #json = REPLACE(#json,'null,','');
//Insert into table
Insert (name,value) into TABLE
This is essentially getting the topmost value from the json array, and then setting it to 'NULL' and manipulating the string to get rid of it. Meaning now the json array has a new topmost value.
Then I would continue on like this and insert data into my table for each name/value.
This all seems like a terrible way to be doing it, and I can't imagine this is good, nor fast.
Am I missing something obvious? Some other-way I could be doing this (without access to OPEN_JSON()) ?

Variant column separate json values in file by comma?

This is a basic question, I am trying to break one variant row into multiple columns and running into an error.
Create or replace table App_versions(data variant);
CREATE or Replace FILE FORMAT x_json
TYPE = "JSON"
COMPRESSION = "GZIP"
FILE_EXTENSION= 'json.gz'
COPY INTO App_versions
FROM #~/staged
file_format = 'x_json'
on_error = 'skip_file';
list #~;
SELECT * FROM App_versions limit 10;
Select data:available,value::boolean as avail, data:color.value::string as col, data:name.value::string as title, data:version.value::float as version from App_versions;
Data Stored in Column
[
{
"available": false,
"color": "Indigo",
"name": "Bigtax",
"version": "2.2.9"
},
{
"available": false,
"color": "Khaki",
"name": "Solarbreeze",
"version": "7.00"
}
]
And I am running into the columns all to be Null values. What am I doing wrong?
I based it off of:https://support.snowflake.net/s/article/json-data-parsing-in-snowflake
If you want each { ... } object to land in it's own row, then use the STRIP_OUTER_ARRAY = TRUE file format option. Or you can FLATTEN() data on the fly after loading. To access multiple objects in single row without flattening, you have to include an index to specify which object you want -- for example... select data[0].available::boolean as avail ....

How to transform a JSON array nested inside an object inside another array in Postgres?

I'm using Postgres 9.6 and have a JSON field called credits with the following structure; A list of credits, each with a position and multiple people that can be in that position.
[
{
"position": "Set Designers",
people: [
"Joe Blow",
"Tom Thumb"
]
}
]
I need to transform the nested people array, which are currently just strings representing their names, into objects that have a name and image_url field, like this
[
{
"position": "Set Designers",
people: [
{ "name": "Joe Blow", "image_url": "" },
{ "name": "Tom Thumb", "image_url": "" }
]
}
]
So far I've only been able to find decent examples of doing this on either the parent JSON array or on an array field nested inside a single JSON object.
So far this is all I've been able to manage and even it is mangling the result.
UPDATE campaigns
SET credits = (
SELECT jsonb_build_array(el)
FROM jsonb_array_elements(credits::jsonb) AS el
)::jsonb
;
Create an auxiliary function to simplify the rather complex operation:
create or replace function transform_my_array(arr jsonb)
returns jsonb language sql as $$
select case when coalesce(arr, '[]') = '[]' then '[]'
else jsonb_agg(jsonb_build_object('name', value, 'image_url', '')) end
from jsonb_array_elements(arr)
$$;
With the function the update is not so horrible:
update campaigns
set credits = (
select jsonb_agg(jsonb_set(el, '{people}', transform_my_array(el->'people')))
from jsonb_array_elements(credits::jsonb) as el
)::jsonb
;
Working example in rextester.

Querying an array within an array with Postgres JSONB query

I have some JSON in a field in my Postgres 9.4 db and I want to find rows where the given name is a certain value, where the field is named model and the JSON structure is as follows:
{
"resourceType": "Person",
"id": "8a7b72b1-49ec-43e5-bd21-bc62674d9875",
"name": [
{
"family": [
"NEWMAN"
],
"given": [
"JOHN"
]
}
]
}
So I tried this: SELECT * FROM current WHERE model->'name' #> '{"given":["JOHN"]}'; (as well as various other guesses) but that does not match the above data. How should I do this?
Use the function jsonb_array_elements():
select t.*
from current t,
jsonb_array_elements(model->'name') names
where names->'given' ? 'JOHN'

Json creation in ruby for a list

I am new to Ruby.
I want to create a JSON file for a group of elements.
For this, I am using eachfunction to retrieve the datas. I want to create json as follows for the 4 length array,
'{
"desc":{
"1":"1st Value",
"2":"2nd value"
"3":"3rd Value",
"4":"4th value"
},
}'
This is my array iteration,
REXML::XPath.each( doc, "//time" ) { |element1|
puts element1.get_text
}
I know here is the simple code to generate a JSON,
require 'json/add/core'
class Item < Struct.new(:id, :name); end
chair = Item.new(1, 'chair')
puts JSON.pretty_generate(chair)
This syntax will generate a json as follows,
{
"json_class": "Item",
"v": [
1,
"chair"
]
}
But I'm not sure how to do that to make JSON for my elements as stated above. Google search didn't give me a proper way to do this.
Can anyone help me here?
it means this?:
require 'json'
my_arr= ["1st Value","2nd Value","3rd Value","4th Value"]
tmp_str= {}
tmp_str["desc"] = {}
my_arr.each do |x|
tmp_str["desc"]["#{x[0]}"] = x
end
puts JSON.generate(tmp_str)
you can iterate the string array ,then take the strings to hash object.JSON can easy to parse Hash objcect .

Resources