I've seen that Solr will allow you to index JSON:
http://wiki.apache.org/solr/UpdateJSON
However, none of the examples are nested. Can you index something like this and if not how is it normally handled?
{
name: 'ben',
state: 'california',
country: 'united states',
companies: [
{
name: 'google',
title: 'software engineer',
},
{
name: 'sherwin-williams',
title: 'web developer'
}
],
}
There are a couple ways to go. A json string can be stored explicitly, with serialization handled in the application layer. Elasticsearch uses this approach transparently.
For indexing, you can flatten the data using naming conventions. Mongodb uses such a syntax.
companies.name: ['google', 'sherwin-williams']
companies.title: ['software engineer', 'web developer']
Note in such a case a query like
<BooleanQuery: +companies.name:google +companies:web developer>
would match. If the position should matter, a more advanced SpanQuery would have to be used.
I had the same issue. We wanted to index in solr complicated json documents with arrays and maps ( much more complicated than the example that you posted).
At the end I modified the JsonLoader class to accept this kind of docuemnts. What it does , it flatten the json structure and allows the indexing of the fields and keeps the original json structure [company]. Finally it supports deep nesting
you can find the source code with some explanation on
http://www.solrfromscratch.com/2014/08/20/embedded-documents-in-solr/
On your example it will store/index [based on how you configure the fields] the following structure
name: 'ben',
state: 'california',
country: 'united states',
companies.0.name: 'google',
companies.0.title: 'software engineer',
companies.1.name: 'sherwin-williams',
companies.1.title: 'web developer'
companies_json:[
{
name: 'google',
title: 'software engineer',
},
{
name: 'sherwin-williams',
title: 'web developer'
}
]
M.
Nested Jsons can be indexed with the help of child documents in solr. We can make use of Block and join query parsers to query it.
Refer to this question
Related
I have a data that is set up like this in MongoDB Compass:
"person": "Isaac",
"city": { "$oid" : "049485053930a029g" }
I need it to be like this below, as a single object can have multiple cities:
"person": "Isaac",
"city": [{ "$oid" : "049485053930a029g" }]
I imported my data with a CSV file.
I tried to use this aggregation in Compass, but I got an error:
{
country: {$objectToArray: "$country"}
}
//$objectToArray requires a document input, found: objectId
WWhat solution can I implement to solve this problem?
I have a lot of data to add to my database, I do not want to do it manually on my back-end CMS.
You can try an update with aggregation pipeline query starting from MongoDB 4.2, to update all the document's city in an array,
db.collection.updateMany(
{},
[{
$set: {
city: ["$city"]
}
}]
)
Playground
Working with a frontend variable like below in javascript with many objects... (Note - the scores would be different for each user therefore I would need to be able to get the info from the frontend)
var campgrounds = [{ name: "State Park #1" score: 0, }, { name: "State Park #2" score: 0, }, { name: "State Park #3" score: 0 }]
How would I be able to store that data in MongoDB? Could I somehow loop through the data? Or could I store the entire variable "campgrounds" in Mongo? like $("#quizData").value(campgrounds)
Would I set up a Schema like below? Even if I did I'm not sure how to actually get the data in the variable into Mongo. Not sure how to accomplish my goal. Thanks for any help! :)
var campgroundsSchema = new Schema ({
"campgrounds" : [{
name : {type: String},
score : {type : Number}
}],
});
If you connect to mongo using mongoshell ...
mongo --host localhost:27017
you can issue the following javascript commands...
use campgroundsdb
var campgrounds = [{ name: "State Park #1", score: 0 }, { name: "State Park #2", score: 0, }, { name: "State Park #3", score: 0 }]
db.campgroundscollection.insertMany(campgrounds)
Then find them by issuing...
db.campgroundscollection.find().pretty()
If you want to create a program written in JavaScript it will need the ability to connect to the mongodb process. Node.js tutorials fit this requirement...
https://www.w3schools.com/nodejs/nodejs_mongodb.asp
I had to upgrade from 4.1.2 to 5.1.1 for the sole sake of widget columns. I'm having trouble getting hasone associations to work.
I've got a model that looks like this:
Ext.define('PP.model.LM.FOOModel', {
extend: 'Ext.data.Model',
requires: [
'Ext.data.field.Field'
],
fields: [
{
name: 'ID'
},
//Boatload of simple fields
],
hasOne: {
model: 'PP.model.LM.FOO1Model',
name: 'FOO1',
associationKey: 'FOO1'
}
});
When I interact with the model, there is no getter \ setter methods, and FOO1Model's data is only present as an object that can be accessed by
record.get('FOO1');
Could someone please point out what exactly am I doing wrong?
I tried doing it with the new approach - creating a field with a reference to the desired model. It works fine when I call setFoo1, and then do a get. But.
When I make an Ajax request, and try reading received JSON using Ext.data.reader.Json, it seems to fail to understand that a certain property in the object is in fact an associated model. The data in Foo1 appears in the model as an object in Foo1 property. The reqest returns an array of models that have many FooModels.
The Json looks like this:
{
"root": [{
"ID": 4241,
"Foos": [{
ID: 2237,
"Foo1": {
"ID": 1216
}
}]
}],
"success": true
}
It seems that hasOne doesn't exist in ExtJS 5.1 it is now done like this:
fields: [{
name: 'addressId',
reference: 'Address',
unique: true
}]
Check Model api for more info.
Had to debug the Ext.data.reader.Json to understand. The key was passing associationKey in reference config. Unless it's specified the reader will assume that the data for the association resides under '_foo1' field in JSON.
Is it mentioned anywhere in sencha docs? I don't think so. Am I supposed to feel like an imbecile for not guessing that?
Sample for unfortunate poor sods like me, who might encounter the problem in future:
{
name: 'Foo1',
reference: {
type: 'FOO1Model',
association: 'Foo1',
associationKey: 'Foo1'
},
unique: true
}
I have unusual response from server like this
[
{
id: 1,
name: "Alexandr",
children: [
{
id: 2,
name: "Stephan"
},
{
id: 3,
name: "Nick"
}
]
},
{
id: 4,
name: "David",
children: [
{
id: 3,
name: "Nick"
},
{
id: 6,
name: "Paul"
}
]
}
]
i would like to normalize this response to receive a diction with all people. So, i use normalizr go flat this
const people= new Schema('people');
people.define({
Children: arrayOf(people),
NotOwnChildren: arrayOf(people)
});
let normalized = normalize(response.data, arrayOf(people));
but doing like this i get an error
"When merging two people, found unequal data in their "Children" values. Using the earlier value."
How can i adjust normalizr to merge people with same id (update entities with newest data)?
It looks like you're getting two people that have differing values for one of their keys (I'm assuming your example input is truncated).
For Normalizr#2:
You can use a custom mergeIntoEntity function to resolve the issue manually.
For Normalizr#>=3.0.0, you'll need use mergeStrategy.
I'm not able to describe my thought precisely in words, so here's an example:
[
{
'description': 'fruits',
'examples': [
{
'name': 'Apple',
'color': ['red', 'green']
},
{
'name': 'Banana',
'color': 'yellow'
}
]
},
{
'description': 'vegetables',
'examples': [
{
'name': 'Tomato',
'color': 'red'
},
{
'name': 'Carrot',
'color': 'orange'
}
]
},
{
'description': 'Wweets',
'examples': [
{
'name': 'Chocolate',
'color': ['brown', 'black', 'white']
},
{
'name': 'Candy',
'color': 'various'
}
]
}
]
Let's go step by step:
If I want to see all food categories, I query by the following command
db.food.find()
I want to see the vegetables
db.food.find({ 'description': 'vegetables' })
Now let's say I forgot how a Carrot looks like (lol). What do I do? I tried the following (Native node.js MongoDB driver):
collection.find({'examples.name': 'Carrot'}, function(err, example){
console.log(example)
// It still returns me the whole object!
});
As a result I expected MongoDB to return the next highest instance of the object. For example I wanted to do this.
console.log(example.color)
// 'orange'
Do you have any idea? I'm new to document oriented DBs :/
When you store a bunch of objects in a single document you will (by default) get back the entire document. [*]
When one of the fields of the document is an array, you will get back the full array if the item you are trying to match is found in the array.
Don't fall for the temptation to cram everything into a single document if you will normally be getting back only a subset of those things.
You have an alternative available:
You can store a collection of foods where every food has a field "type" which is "fruit" or "vegetable" or "...". You can still query for all foods, or just foods of type "fruit" or just food with name "carrot", etc.
Arrays are great for lists of attributes of a particular object/document, they are not as good when you cram documents into them that you then want to get back as first-class objects.
[*] there is a way to project and get only a subset of the fields, but you will still get back entire fields.