Batch node relationship creation in cypher/neo4j - database

What is the most efficient way to break down this CREATE cypher query?
The end pattern is the following:
(newTerm:term)-[:HAS_META]->(metaNode:termMeta)
In this pattern this is a single newTerm node and about ~25 termMeta nodes. The HAS_META relationship will have a single property (languageCode) that will be different for each termMeta node.
In the application, all of these nodes and relationships will be created at the same time. I'm trying to determine the best way to add them.
Is there anyway to add these without having to have perform individual query for each TermMeta node?
I know you can add multiple instances of a node using the following query format:
"metaProps" : [
{"languageCode" : "en", "name" : "1", "dateAdded": "someDate1"},
{"languageCode" : "de", "name" : "2", "dateAdded": "someDate2"},
{"languageCode" : "es", "name" : "3", "dateAdded": "someDate3"},
{"languageCode" : "fr", "name" : "3", "dateAdded": "someDate4"}
]
But you can only do that for one type of node at a time and there (as far as I can tell) is no way to dynamically add the relationship properties that are needed.
Any insight would be appreciated.

There's no really elegant way to do it, as far as I can tell—from your example, I'm assuming you're using parameters. You can use a foreach to loop through the params and do a create on each one, but it's pretty ugly, and requires you to explicitly specify literal maps of your properties. Here's what it would look like for your example:
CREATE (newTerm:term)
FOREACH ( props IN {metaProps} |
CREATE newTerm-[:HAS_META {languageCode: props.languageCode}]->
(:termMeta {name: props.name, dateAdded: props.dateAdded})
)
WITH newTerm
MATCH newTerm-[rel:HAS_META]->(metaNode:termMeta)
RETURN newTerm, rel, metaNode
If you don't need to return the results, you can delete everything after the FOREACH.

Select and name each vertex differently and then create relations using it.
For ex
match (n:Tag), (m:Account), (l:FOO) CREATE (n)-[r:mn]->(m),(m)-[x:ml]->(l)
match (n:Tag{a:"a"}), (m:Account{b:"x"}), (l:FOO) CREATE (n)-[r:mn]->(m),(m)-[x:ml]->(l)

Related

Is there a way to auto generate ObjectIds inside arrays of MongoDB?

Below is my MongoDb collection structure:
{
"_id": {
"$oid": "61efa44933eabb748152a250"
},
"title": "my first blog",
"body": "Hello everyone,wazuzzzzzzzzzzzzzzzzzup",
"comments": [{
"comment": "fabulous work bruhv",
}]
}
}
Is there a way to auto generate ids for comments without using something like this:
db.messages.insert({messages:[{_id:ObjectId(), message:"Message 1."}]});
I found the above method from the SO question:
mongoDB : Creating An ObjectId For Each New Child Added To The Array Field
But someone in the comments pointed out that:
"I have been looking at how to generate a JSON insert using ObjectID() and in my travels have found that this solution is not very good for indexing. The proposed solution has _id values that are strings rather than real object IDs - i.e. "56a970405ba22d16a8d9c30e" is different from ObjectId("56a970405ba22d16a8d9c30e") and indexing will be slower using the string. The ObjectId is actually represented internally as 16 bytes."
So is there a better way to do this?

loopbackjs/mongodb returning duplicate key error on bulk upsert operation

I'm trying to upsert multiple documents at once by using a PUT verb to a loopback based Rest API.
The body of the message contains an array of json objects.
[
{"_id" : "1",
"data" : "foo"
},
{"_id" : "2",
"data" : "bar"
}
]
On an empty database this works just fine (create). All documents are created as expected.
But if I run the same call again (update), I receive an error containing an array of similar error messages:
E11000 duplicate key error index: testdatabase.node.$id dup key: { : "1" }
After some further investigation, I found out that if I pass a single object, the upsert works fine.
{"_id" : "1",
"data" : "foo"
}
BUT: if I pas an array with the same single object, the error is back.
[
{"_id" : "1",
"data" : "foo"
}
]
Single upserts are not an option, because I have to update thousands of documents using the Rest Api.
loopback version: 2.22
The upsert function takes a single model as a paramater.
Take a look at the documentation.
In order to update different models as you want, you will have to use the native driver.
EDIT
I dug into loopback source code and found out that the PUT request uses the upsert function of the connector.
Which means that it expects one model and not an array, maybe loopback doesn't enforce it.
You can find the relevant code here.
Anyway, the closest thing to upserting multiple models is the bulk operation of mongodb, but this leads to self implementation.

Referencing arrays that are nested within multiple objects within MongoDB with Express js

So in my MongoDB Collection I have this structure:
"_id" : "Object("-----------")
"name" : "John Doe"
"tool" : {
"hammer" : {
"name" : "hammer 1",
"characteristics" : [
{
"length" : "9 inches"
},
{
"weight" : "4 pounds"
}
]
I know the data may seem a little strange but I can't put the actual data online so I had to input some dummy data. So essentially what I would like to do is be able to update the array that is nested within those objects. So I would like to be able to update the weight or add a new characteristic that I haven't previously entered into it. So for example, add in "metal" : "steel" as a new entry into the array. Currently I'm using a Rest API built in Node.js and Express.js to edit the db. When I was trying to figure out how to dig down this deep I was able to do it with an array at the highest level, however I haven't been able to figure out how to access an array when its embedded like this. So what I was wondering if anybody knew if it was even possible to edit an array this far down? I can post code from controller.js and server.js file if needed but I figured I'd see if it's even possible to do before I start posting it. Any help would be greatly appreciated!
You can use findAndModify to $push it into the array. You have to specify the path precisely though:
db.tools.findAndModify( {
query: { name: "John Doe"},
update: { $push: { tool.hammer.characteristics: {metal: "steel"} }
} );

MongoDB - Pull from array of objects

I have a collection
{
"_id" : ObjectId("534bae30bf5049a522e502fe"),
"data" : [
{
"0" : {
"content" : "1",
"type" : "text",
"ident" : true
},
"1" : {
"content" : "",
"type" : "text",
"ident" : false
}
},
{
"0" : {
"content" : "2",
"type" : "text",
"ident" : true
},
"1" : {
"content" : "",
"type" : "text"
}
}
]
}
content is unique.
How would i remove the object that matches content: '2'?
I have tried this:
data:{$pull:{"content": deletions[i]}}
where deletions [i] is the content.
and several variations, but i can not get it to work. What am I missing?
As per you comment, you should be worried. I have seen this a few times particularly with PHP applications ( and PHP has this funny kind of notation in dumping arrays ).
The problem is the elements such as the "0" and "1" create sub-documents in themselves, and as opposed to just leaving everything under that structure as the sub-document in itself as the the array member then you run into a problem with accessing individual members of the array as the paths used need to be "absolute".
So this actually "forces" no other possible option to access the elements by what would be the equivalent "dot notation" form. Except in this case it's not just the "nth" element of the array, but the actual path you need to address.
But if this does actually work for you, and it does seem like "someone" was trying to avoid
the problems with positional updates under "nested" arrays ( see the positional $ operator documentation for details ) then you update can be performed like this:
The basic statement is as follows:
db.collection.update(
{
"data.0.context": 2
},
{
"$pull": { "data.$.0.context": 2 }
}
)
That does "seem" to be a bit of a funny way to write this, but on investigating the actual structure you have then you should be able to see the reasons why this is needed. Essentially this meets the requirements of using the positional $ operator to indicate the index of the first matched element in the array ( aka "data" ) and then uses the standard sub-document notation in order to specify the path to the element to be updated.
So of course this poses a problem if the element is actually in an unknown position. But the thing to consider is which usage of the array is actually important to you given the documented limitation? If yo need to match the position of the "inner" element, then change the structure to put the arrays in there.
But always understand the effects of the limitation, and try to model according to what the engine can actually do.

mongodb - retrieve array subset

what seemed a simple task, came to be a challenge for me.
I have the following mongodb structure:
{
(...)
"services": {
"TCP80": {
"data": [{
"status": 1,
"delay": 3.87,
"ts": 1308056460
},{
"status": 1,
"delay": 2.83,
"ts": 1308058080
},{
"status": 1,
"delay": 5.77,
"ts": 1308060720
}]
}
}}
Now, the following query returns whole document:
{ 'services.TCP80.data.ts':{$gt:1308067020} }
I wonder - is it possible for me to receive only those "data" array entries matching $gt criteria (kind of shrinked doc)?
I was considering MapReduce, but could not locate even a single example on how to pass external arguments (timestamp) to Map() function. (This feature was added in 1.1.4 https://jira.mongodb.org/browse/SERVER-401)
Also, there's always an alternative to write storedJs function, but since we speak of large quantities of data, db-locks can't be tolerated here.
Most likely I'll have to redesign the structure to something 1-level deep, like:
{
status:1,delay:3.87,ts:138056460,service:TCP80
},{
status:1,delay:2.83,ts:1308058080,service:TCP80
},{
status:1,delay:5.77,ts:1308060720,service:TCP80
}
but DB will grow dramatically, since "service" is only one of many options which will append each document.
please advice!
thanks in advance
In version 2.1 with the aggregation framework you are now able to do this:
1: db.test.aggregate(
2: {$match : {}},
3: {$unwind: "$services.TCP80.data"},
4: {$match: {"services.TCP80.data.ts": {$gte: 1308060720}}}
5: );
You can use a custom criteria in line 2 to filter the parent documents. If you don't want to filter them, just leave line 2 out.
This is not currently supported. By default you will always receive the whole document/array unless you use field restrictions or the $slice operator. Currently these tools do not allow filtering the array elements based on the search criteria.
You should watch this request for a way to do this: https://jira.mongodb.org/browse/SERVER-828
I'm attempting to do something similar. I tried your suggestion of using the GROUP function, but I couldn't keep the embedded documents separate or was doing something incorrectly.
I needed to pull/get a subset of embedded documents by ID. Here's how I did it using Map/Reduce:
db.parent.mapReduce(
function(parent_id, child_ids){
if(this._id == parent_id)
emit(this._id, {children: this.children, ids: child_ids})
},
function(key, values){
var toReturn = [];
values[0].children.forEach(function(child){
if(values[0].ids.indexOf(product._id.toString()) != -1)
toReturn.push(child);
});
return {children: toReturn};
},
{
mapparams: [
"4d93b112c68c993eae000001", //example parent id
["4d97963ec68c99528d000007", "4debbfd5c68c991bba000014"] //example embedded children ids
]
}
).find()
I've abstracted my collection name to 'parent' and it's embedded documents to 'children'. I pass in two parameters: The parent document ID and an array of the embedded document IDs that I want to retrieve from the parent. Those parameters are passed in as the third parameter to the mapReduce function.
In the map function I find the parent document in the collection (which I'm pretty sure uses the _id index) and emit its id and children to the reduce function.
In the reduce function, I take the passed in document and loop through each of the children, collecting the ones with the desired ID. Looping through all the children is not ideal, but I don't know of another way to find by ID on an embedded document.
I also assume in the reduce function that there is only one document emitted since I'm searching by ID. If you expect more than one parent_id to match, than you will have to loop through the values array in the reduce function.
I hope this helps someone out there, as I googled everywhere with no results. Hopefully we'll see a built in feature soon from MongoDB, but until then I have to use this.
Fadi, as for "keeping embedded documents separate" - group should handle this with no issues
function getServiceData(collection, criteria) {
var res=db[collection].group({
cond: criteria,
initial: {vals:[],globalVar:0},
reduce: function(doc, out) {
if (out.globalVar%2==0)
out.vals.push({doc.whatever.kind.and.depth);
out.globalVar++;
},
finalize: function(out) {
if (vals.length==0)
out.vals='sorry, no data';
return out.vals;
}
});
return res[0];
};

Resources