I have a large collection of different products with various fields.
Once field is named 'ar_related'
This holds 0 or many different arrays. For example it could hold:
also_viewed, also_bought, bought_together, buy_after_viewing
Here is an example of a document:
{
"_id" : ObjectId("5a95cef390bd8fbf1c699d6d"),
"ar_asin" : "0078764343",
"ar_description" : "Brand new sealed!",
"ar_price" : 37.98,
"ar_imUrl" : "http://ecx.images-amazon.com/images/I/513h6dPbwLL._SY300_.jpg",
"ar_related" : {
"also_bought" : [
"B000TI836G",
"B003Q53VZC",
"B00EFFW0HC",
"B003VWGBC0",
"B003O6G5TW",
"B0037LTTRO",
"B002I098JE",
"B008OQTS0U",
"B005EVEODY",
"B008B3AVNE",
"B000PE0HBS",
"B00354NAYG",
"B0050SYPV2",
"B00503E8S2",
"B0050SY77E",
"B0022TNO7S",
"B0056WJA30",
"B0023CBY4E",
"B002SRSQ72",
"B005EZ5GQY",
"B004XACA60",
"B00273Z9WM",
"B004HX1QFY",
"B002I0K50U"
],
"bought_together" : [
"B002I098JE"
],
"buy_after_viewing" : [
"B0050SY5BM",
"B000TI836G",
"B0037LTTRO",
"B002I098JE"
]
},
"ar_salesRank" : {
"Video Games" : 28655
},
"ar_categories" : [
[
"Video Games",
"Xbox 360",
"Games"
]
]
}
What I am trying to achieve is to return (and count) all products which contain the 'bought_together' array within the ar_related object. Some products contain it and some do not.
I have tried a few different approaches, mostly with the $elemMatch operator but am having no luck. I am new to NoSQL as from a SQL background.
Any response will be appreciated.
Try this
db.getCollection('products').find({ "ar_related": {
$exists: true,
$elemMatch: { "bought_together": {$exists: true} }
}
})
After some painful research, Ive found this answer.
db.P14162135_products.find({
"ar_related.brought_together":{$exists: true}
}).count();
Thank you all for the responses
Related
I am starting to use the NoSQL database MongoDB and doing some courses of MongoDB.
I am trying to resolve two querys but I can't :(
Are the followings:
What is the name of the listing in the sample_airbnb.listingsAndReviews dataset that accommodates more than 6 people and has exactly 50 reviews?
db.listingsAndReviews.find({"$and": [{"accommodates" {"$gt": 6}}, {"reviews": {"$size": 50}}]}).count()
How many companies in the sample_training.companies dataset were either founded in 2004 and either have the social category_code or web category_code, or were founded in the month of October and also either have the social category_code or web category_code?
db.companies.find({"$and": [{"$or": ["foundation_year": 2004], ["founded_month": 10]}],[{"$or": ["category_code": "web"], ["category_code": "social"]}]}).count()
But any of these two querys works, what can I be doing wrong?
Thanks in advance!
MongoDB provides an implicit AND operation when specifying a comma separated list of expressions. You only need to use $AND if you are evaluating the same field.
For the first question you must use the number_of_reviews, like so:
{ number_of_reviews: 50, accommodates: { $gt: 6 } }
For the second question you must use the correct sintax for $OR, like so:
{ $and: [ { $or: [ { founded_year: 2004 }, { founded_month: 10 } ] }, { $or: [ { category_code: 'social'}, { category_code: 'web' } ] } ] }
This was the query that I used :
db.companies.find({ "$or" : [ {"founded_year" : 2004 , "$or" : [ {"category_code" : "social"} , {category_code : "web"} ] } , {"founded_month" : 10 , "$or" : [ {"category_code" : "social"} , {category_code : "web"} ] } ] }).count()
Answer was correct.
db.companies.find({"$and":[{"$or":[{"founded_year": 2004},{"founded_month": 10}]},{"$or":[{"category_code": "social"}, {"category_code": "web"}]}]}).count()
I've got an array of Project ID's, for example:
[ 'ExneN3NdwmGPgRj5o', 'hXoRA7moQhqjwtaiY' ]
And in my Questions collection, I've got a field called 'project', which has a string of a project Id. For example:
{
"_id" : "XPRbFupkJPmrmvcin",
"question" : "Vraag 13",
"answer" : "photo",
"project" : "ExneN3NdwmGPgRj5o",
"datetime_from" : ISODate("2017-01-10T08:01:00Z"),
"datetime_till" : ISODate("2017-01-10T19:00:00Z"),
"createdAt" : ISODate("2017-01-10T08:41:39.950Z"),
"notificationSent" : true
}
{
"_id" : "EdFH6bo2xBPht5kYW",
"question" : "sdfadsfasdf",
"answer" : "text",
"project" : "hXoRA7moQhqjwtaiY",
"datetime_from" : ISODate("2017-01-11T11:00:00Z"),
"datetime_till" : ISODate("2017-01-11T17:00:00Z"),
"createdAt" : ISODate("2017-01-10T10:21:42.147Z"),
"notificationSent" : false
}
Now I want to return all documents of the Questions collection, where the Project (id) is one of the value's from the Array.
To test if it's working, I'm first trying to return one document.
Im console.logging like this:
Questions.findOne({project: { $eq: projectArray }})['_id'];
but have also tryed this:
Questions.findOne({project: { $in: [projectArray] }})['_id'];
But keep getting 'undefined'
Please try this.
Questions.find({project: { $in: projectArray }}) => for fetching all docs with those ids
Questions.findOne({project: { $in: projectArray }}) => if you want just one doc
I try to update arrays of multiple document with this query :
db.BusinessRequest.update({"DealTypes": { $exists: true }, "DealTypes.DisplayName": "Minority trade sale" }, {$set:{"DealTypes.$.DisplayName":"Minority"}}, false,true );
but when there is a match, it only updates the first row of my array whereas the displayName does not match with the first.
I use IntelliShell of MongoChef software.
My document looks like this :
{
"_id" : BinData(4, "IKC6QJRGSIywmKTKKRfTHA=="),
"_t" : "InvestorBusinessRequest",
"Title" : "Business Request 000000002",
"DealTypes" : [
{
"_id" : "60284B76-1F45-49F3-87B5-5278FF49A304",
"DisplayName" : "Majority",
"Order" : "001"
},
{
"_id" : "64A52AFE-2FF5-426D-BEA7-8DAE2B0E59A6",
"DisplayName" : "Majority trade sale",
"Order" : "002"
},
{
"_id" : "C07AE70D-4F62-470D-BF65-06AF93CCEBFA",
"DisplayName" : "Minority trade sale",
"Order" : "003"
},
{
"_id" : "F5C4390A-CA7D-4AC8-873E-2DC43D7F4158",
"DisplayName" : "Equity fund raising",
"Order" : "004"
}
]
}
How can I achieve this please ? Thanks in advance
EDIT :
This line works :
db.BusinessRequest.update({"DealTypes": { $exists: true }, "DealTypes": { $elemMatch: {"DisplayName": "Majority trade sale"}}}, {$set:{"DealTypes.$.DisplayName":"Majority"}}, false,true );
Please try this :
db.BusinessRequest.find().forEach( function(doc) {
do {
db.BusinessRequest.update({{"DealTypes": { $exists: true }, "DealTypes.DisplayName": "Minority trade sale" },
{$set:{"DealTypes.$.DisplayName":"Minority"}});
} while (db.getPrevError().n != 0);
})
or
You cannot modify multiple array elements in a single update operation. Thus, you'll have to repeat the update in order to migrate documents which need multiple array elements to be modified. You can do this by iterating through each document in the collection, repeatedly applying an update with $elemMatch until the document has all of its relevant comments replaced.
db.BusinessRequest.update({"DealTypes": { $exists: true }, "DealTypes": { $elemMatch: {"DisplayName": "Majority trade sale"}}}, {$set:{"DealTypes.$.DisplayName":"Majority"}}, false,true );
If you need efficiency in the search then I suggest you to normalise schema where each row is kept in separate document.
Please execute the following script in your mongo shell :
db.BusinessRequest.find({"DealTypes":{$exists:true}}).forEach(function(item)
{
for(i=0;i < item.DealTypes.length;i++)
{
if(item.DealTypes[i].DisplayName === 'Minority trade sale'){
item.DealTypes[i].DisplayName = 'Minority';
}
}
db.BusinessRequest.save(item);
});
Last two arguments in your update have a problem.
This is the form of update() method in mongodb
db.collection.update(
<query>,
<update>,
{
upsert: <boolean>,
multi: <boolean>,
writeConcern: <document>
}
)
I believe your update should be like this;
db.BusinessRequest.update
( {"DealTypes": { $exists: true }, "DealTypes.DisplayName": "Minority trade sale" }
, {$set:{"DealTypes.$.DisplayName":"Minority"}}
{ upsert : false, multi : true });
I have a collection in mongo that I need to migrate to Neo4j. To do that, I will export it to CSV. Then, I'll import the resultant CSV to Neo4j using Cypher. The documents from the collection have an object with an array that contains objects with arrays inside them. Take a look at the JSON above:
"services" : [
{
"max_id" : "646767779849326594",
"log" : [
{
"date" : 1443024000,
"steps" : 6
},
{
"date" : 1442512800,
"steps" : 1
}
],
"service" : "home_timeline"
},
{
"max_id" : 0.0,
"log" : [
{
"date" : 1443024000,
"steps" : 4
},
{
"date" : 1442512800,
"steps" : 1
}
],
"service" : "user_timeline"
},
{
"max_id" : 0.0,
"log" : [
{
"date" : 1443024000,
"steps" : 6
},
{
"date" : 1442512800,
"steps" : 1
}
],
"service" : "mentions_timeline"
}
]
How can I import this to Neo4 properly?? I already found a solution to import arrays. But I didn't find nothing similar to my problem. How should be the header of the CSV? How should be the Cypher code to get these objets??
You can use JSON as a parameter to a Cypher query. There are a few examples of this here and here.
With your example something like this:
WITH {json} AS data
UNWIND data.services AS service
// Insert data for each service.
MERGE (s:Service { "service_name": service.service})
SET s.max_id = service.max_id
FOREACH (log IN service.logs | CREATE (l:Log {date: log.date, steps: log.steps})<-[:LOGGED]-(s))
There is also a tool for translating data from MongoDB document data model to Neo4j property graph model that you might find useful: https://github.com/neo4j-contrib/neo4j_doc_manager
I have vehicles collection with the following schema, all the articles are just general products (no child products included):
{
"_id" : ObjectId("554995ac3d77c8320f2f1d2e"),
"model" : "ILX",
"year" : 2015,
"make" : "Acura",
"motor" : {
"cylinder" : 4,
"liters" : "1.5"
},
"products" : [
ObjectId("554f92433d77c803836fefe3"),
...
]
}
And I have products collection, some of them are general products related with warehouse sku's and some products are "son" products that fit in multiples general products, these son products are also related with warehouse sku's:
general products
{
"_id" : ObjectId("554b9f223d77c810e8915539"),
"brand" : "Airtex",
"product" : "E7113M",
"type" : "Fuel Pump",
"warehouse_sku" : [
"1_5551536f3d77c870fc388a04",
"2_55515e163d77c870fc38b00a"
]
}
child product
{
"_id" : ObjectId("55524d0c3d77c8ba9cb2d9fd"),
"brand" : "Performance",
"product" : "P41K",
"type" : "Repuesto Bomba Gasolina",
"general_products" : [
ObjectId("554b9f223d77c810e8915539"),
ObjectId("554b9f123d77c810e891552f")
],
"warehouse_sku" : [
"1_555411043d77c8066b3b6720",
"2_555411073d77c8066b3b6728"
]
}
My question is to obtain a list of general products (_id and general_products inside child products) for warehouse_sku that follow the pattern : 1_
I have created an aggregate query with the following structure:
list_products = db.getCollection('products').aggregate([
... {$match: {warehouse_sku: /^1\_/}},
... {$group: { "_id": "$_id" } }
... ])
And that query give me successfully a result :
{ "_id" : ObjectId("55524d0c3d77c8ba9cb2d9fd") }
{ "_id" : ObjectId("554b9f223d77c810e8915539") }
but I need to obtain a list of general products so I can use $in in the vehicles collection.
list_products = [ ObjectId("55524d0c3d77c8ba9cb2d9fd"), ObjectId("554b9f223d77c810e8915539")]
example: db.vehicles.find({products:{$in: list_products}})
This last query I could not achieve it.
Use the aggregation cursor's map() method to return an array of ObjectIds as follows:
var pipeline = [
{$match: {warehouse_sku: /^1\_/}},
{$group: { "_id": "$_id" } }
],
list_products = db.getCollection('products')
.aggregate(pipeline)
.map(function(doc){ return doc._id });
The find() cursor's map() would work here as well:
var query = {'warehouse_sku': /^1\_/},
list_products = db.getCollection('products')
.find(query)
.map(function(doc){ return doc._id });
UPDATE
In pymongo, you could use a lambda function with the map function. Because map expects a function to be passed in, it also happens to be one of the places where lambda routinely appears:
import re
regx = re.compile("^1\_", re.IGNORECASE)
products_cursor = db.products.find({"warehouse_sku": regx})
list_products = list(map((lambda doc: doc["_id"]), products_cursor))