MongoDb Collection Example (Person):
{
"id": "12345",
"schools": [
{
"name": "A",
"zipcode": "12345"
},
{
"name": "B",
"zipcode": "67890"
}
]
}
Desired output:
{
"id": "12345",
"schools": [
{
"zipcode": "12345"
},
{
"zipcode": "67890"
}
]
}
My current partial code for retrieving all:
collection.find({}, {id: true, schools: true})
I am querying the entire collection. But I only want to return zipcode part of school element, not other fields (because the actual school object might contain much more data which I do not need). I could retrieve all and remove those un-needed fields (like "name" of school) in code, but that's not what I am looking for. I want to do a MongoDb query.
You can use the dot notation to project specific fields inside documents embedded in an array.
db.collection.find({},{id:true, "schools.zipcode":1}).pretty()
Related
I have a collection in my MongoDB that contains one field that is an array.
Refer to the data above, the field 'Courses' is an array.
The JSON format of the data is like this:
{
"_id": {
"$oid": "60eb59b98a970a20865142e8"
},
"Name": "Sadia",
"Age": 24,
"Institute": "IBA",
"Courses": [{
"Name": "ITP",
"Grade": "A-"
}, {
"Name": "OOP",
"Grade": "A-"
}]
}
I am aware that there is a way in case its an object, but could not find a way on how to read this data using Talend since it contains an array.
I am having trouble with querying a MongoDB collection with an array inside.
Here is the structure of my collection that I am querying. This is one record:
{
"_id": "abc123def4567890",
"profile_id": "abc123def4567890",
"image_count": 2,
"images": [
{
"image_id": "ABC123456789",
"image_url": "images/something.jpg",
"geo_loc": "-0.1234,11.234567890",
"title": "A Title",
"shot_time": "01:23:33",
"shot_date": "11/22/2222",
"shot_type": "scenery",
"conditions": "cloudy",
"iso": 16,
"f": 2.4,
"ss": "1/545",
"focal": 6.0,
"equipment": "",
"instructions": "",
"upload_date": 1234567890,
"update_date": 1234567890
},
{
"image_id": "ABC123456789",
"image_url": "images/something.jpg",
"geo_loc": "-0.1234,11.234567890",
"title": "A Title",
"shot_time": "01:23:33",
"shot_date": "11/22/2222",
"shot_type": "portrait",
"conditions": "cloudy",
"iso": "16",
"f": "2.4",
"ss": "1/545",
"focal": "6.0",
"equipment": "",
"instructions": "",
"upload_date": 1234567890,
"update_date": 1234567890
}
]
}
Forgive the formatting, I didn't know how else to show this.
As you can see, it's a profile with a series of images within an array called 'images' and there are 2 images. Each of the 'images' array items contain an object of attributes for the image (url, title, type, etc).
All I want to do is to return the object element whose attributes match certain criteria:
Select object from images which has shot_type = "scenery"
I tried to make it as simple as possible so i started with:
find( { "images.shot_type": "scenery" } )
This returns the entire record and both the images within. So I tried projection but I could not isolate the single object within the array (in this case object at position 0) and return it.
I think the answer lies with projection but I am unsure.
I have gone through the MongoDB documents for hours now and can't find inspiration. I have read about $elemMatch, $, and the other array operators, nothing seems to allow you to single out an array item based on data within. I have been through this page too https://docs.mongodb.com/manual/tutorial/query-arrays/ Still can't work it out.
Can anyone provide help?
Have I made an error by using '$push' to populate my images field (making it an array) instead of using '$set' which would have made it into an embedded document? Would this have made a difference?
Using aggregation:
db.collection.aggregate({
$project: {
_id: 0,
"result": {
$filter: {
input: "$images",
as: "img",
cond: {
$eq: [
"$$img.shot_type",
"scenery"
]
}
}
}
}
})
Playground
You can use $elemMatch in this way (simplified query):
db.collection.find({
"profile_id": "1",
},
{
"images": {
"$elemMatch": {
"shot_type": 1
}
}
})
You can use two objects into find query. The first will filter all document and will only get those whose profile_id is 1. You can omit this stage and use only { } if you wnat to search into the entire collection.
Then, the other object uses $elemMatch to get only the element whose shot_type is 1.
Check an example here
I have a simple string "PART_NUMBER" value as a field in solr. I would like to add an additional field which places that value in a URL field. To do this, I created a new field type, field, and copy field
"add-field-type": {
"name": "endpoint_url",
"class": "solr.TextField",
"positionIncrementGap": "100",
"analyzer": {
"tokenizer": {
"class": "solr.KeywordTokenizerFactory"
},
"filters": [
{
"class": "solr.PatternReplaceFilterFactory",
"pattern": "([\\s\\S]*)",
"replacement": "http://myurl/$1.jpg"
}
]
}
},
"add-field": {
"name": "URL",
"type": "endpoint_url",
"stored": true,
"indexed": true
},
"add-copy-field":{ "source":"PART_NUMBER", "dest":"URL" }
As some of you probably guessed, my query output looks like
{
"id": "1",
"PART_NUMBER": "ABCD1234",
"URL": "ABCD1234",
"_version_": 1645658574812086272
}
Because the endpoint_url fieldtype only modifies the index. Indeed, when doing my analysis, I get
http://myurl/ABCD1234.jpg
My question: Is there any way to apply a tokenizer or filter and feed it back in to the field value? I would prefer this output when returning the result:
{
"id": "1",
"PART_NUMBER": "ABCD1234",
"URL": "http://myurl/ABCD1234.jpg",
"_version_": 1645658574812086272
}
Is this possible to do in Solr?
Solution was posted here:
Custom Solr analyzers not being used during indexing
I need to use an Update Processors In order to change the field value before analysis. The process can be found here:
https://lucene.apache.org/solr/guide/8_1/update-request-processors.html
I have a noSql (Cloudant) database
-Within the database we have documents where one of the document fields represents “table” (type of document)
-Within the documents we have fields that represent links other documents within the database
For example:
{_id: 111, table:main, user_id:222, field1:value1, other1_id: 333}
{_id: 222, table:user, first:john, other2_id: 444}
{_id: 333, table:other1, field2:value2}
{_id: 444, table:other2, field3:value3}
We want of way of searching for _id:111
And the result be one document with data from linked tables:
{_id:111, user_id:222, field1:value1, other1_id: 333, first:john, other2_id: 444, field2:value2, field3:value3}
Is there a way to do this?
There is flexibility on the structure of how we store or get the data back—any suggestions on how to better structure the data to make this possible?
The first thing to say is that there are no joins in Cloudant. If you're schema relies on lots of joining then you're working against the grain of Cloudant which may mean extra complication for you or performance hits.
There is a way to de-reference other documents' ids in a MapReduce view. Here's how it works:
create a MapReduce view to emit the main document's body and its linked document's ids in the form { _id: 'linkedid'}
query the view with include_docs=true to pull back the document AND the de-referenced ids in one go
In your case, a map function like this:
function(doc) {
if (doc.table === 'main') {
emit(doc._id, doc);
if (doc.user_id) {
emit(doc._id + ':user', { _id: doc.user_id });
}
}
}
would allow you to pull back the main document and its linked user document in one API by hitting the GET /mydatabase/_design/mydesigndoc/_view/myview?startkey="111"&endkey="111z"&include_docs=true endpoint:
{
"total_rows": 2,
"offset": 0,
"rows": [
{
"id": "111",
"key": "111",
"value": {
"_id": "111",
"_rev": "1-5791203eaa68b4bd1ce930565c7b008e",
"table": "main",
"user_id": "222",
"field1": "value1",
"other1_id": "333"
},
"doc": {
"_id": "111",
"_rev": "1-5791203eaa68b4bd1ce930565c7b008e",
"table": "main",
"user_id": "222",
"field1": "value1",
"other1_id": "333"
}
},
{
"id": "111",
"key": "111:user",
"value": {
"_id": "222"
},
"doc": {
"_id": "222",
"_rev": "1-6a277581235ca01b11dfc0367e1fc8ca",
"table": "user",
"first": "john",
"other2_id": "444"
}
}
]
}
Notice how we get two rows back, the first is the main document body, the second the linked user.
I am trying to identify documents where both key-value pairs within an array match using the aggregate pipeline. Specifically, if I want to find documents where one array contains user_attribute.Name = Quests_In_Progress and user_attribute.Value =3. Below is an example of such a document that I'm trying to match.
If I use
db.myCollection.aggregate({
$match: {
"user_attribute.Name": "Quests_In_Progress",
"user_attribute.Value": "3"
}
})
It will match every document that contains Quests_In_Progress for user_attribute.Name in one element of the array and contains "3" for user_attribute.Value, regardless of whether they exist in the same element of the array or not.
i.e.
db.myCollection.aggregate({
$match: {
"user_attribute.Name": "Quests_In_Progress",
"user_attribute.Value": "0"
}
})
will match the same document simply because one element of the array has a key:Value pair of Value:0 and another element of the array contains a key:value pair of Quests_In_Progress.
What I want to do is identify documents where both of those conditions are met within one element of the array.
I tried to do this with $elemMatch, but I couldn't get it to work. Plus the aggregate documentation doesn't indicate that $elemMatch works, so maybe that's why I couldn't get it to work.
Lastly, I need to use the aggregate pipeline, because there are a bunch of other things I have to do after finding these documents- specifically unwinding them.
{
"_id": ObjectId("5555bb32de938ce667f78ce00"),
"user_attribute": [{
"Value": "Facebook",
"Name": "Social_Connection"
}, {
"Name": "Total_Fireteam_Missions_Initiated",
"Value": "0"
}, {
"Name": "Quests_Completed",
"Value": "3"
}, {
"Name": "Item_Slots_Owned",
"Value": "36"
}, {
"Name": "Quests_In_Progress",
"Value": "3"
}, {
"Name": "Player_Progression",
"Value": "0"
}, {
"Value": "1",
"Name": "Characters_Owned"
}, {
"Name": "Quests_Started",
"Value": "6"
}, {
"Name": "Total_Friends",
"Value": "0"
}, {
"Name": "Device_Type",
"Value": "Phone"
}]
}
Try using $elemMatch
db.myCollection.aggregate([{$match: {"user_attribute": {$elemMatch: {"Name":"Quests_In_Progress", "Value":"0"}}}}, { $out, "temp"}])
That query will find anyone who has element of their array "Quests_In_Progress" with a value of 0 and put it into the collection temp