I am clearly new to OData querying and could use some guidance. I am hoping to use $filter to filter out results and I can't seem to find a good resource for syntax. I am using Odata to select table A, expand on navigation propertyB and count all of occurrences where B is 1) an empty array, 2) has a count of 1 object or 3) has a count of greater than 1 objects. Here is an example of where I want to expand on propertyB. Any help, constructive criticism is greatly appreciated!
http://{{baseUrl}}/odata/tableA?&$top=50&$count=true&$expand=propertyB
Now, I am trying to use the filter function to filter out only the data that has propertyB as an empty array.
{
"myNumber": "00000056218",
"actionType": null,
"numberXTimes": [],
"propertyB": []
}
Here is an example where I would like to have values returned that show "propertyB" has one object:
http://{{baseUrl}}/odata/tableA?&$top=50&$expand=propertyB&$filter=contains(propertyB/myNumber, (myNumber.Length ==1))&$count=true
{
"myNumber": "00000056218",
"actionType": null,
"numberXTimes": [],
"propertyB": [
{
"myNumber": "00000056218",
"desiredId": 300387799,
}
]
}
Unfortunately, my querys aren't working. I would love your help to figure this out. Thanks in advance!
I don't have an exact solution for you, but I can provide a couple of workarounds.
First, you could use $count on your expand parameter to get the count of elements present in propertyB. This would look something like:
http://{{baseUrl}}/odata/tableA?&$top=50&$count=true&$expand=propertyB($count=true)
the response would be:
{
"myNumber": "00000056218",
"actionType": null,
"numberXTimes": [],
"propertyB#odata.count": 1,
"propertyB": [
{
"myNumber": "00000056218",
"desiredId": 300387799
}
]
}
But with this, you are just getting the count and are not actually filtering the results.
As an alternative, you could use the any() lambda function of OData to get only those entries for which propertyB is not empty, and filter out rest.
http://{{baseUrl}}/odata/tableA?&$top=50&$count=true&$expand=propertyB&$filter=propertyB/any()
I hope these workarounds work for your situation.
Related
I have mongodb data model where I have some array fields that contain embedded objects or arrays. I have some inconsistencies in the field in question because I've tweaked my application logic. Initially, my model looked like this:
Initial Setup of Results collection
"competition" : "competition1",
"stats" : [
{
"stat1" : [],
"stat2" : []
}
]
However, I saw that this wasn't the best setup for my needs. So I changed it to the following:
New Setup of Results collection
"competition" : "competition1",
"stats" : [
{
"stat1" : 3,
"stat2" : 2
}
]
My problem now is that documents that have the initial setup cause an error. So what I want is to find all documents that have the initial setup and convert them to have the new setup.
How can I accomplish this in mongodb?
Here is what I've tried, but I'm stuck...
db.getCollection('results').find({"stats.0": { "$exists": true }})
But what I want is to be able to do something like
db.getCollection('results').find({"stats.0".stat1: { "$type": Array}})
Basically I want to get documents where the value of stats[0].stat1 is of type array and override the entire stats field to be an empty array.
This would fix the errors I'm getting.
$type operator for arrays in older versions works little differently than what you might think than $type in 3.6.
This will work in 3.6
db.getCollection('results').find( { "stats.0.stat1" : { $type: "array" } } )
You can do it couple of ways for lower versions and It depends what you are looking for.
For empty arrays you can just check
{"stats.0.stat1":{$size:0}}
For non empty arrays
{"stats.0.stat1": {$elemMatch:{ "$exists": true }}}
Combine both using $or for finding both empty and non empty array.
For your use case you can use below update
db.getCollection('results').update({"stats.0.stat1":{$size:0}}, {$set:{"stats":[]}})
I am trying to pass an array of integers to ElasticSearch template using the below mustache template.
{{#filter5_terms}}
"terms": {
"{{filter5_name}}": [
"{{#filter5_lt}}",
"{{.}}",
"{{/filter5_lt}}" ]
}
{{/filter5_terms}}
Above works, If I pass a string array (Ex: ["A","B"]. But the same is failing with the int array [1,2] with Nested: NumberFormatException[For input string: ""]; error.
Reference: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-template.html#_passing_an_array_of_strings
Can you please let me know if I am missing anything?
Thanks
Anil
You really shouldn't rely on that, as the format is an inner implementation of Mustache and thus, subject to change. For example, if you try to emulate that using mustache.js, you'll get something like:
"terms: {
"property": 3,4
}
To workaround this problem, you should add square brackets to the templated values. So, your example becomes:
"terms": {
"{{filter5_name}}": [{{filter5_lt}}]
}
And that will get you what you want.
At least, this is true in mustache.js 2.2.1
I did fix this.
We can use the below to replace the integer array into ElasticSearch query.
"terms": {
"{{filter5_name}}": {{filter5_lt}}
}
ElasticSearch documentation has an example to replace string arrays and I tried to use the same for integer arrays and it did not work.
So I had to use the above which is provided in Mustache templating examples.
Thanks
Anil
I recently found jsonschema and I've been loving using it, however recently I've come across something that I want to do that I just haven't been able to figure out.
What I want to do is to validate that an array must contain an element that matches a schema, but I don't want to have validation fail on other elements that would be in the list.
Say that I have an array like the following:
arr = [
{"some object": True},
False,
{"AnotherObj": "a string this time"},
"test"
]
I want to be able to do something like "validate that arr contains an object that has a property 'some object' that is a boolean, and error if it doesn't, but don't care about other elements."
I don't want it to validate the other items in the list. I just want to make sure that the list contains an element that matches the schema at least once. I also do not know the order which the elements will arrive in the array.
I've tried this already with a schema like:
{"type": "array",
"items": {
"type": "object",
"properties": {
"tool": {
# A schema here to validate tool
},
"required": ["tool"]
}
}
The problem is that it requires every item in the array to have the property "tool", and not what I actually want.
Any help anyone can give me with this would be much appreciated! I've been stumped on this for a really long time with no forward progress.
Thanks!
I've gotten an answer to this question:
The schema used is (where ... B ... is the schema to require):
{
"type": "array",
"not": {
"items": {
"not": {... B ...}
}
}
}
It basically works out to be something like "Ensure that not (items don't match B)". I'm not 100% clear on why this works the way it does, but it does so I figured I'd share it for posterity.
Hello everyone and thanks in advance for any ideas, suggestions or answers.
First, the environment: I am using CouchDB (currently developing on 1.0.2) and couchdb-lucene 0.7. Obviously, I am using couchdb-lucene ("c-l" hereafter) to provide full-text searching within couchdb.
Second, let me provide everyone with an example couchdb document:
{
"_id": "5580c781345e4c65b0e75a220232acf5",
"_rev": "2-bf2921c3173163a18dc1797d9a0c8364",
"$type": "resource",
"$versionids": [
"5580c781345e4c65b0e75a220232acf5-0",
"5580c781345e4c65b0e75a220232acf5-1"
],
"$usagerights": [
{
"group-administrators": 31
},
{
"group-users": 3
}
],
"$currentversionid": "5580c781345e4c65b0e75a220232acf5-1",
"$tags": [
"Tag1",
"Tag2"
],
"$created": "/Date(1314973405895-0500)/",
"$creator": "administrator",
"$modified": "/Date(1314973405895-0500)/",
"$modifier": "administrator",
"$checkedoutat": "/Date(1314975155766-0500)/",
"$checkedoutto": "administrator",
"$lastcommit": "/Date(1314973405895-0500)/",
"$lastcommitter": "administrator",
"$title": "Test resource"
}
Third, let me explain what I want to do. I am trying to figure out how to index the '$usagerights' property. I am using the word index very loosely because I really do not care about being able to search it, I simply want to 'store' it so that it is returned with the search results. Anyway, the property is an array of json objects. Now, these json objects that compose the array will always have a single json property.
Based on my understanding of couchdb-lucene, I need to reduce this array to a comma separated string. I would expect something like "group-administrators:31,group-users:3" to be a final output.
Thus, my question is essentially: How can I reduce the $usagerights json array above to a comma separated string of key:value pairs within the couchdb design document as used by couchdb-lucene?
A previous question I posted regarding indexing of tagging in a similar situation, provided for reference: How-to index arrays (tags) in CouchDB using couchdb-lucene
Finally, if you need any additional details, please just post a comment and I will provide it.
Maybe I am missing something, but the only difference I see from your previous question, is that you should iterate on the objects. Then the code should be:
function(doc) {
var result = new Document(), usage, right;
for(var i in doc.$usagerights) {
usage = doc.$usagerights[i];
for(right in usage) {
result.add(right + ":" + usage[right]);
}
}
return result;
}
There's no requirement to convert to a comma-separated list of values (I'd be intrigued to know where you picked up that idea).
If you simply want the $usagerights item returned with your results, do this;
ret.add(JSON.stringify(doc.$usagerights),
{"index":"no", "store":"yes", "field":"usagerights"});
Lucene stores strings, not JSON, so you'll need to JSON.parse the string on query.
what seemed a simple task, came to be a challenge for me.
I have the following mongodb structure:
{
(...)
"services": {
"TCP80": {
"data": [{
"status": 1,
"delay": 3.87,
"ts": 1308056460
},{
"status": 1,
"delay": 2.83,
"ts": 1308058080
},{
"status": 1,
"delay": 5.77,
"ts": 1308060720
}]
}
}}
Now, the following query returns whole document:
{ 'services.TCP80.data.ts':{$gt:1308067020} }
I wonder - is it possible for me to receive only those "data" array entries matching $gt criteria (kind of shrinked doc)?
I was considering MapReduce, but could not locate even a single example on how to pass external arguments (timestamp) to Map() function. (This feature was added in 1.1.4 https://jira.mongodb.org/browse/SERVER-401)
Also, there's always an alternative to write storedJs function, but since we speak of large quantities of data, db-locks can't be tolerated here.
Most likely I'll have to redesign the structure to something 1-level deep, like:
{
status:1,delay:3.87,ts:138056460,service:TCP80
},{
status:1,delay:2.83,ts:1308058080,service:TCP80
},{
status:1,delay:5.77,ts:1308060720,service:TCP80
}
but DB will grow dramatically, since "service" is only one of many options which will append each document.
please advice!
thanks in advance
In version 2.1 with the aggregation framework you are now able to do this:
1: db.test.aggregate(
2: {$match : {}},
3: {$unwind: "$services.TCP80.data"},
4: {$match: {"services.TCP80.data.ts": {$gte: 1308060720}}}
5: );
You can use a custom criteria in line 2 to filter the parent documents. If you don't want to filter them, just leave line 2 out.
This is not currently supported. By default you will always receive the whole document/array unless you use field restrictions or the $slice operator. Currently these tools do not allow filtering the array elements based on the search criteria.
You should watch this request for a way to do this: https://jira.mongodb.org/browse/SERVER-828
I'm attempting to do something similar. I tried your suggestion of using the GROUP function, but I couldn't keep the embedded documents separate or was doing something incorrectly.
I needed to pull/get a subset of embedded documents by ID. Here's how I did it using Map/Reduce:
db.parent.mapReduce(
function(parent_id, child_ids){
if(this._id == parent_id)
emit(this._id, {children: this.children, ids: child_ids})
},
function(key, values){
var toReturn = [];
values[0].children.forEach(function(child){
if(values[0].ids.indexOf(product._id.toString()) != -1)
toReturn.push(child);
});
return {children: toReturn};
},
{
mapparams: [
"4d93b112c68c993eae000001", //example parent id
["4d97963ec68c99528d000007", "4debbfd5c68c991bba000014"] //example embedded children ids
]
}
).find()
I've abstracted my collection name to 'parent' and it's embedded documents to 'children'. I pass in two parameters: The parent document ID and an array of the embedded document IDs that I want to retrieve from the parent. Those parameters are passed in as the third parameter to the mapReduce function.
In the map function I find the parent document in the collection (which I'm pretty sure uses the _id index) and emit its id and children to the reduce function.
In the reduce function, I take the passed in document and loop through each of the children, collecting the ones with the desired ID. Looping through all the children is not ideal, but I don't know of another way to find by ID on an embedded document.
I also assume in the reduce function that there is only one document emitted since I'm searching by ID. If you expect more than one parent_id to match, than you will have to loop through the values array in the reduce function.
I hope this helps someone out there, as I googled everywhere with no results. Hopefully we'll see a built in feature soon from MongoDB, but until then I have to use this.
Fadi, as for "keeping embedded documents separate" - group should handle this with no issues
function getServiceData(collection, criteria) {
var res=db[collection].group({
cond: criteria,
initial: {vals:[],globalVar:0},
reduce: function(doc, out) {
if (out.globalVar%2==0)
out.vals.push({doc.whatever.kind.and.depth);
out.globalVar++;
},
finalize: function(out) {
if (vals.length==0)
out.vals='sorry, no data';
return out.vals;
}
});
return res[0];
};