Why does solr-node query gives a wrong document version number? - solr

I am using Solr 7.6. While performing a search query, Solr gives a wrong version field of a document but all the other fields are correct.
In Solr dashboard the query gives the following result:
{
"id":"518fce46-3617-4380-aaf6-8f6d36e08e6a",
"type":"tag",
"count":1,
"_version_":1626999925241806848
}
Whereas, solr-node search function gives:
{
"id": "518fce46-3617-4380-aaf6-8f6d36e08e6a",
"type": "tag",
"count": 1,
"_version_": 1626999925241806800
}

Initial guess is that the solr-node module returns the value as a double (instead of as a string), and the precision of a double isn't good enough to represent the value 1626999925241806848 exactly.
We can confirm this directly in our browser's console:
-> 1626999925241806848
<- 1626999925241806800
i.e. if we input the numeric value 1626999925241806848, it'll be represented by the floating point number that's closest, and that's 1626999925241806800.
solr-node should probably return these values as a string when they exceed the representable value for ints.
Update: solr-node details this at their overview page:
Use json-bigint to handle correctly numbers too large for Javascript Number such as the values of the fields *l and _version. By default json-bigint library is not used because the performance difference compared to the native JSON library is too important with "large" chunk of JSON (https://github.com/lbdremy/solr-node-client/issues/114#issuecomment-54165595), but you want to enable it if you use the Optimistic Concurreny feature available in Solr 4.x, along with RealTime Get and Atomic Updates features because they use the version field. In order to enable it do var client = solr.createClient({ bigint : true}) or directly on the client instance client.options.bigint = true.

Related

Mongo query to filter with an array specific index based on parameters

I have a few documents in a mongoDB that have the following structure
In a API web application that I am developing with spring boot I have to code the following query. I can receive a voltageLevelCode to filter register that will contains this voltage level code in the array (That it is easy) but the problem is that i can also receive a voltageLevelCode and a Type, so in this case I have to filter documents that will contains this voltage level code in the array and also WITHIN this voltage level code filter the ones that contains this type (But remember, the type within the voltage level)
I have been trying to write the query but I dont know how to dynamically set the index to filter the types within this voltage level. Something like:
{"voltageLevel.<TheIndexByTheDefinenVoltageLevelCode>.types" : "X" }
Example:
public List<MyClassRepresenting> findByFilter(String type,String voltageLevelCode);
{$and: [{'voltageLevel.voltageLevelCode' : ?1 },{'voltageLevel.<HowTogetIndexForSelectingVoltageLevelCode>.types' : ?2}]}
In this case depending on the tensionLevel received the type parameter must filter according to types within this tensionLevel
Same happens to me with another query. In SQL the equivalent is the SELECT within another SELECT to select the sub registers but no idea about how to do it in mongo.
When asking a question on stackoverflow, it's always interesting to include what you already tried.
I think what you need is a simple $elemMatch:
db.mycoll.find(
{ voltageLevel: { $elemMatch: { voltageLevelCode: "MT", types: "E" } } }
)

Is there a new way to compare to NaN in an odata filter in Azure Search after the regression in 2016-09-01-Preview?

What is the official way to find documents that match NaN in an Edm.Double field via the rest API on version 2016-09-01-Preview and above?
Specifically we see this json post to the /search endpoint finds the document in version 2016-09-01, but not in version 2016-09-01-Preview and above.
{"filter":"(doublefield eq NaN)","select":"id","top":10,"skip":0,"count":true}
Full reproduction information below
Index definition:
{
"name":"indexnamehere",
"fields":[
{
"name":"doublefield",
"type":"Edm.Double",
"searchable":false,
"filterable":true,
"sortable":true,
"facetable":false,
"key":false,
"retrievable":true
},
{
"name":"id",
"type":"Edm.String",
"searchable":false,
"filterable":true,
"sortable":true,
"facetable":false,
"key":true,
"retrievable":true
}
]
}
Documents:
{
"value":[
{
"id":"60660cd3b9e24d26ae3609926f44aead",
"doublefield":"NaN",
"#search.action":"upload"
},
{
"id":"70660cd3b9e24d26ae3609926f44aead",
"doublefield":"2",
"#search.action":"upload"
}
]
}
Query 1 - finds the document:
{"filter":"(doublefield eq 2)","select":"id","top":10,"skip":0,"count":true}
Query 2 - fails to find the document (in 2016-09-01-Preview and above):
{"filter":"(doublefield eq NaN)","select":"id","top":10,"skip":0,"count":true}
ps. this is our current work around not(doubleField ge -INF) and doubleField ne null
Starting with API version 2016-09-01-Preview, Azure Search now implements the IEEE 754 semantics for comparisons with NaN, per the clarified OData V4.01 specification.
The special value NaN is not equal to anything, even to itself.
Prior to this, NaN comparisons were unreliable. In particular, the inequality operators did not behave correctly, or even consistently between different Search services.
We made this change on an API version boundary to avoid breaking existing applications, but going forward applications using version 2016-09-01-Preview or later cannot detect NaN in double fields. If you need to have some kind of sentinel value for Edm.Double fields, we recommend using null instead since you can check for null with filters.
Allowing NaN to be indexed has turned out to be problematic, and we're considering disallowing it altogether in a future API version.

Update AngularJS data from separate source

I'm working on my first Angular project and I've been thinking of the best way to word this question for a while now but I'm going to give this a shot.
I'm building an app that uses the Veralite (MiOS) to return device data in json format. The first request returns all of the devices on the system (example below).
"devices":[
{
"name":"Bedroom Light",
"altid":"4",
"id":6,
"category":2,
"subcategory":0,
"room":0,
"parent":1,
"status":"0",
"level":"0",
"state":-1,
"comment":""
},
{
"name":"Office Light",
"altid":"6",
"id":18,
"category":2,
"subcategory":0,
"room":0,
"parent":1,
"level":"0",
"status":"0",
"state":-1,
"comment":""
}
Once all of the devices are returned, my script begins long polling the vera engine. Once a change to a device is made the results of the long poll are returned, but the results only include the devices that were changed (example below).
"devices":[
{
"altid":"6",
"id":"18",
"subcategory":"0",
"room":"0",
"parent":"1",
"level":"20",
"status":"1",
"state":"4",
"comment":"Office Light: Transmit was ok"
}
What I am trying to wrap my head around, is the proper way to update the existing devices array with the newly updated data. Would I need to convert them to arrays, then loop through each array and try to match them by keys?
Hopefully I asked this as clearly as possible.
EDIT: Just to update this a bit for anyone that stumbles across this, specifically people interested in developing for the Veralite. The ID returned from the original request will be returned as an integer, but when long polling the engine, the ID will be returned as a string. So even though the selected answer is correct, you'll need to either parse the updated device ID as an integer (parseInt), or only use a == instead of a === when filtering the devices.
You can loop through the object fields just like you can loop through an array with Object.keys.
Conceptually something like:
Step 1) Find the updated product, by the field id:
var previousVersionDevice = $scope.devices.filter(function(item) {
return item.id === updateDevice.id; // Keep only the one existing device that matches id
})[0];
Step 2) Loop through the keys in the product and overwrite the previous values with the ones received.
Object.keys(updatedDevice).forEach(function(key) {
previousVersionDevice[key] = updatedDevice[key]; // Overwrite/add all from updated version
});
yes, you can loop through and find the matching device via some sort of unique key (assuming id is unique and won't change, you can use that).

JMeter: How to count JSON objects in an Array using jsonpath

In JMeter I want to check the number of objects in a JSON array, which I receive from the server.
For example, on a certain request I expect an array with 5 objects.
[{...},{...},{...},{...},{...}]
After reading this: count members with jsonpath?, I tried using the following JSON Path Assertion:
JSON Path: $
Expected value: hasSize(5)
Validate against expected value = checked
However, this doesn't seem to work properly. When I actually do receive 5 objects in the array, the response assertion says it doesn't match.
What am I doing wrong?
Or how else can I do this?
Although JSONPath Extractor doesn't provide hasSize function it still can be done.
Given the example JSON from the answer by PMD UBIK-INGENIERIE, you can get matches number on book array in at least 2 ways:
1. Easiest (but fragile) way - using Regular Expression Extractor.
As you can see, there are 4 entries for category like:
{ "category": "reference",
{ \"category\": \"fiction\"
...
If you add a Regular Expression Extractor configured as follows:
It'll capture all the category entries and return matches number as below:
So you will be able to use this ${matches_matchNr} variable wherever required.
This approach is straightforward and easy to implement but it's very vulnerable to any changes in the response format. If you expect that JSON data may change in the foreseeable future continue with the next option.
2. Harder (but more stable) way - calling JsonPath methods from Beanshell PostProcessor
JMeter has a Beanshell scripting extension mechanism which has access to all variables/properties in scope as well as to the underlying JMeter and 3rd-party dependencies APIs. In this case you can call JsonPath library (which is under the hood of JsonPath Extractor) directly from Beanshell PostProcessor.
import com.jayway.jsonpath.Criteria;
import com.jayway.jsonpath.Filter;
import com.jayway.jsonpath.JsonPath;
Object json = new String(data);
List categories = new ArrayList();
categories.add("fiction");
categories.add("reference");
Filter filter = Filter.filter(Criteria.where("category").in(categories));
List books = JsonPath.read(json, "$.store.book[?]", new Filter[] {filter});
vars.put("JSON_ARRAY_SIZE", String.valueOf(books.size()));
The code above evaluates JSONPath expression of $.store.book[?] against parent sampler response, counts matches number and stores it into ${JSON_ARRAY_SIZE} JMeter Variable
which can later be reused in an if clause or an assertion.
References:
JMeter – Working with JSON – Extract JSON response
JMeter's User Manual Regular Expressions entry
JSON Path Documentation and Examples
How to use BeanShell: JMeter's favorite built-in component
This is not possible with the plugin you are using (JMeter-plugins).
But it can be done with JSON Extractor since JMeter 3.0, this plugin has been donated by UbikLoadPack (http://jmeter.apache.org/changes_history.html)
Example:
Say you have this JSON that contains an array of books:
{ "store": {"book": [
{ "category": "reference","author": "Nigel Rees","title": "Sayings of the Century","price": 8.95},
{ "category": "fiction","author": "Evelyn Waugh","title": "Sword of Honour","price": 12.99},
{ "category": "fiction","author": "Herman Melville","title": "Moby Dick","isbn": "0-553-21311-3","price": 8.99},
{ "category": "fiction","author": "J. R. R. Tolkien","title": "The Lord of the Rings","isbn": "0-395-19395-8","price": 22.99}
],
"bicycle": {"color": "red","price": 19.95}} }
To have this count:
1/ Add JSON Extractor:
The count will be then available bookTitle_matchNr which you can access through:
${bookTitle_matchNr}
Running this Test Plan would display this:
As you can see, Debug Sampler-${bookTitle_matchNr} shows Debug Sampler-4

Parsing Solr Results - javabin format

I am trying to integrate solr with java using solrj. The result retrieved are of the format
{
numFound=3,
start=0,
docs=[
SolrDocument{
id=IW-02,
name=iPod&iPodMiniUSB2.0Cable,
manu=Belkin,
manu_id_s=belkin,
cat=[
electronics,
connector
],
features=[
carpoweradapterforiPod,
white
],
weight=2.0,
price=11.5,
price_c=11.50,
USD,
popularity=1,
inStock=false,
store=37.7752,
-122.4232,
manufacturedate_dt=TueFeb1418: 55: 59EST2006,
_version_=1452625905160552448
}
Now this is the javabin format. How do I extract results from this? Have heard that solrj does convert the results to objects by itself. But cant figure out how.
Thanks for the help in advance.
Let solrReply be the response object. The you can access different parts of the result using appropriate params. Say you want docs, you can do:
docs = solrReply['docs']
if you want the first result you could do:
first = solrReply['docs'][0]
Within a result you can access each field in the same way.

Resources