Elasticsearch groovy script to check parameter inclusion in array - arrays

I'm using elasticsearch (v2.0.0) for search in Rails and want to add to our custom script for scoring, but I'm either messing up the syntax or just missing something else entirely. It all works without the check in the script for the array, so that's the only part that's not working.
So for the index, recipe_user_ids is an array of integers:
indexes :recipe_user_ids, type: 'integer'
Then in the search query I specify the parameter for the script file and which script file:
functions: [{
script_score: {
params: { current_user_id: user.id },
script_file: 'ownership_script'
}
}]
And the ownership-script.groovy file:
if (current_user_id == doc['user_id'].value) { owner_modifier = 1.0 } else { owner_modifier = 0.0 }
if (doc['recipe_user_ids'].values.contains(current_user_id)) { recipe_user_modifier = 50.0 } else { recipe_user_modifier = 0.0 }
(_score + (doc['score_for_sort'].value + owner_modifier + recipe_user_modifier)*5)/_score
I'm not getting any errors, but the results don't seem to match what I'd expect when the recipe_user_ids array does contain current_user_id, so everything is falling into the else statement. Is it a type issue, syntax? Any tips greatly appreciated.

This seems to occur due to mismatch in type caused by autoboxing.
The doc['field_name].values for field mapping short, integer, long types seems to be returning a collection always of type 'Long' and the argument to contains is autoboxed to Integercausing contains to fail.
You could probably explictly cast current_user_id to the type of Long:
Example:
doc['recipe_user_ids'].values.contains(new Long(current_user_id))
Or better to use the 'find' method
doc['recipe_user_ids'].values.find {it == current_user_id}

Related

Typescript: Member of union type with incompatible signature when using find on array of objects

I want to check if a value exists in an array of objects.
My array looks something like this:
[
0: {
id: 'unique_obj_id',
item: {
id: 'unique_item_id',
...
},
...
},
1: {...}
]
The objects in the array can be one of many interface types (depending on my api call, here: resource strings represent these interfaces, which are defined somewhere else). But one array will always consist of the same interface types for a given data request.
I'm trying to write a reusable function to check whether given_id exists for any object in the array for obj.item.id.
So far I've managed to write the correct logic but typescript throws some exceptions that I can't seem to figure out. shopApprovalData is a collection of interfaces each following the above object structure accessible by the indices of resource.
export type ApprovalResource = 'str1' | 'str2' | 'str3' | 'str4' | 'str5';
export const checkApprovalItem = (given_id: string, resource: ApprovalResource) => {
const shopApprovalData = useShopApprovals();
if (shopApprovalData && shopApprovalData[resource]) {
const resourceApprovalData = shopApprovalData[resource];
if (resourceApprovalData.find(e => e.item.id === id)) {
return true;
}
}
return false;
}
Typescript shows me that shopApprovalData and itemApprovalData is possibly undefined and that the expression find() is not callable since signatures of members of the union type are not compatible with each other. (Apparently, some removes the error, why?)
What approach should I choose instead to check whether the given_id exists in any object of the array?
Based on your usecase, i create this code sandbox: https://codesandbox.io/s/stackoverflow-answer-ke9drk?file=/src/index.ts
explanation:
Type wont compile, so we need something to mark what the interface of the object.
Also i may confused by your code shopApprovalData[resource] what do you want to achieve here? for example, if resource is str1 shopApprovalData[resource] will always return undefined because no index str1 in Array
I hope it help,
Best Regards

Summing the values of multiple Vectors in the session

In my gatling scenario, I need to check the session for a few entries that will be Vectors of numbers. I can get the Vectors if present, but when I attempt to add them using .sum I get a ClassCastException stating that java.lang.String can't be cast to java.lang.Integer
I've debugged by printing out the value retrieved from the session (Vector(100,200,300)), and even confirmed that the individual elements are Ints. However when I try to add any of them, either with something like values.sum or values(0)+values(1) I get the class cast exception
I'm setting values in the session with checks like
.check(jsonPath("$..payments..paymentAmount").findAll.optional.saveAs("payments"))
.check(jsonPath("$..receipts..receiptAmount").findAll.optional.saveAs("receipts"))
in my app these will always result in things like Vector(100, 200, 300) if the path was there
then later I want to sum all the values in these lists so I have the action
.exec(session => {
def addAmounts(listNames: Array[String]): Int = {
listNames.foldLeft(0)((acc, listName) => {
session(listName).validate[Seq[Int]] match {
case Success(ints) => ints.sum + acc
case Failure(error) => acc
}})
}
val transactionsTotal = addAmounts(Array("payments", "receipts"))
session.set("total",transactionsTotal)
}
As mentioned, this fails on the listName.sum + acc statement - since they're both Ints I'd expect there'd be no need to cast from a string
The Failure case where nothing was stored from the check works fine
I think this is a scala type inference issue - I got it working by manually casting to Int before doing addition

Using .Filter When the Filter Criteria are in Pre-existing String (TypeScript/Node.js)

I'm trying to process an array of JSON objects that have various common attributes, filtering each array entry on one or more of those attributes.
Normally, I'd just do it something like this:
let filteredResultsArray = originalArray.filter((obj) => {
return obj.attribute1 <= 3 && obj.attribute2 > 0 && obj.attribute3 === 10;
});
My problem is that the filter parameters (the part after "return" in the code above) are highly variable (and unpredictable) from run to run, so I can't hard-code them in the filter. I compute them on the fly and store the whole thing in a string in my code. For example, on one run it might be:
myAttributeString = "obj.attribute1 <= 3 && obj.attribute2 > 0 && obj.attribute3 === 10";
I've tried doing this:
let filteredResultsArray = originalArray.filter((obj) => {
return myAttributeString;
});
That's failing to filter anything. Apparently .filter() is not properly interpreting what I've stored in myAttributeString as filter criteria.
I have a sneaking suspicion that eval(myAttributeString) might be one way to pull this off, but unfortunately I'm working on a team where we've got tslint set to disallow the use of eval(), so that's not an option.
Anybody have an idea how I can get this to work?
When you "compute them on the fly", instead of creating a string, create a callback function that you can then pass to filter. For example, instead of
const myAttributeString = "obj.attribute1 <= 3 && obj.attribute2 > 0 && obj.attribute3 === 10";
do
const filterCallback = obj => obj.attribute1 <= 3 && obj.attribute2 > 0 && obj.attribute3 === 10
Then, later, when the appropriate time comes to .filter, simply pass that as the callback:
const filteredResultsArray = originalArray.filter(filterCallback);
If you can't pass functions around, another option would be to build an array of conditions, for example
[
{
prop: "attribute1",
constraint: "<=",
value: 3
},
{
prop: "attribute2",
constraint: ">",
value: 0
},
// ...
]
and then turn the object into the filter function needed.
****************************************UPDATE******************************
As I suspected, eval() did work, but since I can't use it in my delivered code, and thanks to CertainPerformance's suggestion (which put my thinking on the right track) as well as the Node.js documentation site (via a lucky Google search), I was able to find a workaround using the vm module:
import * as vm from "vm";
let filteredResultsArray = originalArray.filter(
vm.runInThisContext("(obj) => {
return " + myAttributeString + ";}"));
Case closed.

Convert NUMERIC to VARCHAR using sequelize model

I have this NUMERIC(20) field on mssql and I'm trying do read it as a data type string to get the full number because int limits on javascript
code: {
type: DataTypes.STRING
}
But the returned value is a truncated INT
[ {code: 4216113112911594000 } ]
No matter what kind of data type choose it'll return as truncated int
The original values is 4216113112911594192. This is a java UUID
How can a convert this value to using model.findAll()
This is a already created table for another application and I'm trying to read it using sequelize
Here you go :
code: {
type: Sequelize.INTEGER , // <------ keep is as DB's datatype
get() { // <------ Use getter method to modify the output of query
return this.getDataValue('code').toString();
}
}
I think this might help you
model.findOne({
attributes: [[db.sequelize.literal('cast(`code` as varchar)'), 'code_string']] ,
where : { id : YOUR_ID }
}).then(data => {
console.log(data); // <------ Check your output here
})
I have not fully understood what you are trying to achieve.
Nor my code is tested.
Idea 1 : convert into float.
Idea 2 : Append any string alphabet to originial int value before sending to javascript.
The original values is 4216113112911594192.
So the string become 'A'+'4216113112911594192'='A4216113112911594192'
Now you can play with 'A4216113112911594192'
There are many javascript library to support big integer.
BigInteger
This worked for postgres (thanks Bessonov & vitaly-t!)
Just add this before you initialize sequelize
import pg from 'pg'
// Parse bigints and bigint arrays
pg.types.setTypeParser(20, BigInt) // Type Id 20 = BIGINT | BIGSERIAL
const parseBigIntArray = pg.types.getTypeParser(1016) // 1016 = Type Id for arrays of BigInt values
pg.types.setTypeParser(1016, (a) => parseBigIntArray(a).map(BigInt))
A little background:
I didn't want Sequelize to parse bigint columns as strings, so I added pg.defaults.parseInt8 = true. That made pg start parsing bigints as regular numbers, but it truncated numbers that were too big. I ended up going with the above solution.

How do I pass a parameter with Expression as value in ADFv2?

In Azure Data Factory v2 (ADFv2) I am having trouble passing a parameter whose value is an expression that needs evaluated at runtime.
For example, I have a set of extracts I want to download daily from the same LinkedService/Connection. I want a pipeline with a Foreach to be able to input a JSON pipeline parameter with a list of configuration for each report type (this I can do). "but" when I have one of those configuration KVPairs with value that is an expression, the expression does not seem to be evaluated.
here is an example of a Foreach parameter set that works for an SFTP LinkedService :
[ { "dirPath" : "/dirPath" ,"fileFilter" : "this_works_fine_20180307*.txt" } ]
here is an example of a Foreach parameter set that does not match the files I need to get.
(assume utcnow('yyyyMMdd') returns 20180307
[ { "dirPath" : "/dirPath" ,"fileFilter" : "this_does_NOT_work_#{utcnow('yyyyMMdd')}*.txt" } ]
This assumes that in the underlying Copy activity I am passing the dataset parameter fileFilter as
#item().fileFilter
...and in the dataset, the value of the fileFilter is an expression with value
#dataset().fileFilter
...I have also tried to wrap the argument completely as:
[ { "dirPath" : "/dirPath" ,"fileFilter" : "#toLower(concat(string('this_does_NOT_work_'),string(utcnow('yyyyMMdd')),string('*.txt') )))" } ]
...If you have suggestions/guidance, please let me know.
Thanks,
J
Try to put the fileFilter parameter directly in pipeline parameter.
Something like this will work:
[ { "dirPath" : "/dirPath" ,"fileFilter" : "this_works_fine_#{formatDateTime(utcnow(), 'yyyy')}#{formatDateTime(utcnow(), 'MM')}#{formatDateTime(utcnow(), 'dd')}*.txt" } ]

Resources