I have an index with the name 'dev-steps',
mapping for this index:
{
"dev-steps": {
"mappings": {
"steps": {
"properties": {
"cId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"data": {
"properties": {
"ui034": {
"type": "long"
},
"ksms5": {
"type": "long"
},
"ui0tg": {
"type": "long"
},
"vcw5d": {
"type": "long"
}
}
},
"uId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
I see a problem, any time when I add new value to the field 'data', that add a new property to the mapping; how I can create a dynamic mapping?
You can achieve this by adding a dynamic template to your index mapping (Elasticsearch Reference: Dynamic templates).
Thinking I found an answer to my question
Too many fields bad for elasticsearch index?
and sure I found in elasticsearch docs
https://www.elastic.co/blog/found-beginner-troubleshooting#keyvalue-woes
Related
I believe this issue has to do with the different syntax from elasticsearch 6.x -> 7.x
This is my first time using elasticsearch and havent been able to understand where the issue is.
How would I fix my mapping?
{
"settings": {
"analysis": {
"analyzer": {
"ssdeep_analyzer": {
"tokenizer": "ssdeep_tokenizer"
}
},
"tokenizer": {
"ssdeep_tokenizer": {
"type": "ngram",
"min_gram": 7,
"max_gram": 7,
"token_chars": [
"letter",
"digit",
"symbol"
]
}
}
}
},
"mappings": {
"_default_": {
"_all": {
"enabled": false
},
"dynamic": "strict",
"properties": {
"chunksize": {
"type": "integer"
},
"chunk": {
"analyzer": "ssdeep_analyzer",
"type": "text"
},
"double_chunk": {
"analyzer": "ssdeep_analyzer",
"type": "text"
},
"ssdeep": {
"type": "keyword"
},
"sha256": {
"type": "keyword"
}
}
},
"record": {}
}
}
The error that is being raised is
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Root mapping definition has unsupported parameters: [_default_ : {dynamic=strict, _all={enabled=false}, properties={sha256={type=keyword}, chunksize={type=integer}, chunk={analyzer=ssdeep_analyzer, type=text}, double_chunk={analyzer=ssdeep_analyzer, type=text}, ssdeep={type=keyword}}}] [record : {}]')
A bunch of issues here:
The _all field is deprecated (and has been since v6).
"record": {} is not a valid parameter.
_default_ is deprecated too.
Here's a working mapping:
{
"settings": {
"analysis": {
"analyzer": {
"ssdeep_analyzer": {
"tokenizer": "ssdeep_tokenizer"
}
},
"tokenizer": {
"ssdeep_tokenizer": {
"type": "ngram",
"min_gram": 7,
"max_gram": 7,
"token_chars": [
"letter",
"digit",
"symbol"
]
}
}
}
},
"mappings": {
"dynamic": "strict",
"properties": {
"chunksize": {
"type": "integer"
},
"chunk": {
"analyzer": "ssdeep_analyzer",
"type": "text"
},
"double_chunk": {
"analyzer": "ssdeep_analyzer",
"type": "text"
},
"ssdeep": {
"type": "keyword"
},
"sha256": {
"type": "keyword"
}
}
}
}
I'm new working with elastic search and I'm trying to make a simple query work. I'm using elasticseach 2.3. I can't use a newer version I'm limited by another technology I'm using.
Basically I have News stored in the database with a title, a source and a publication date.
The query I'm trying to make should search for all the news that contain a certain keyword, come from some sources A or B and have a publication date in the range given.
So far I have this:
{
"query":{
"bool":{
"must":[
{
"bool":{
"should":[
{
"match":{
"source":"SOURCE_A"
}
},
{
"match":{
"source":"SOURCE_B"
}
},
{
"match":{
"title": "keyword"
}
}
]
}
}
],
"filter":{
"range":{
"publication_date":{
"gte":"DATE_FROM",
"lte":"DATE_TO"
}
}
}
}
}
}
The problem is that if a given source starts exactly the same as another source (for example: "SOURCE" and "SOURCE ABC") they are both included in the result. I would like to match exactly the same source.
Can anyone point me in the right direction?
Thanks!
The index is being created by Django Haystack but given its limitations I need to query the database myself. The index mapping is the following:
{
"myindex": {
"mappings": {
"modelresult": {
"properties": {
"django_ct": {
"type": "string",
"index": "not_analyzed",
"include_in_all": false
},
"django_id": {
"type": "string",
"index": "not_analyzed",
"include_in_all": false
},
"id": {
"type": "string"
},
"publication_date": {
"type": "date",
"format": "strict_date_optional_time||epoch_millis"
},
"source": {
"type": "string",
"analyzer": "ascii_analyser"
},
"summary": {
"type": "string",
"analyzer": "ascii_analyser"
},
"text": {
"type": "string",
"analyzer": "ascii_analyser"
},
"title": {
"type": "string",
"analyzer": "ascii_analyser"
},
"url": {
"type": "string",
"analyzer": "ascii_analyser"
}
}
}
}
}
}
I have data with one parameter which is an array. I know that objects in array are not well supported in Kibana, however I would like to know if there is a way to filter that array with only one value for the key. I mean :
This is a json for exemple :
{
"_index": "index",
"_type": "data",
"_id": "8",
"_version": 2,
"_score": 1,
"_source": {
"envelope": {
"version": "0.0.1",
"submitter": "VF12RBU1D53087510",
"MetaData": {
"SpecificMetaData": [
{
"key": "key1",
"value": "94"
},
{
"key": "key2",
"value": "0"
}
]
}
}
}
}
And I would like to only have the data which contains key1 in my SpecificMetaData array in order to plot them. For now, when I plot SpecificMetaData.value it takes all the values of the array (value of key1 and key2) and doesn't propose SpecificMetaData.value1 and SpecificMetaData.value2.
If you need more information, tell me. Thank you.
you may need to map your data to mappings so as SpecificMetaData should act as nested_type and inner_hits of nested filter can supply you with objects which have key1.
PUT envelope_index
{
"mappings": {
"document_type": {
"properties": {
"envelope": {
"type": "object",
"properties": {
"version": {
"type": "text"
},
"submitter": {
"type": "text"
},
"MetaData": {
"type": "object",
"properties": {
"SpecificMetaData": {
"type": "nested"
}
}
}
}
}
}
}
}
}
POST envelope_index/document_type
{
"envelope": {
"version": "0.0.1",
"submitter": "VF12RBU1D53087510",
"MetaData": {
"SpecificMetaData": [{
"key": "key1",
"value": "94"
},
{
"key": "key2",
"value": "0"
}
]
}
}
}
POST envelope_index/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"inner_hits": {},
"path": "envelope.MetaData.SpecificMetaData",
"query": {
"bool": {
"must": [
{
"term": {
"envelope.MetaData.SpecificMetaData.key": {
"value": "key1"
}
}
}
]
}
}
}
}
]
}
}
}
I am currently using angular-schema-form-dynamic-select and my requirement is to select states based on a country selected. I'm storing data in the db like this country -> state -> city. Can anyone Help me on this?
This is my form:
[
{
"key": "country",
"type": "strapselect",
"placeholder":"country",
"options": {
"httpGet": {
"url": "/countries"
},
"map": { "valueProperty": "readonlyProperties.id", "nameProperty":"name" }
}
},
{
"key": "state",
"type": "strapselect",
"placeholder":"state",
"options": {
"httpGet": {
"url": "/states"
},
"map": { "valueProperty": "readonlyProperties.id", "nameProperty":"name" }
}
},
{
"key": "city",
"type": "strapselect",
"placeholder":"city",
"options": {
"httpGet": {
"url": "/cities"
},
"map": { "valueProperty": "readonlyProperties.id", "nameProperty":"name" }
}
}
]
I think a feature like that would be indeed quite handy. Maybe you write something like this in the json string:
{
"type": "object",
"properties": {
"country": {
"type": "string",
"enumCallback": "getTitlesValues()"
}
}
}
And in your controller you would have that callback defined:
...
$scope.getTitlesValues = function () {
return ['India','Australia', 'Germany', 'Sweden']
}
...
I think a feature like that would be indeed quite handy.
Maybe you write something like this in the json string:
{
"type": "object",
"properties": {
"country": {
"type": "string",
"enumCallback": "getTitlesValues()"
}
}
}
And in your controller you would have that callback defined:
...
$scope.getTitlesValues = function () {
return ['India','Australia', 'Germany', 'Sweden']
}
...
as a beginner of NoSQL-Databases I have following problem in MongoDB:
My "scheme" looks like:
Film document
{
"title": "film",
"type": "object",
"properties":{
"id":{
"type": "integer"
},
"title": {
"type": "string"
},
"genre": {
"type": "string"
},
"ratings":{
"type": "array",
"items": [
{
"type": "object",
"properties": {
"userId": {
"type": "integer"
},
"rating": {
"type": "number"
}
}
}]
}
}
}
With this queue, I get the right result, but not only one object.
db.film.aggregate( [
{ $unwind: "$ratings" },
{ $group: {
_id: '$title',
SumRating: { $sum: '$ratings.rating' }
} } , {$sort:{SumRating:-1}}]);
So my problem is to use the $max operator. I tried something but nothing worked fine for me. Does somebody has an idea how I need to use the operator to get only the movie with the highest/max rating?