I am trying to convert one of our field which is in String to Integer. I tried all methods to convert but all methods are failed.
I tried in Kibana using painless, Logstash using mutate filter and Elasticsearch using reindex API.
This is our logs:
Sep 13 10:37:01 SYSTROMEGGN APP_TRAFFIC:
SerialNum=H000D-8D31U-2000P-H0H5Q-E028T GenTime="2019-09-13 10:37:01"
StartTime="2019-09-13 10:36:00" EndTime="2019-09-13 10:37:00"
Category="search-engine" AppName="truecaller" Traffic=31104
All field types are by default string but I want "Traffic" field in integer.
This is my logstash configuration pipeline:
input {
udp {
port => 5044
type => "syslog"
}
}
filter{
if [type] == "syslog" {
grok {
match => { "message" => "% .
{SYSLOGTIMESTAMP:syslog_timestamp} %{WORD:syslog_type}%
{DATA:syslog_program}:%{GREEDYDATA:syslog_message}"
}
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
kv {
source => "syslog_message"
value_split => "="
}
}
}
output {
elasticsearch {
hosts => ["http://192.168.0.62:9200"]
index => "syslog"
document_type => "system_logs"
user=>"elastic"
password=>"elastic"
}
stdout { codec => rubydebug }
}
I expect the output is that my "Traffic" field converted in Integer type but the actual output is "Traffic" field in String type.
You could add a mutate section inside filter in your logstash config. This should work, please share your config with mutate section to see why it is not working.
filter{
-----
mutate {
convert => { "your_field_name" => "integer" }
}
}
Related
this.crudService.get('user.php?mode=test')
.subscribe((data:any) => {
{ for (var key in data) { this[key] = data[key]; } };
}
);
This use to work on angular 7 now on angular 13 i get this error (look image)
In template i was using the values for example in json string was and array and i had users, in template was {{users}} , {{posts}} etc.. now the this[key] give error , please help me out its very important can't find solution
i'll show an example code, and then applied to your code:
Example
// creating global variables to receive the values
users: any = null;
posts: any = null;
// simulating the data you will receive
data: any[] = [
{users: ['user1', 'user2', 'user3']},
{posts: ['post1', 'post2', 'post3']}
];
getCrudService() {
// access each object of the array
this.data.forEach(obj => {
// getting keys name and doing something with it
Object.keys(obj).forEach(key => {
// accessing global variable and setting array value by key name
this[String(key)] = obj[String(key)]
})
})
}
Apllied to your code
this.crudService.get('user.php?mode=test').subscribe((data:any) => {
data.forEach(obj => {
Object.keys(obj).forEach(key => {
this[String(key)] = obj[String(key)]
});
});
});
I hope it helped you, if you need help, just reply me.
I have a CSV file (local), converted it to a string, part of the string is like:
44,"3845657"
51,"3847489"
1,"3888510"
79,"3840471"
57,"3864492"
After I receive input number (first value), I want to match it to the second value (string).
so if input is 51, I want to be able to return 3847489.
No headers in the csv.
CSV to string:
fetchData() {
fetch('../../../assets/static/mapping.csv')
.then(response => response.text())
.then(data => {
// Do something with your data
console.log(data);
this.mappingCSV = data;
});
}
outputs:
44,"3845657"
51,"3847489"
1,"3888510"
79,"3840471"
57,"3864492"
Other ways to convert a csv to an array of objects is also welcome, not married with my csv to string method.
I'm using HTTPClient in this example which is a built-in service class available in Angular. Here's how to use HTTPClient of Angular for you to read and know its benefits.
In my .ts file, I split the text-converted csv first for any newline. Then I added a loop in which I split the text by comma and pushed the necessary details to the new csvArray.
export class SampleComponent {
public csvArr: CsvArray[] = [];
constructor(private http: HttpClient) {
this.http.get('assets/csv.csv', {
responseType: 'text'
}).subscribe(
(data) => {
const csvToRowArray = data.split('\n');
console.log(csvToRowArray);
for (let index = 0; index < csvToRowArray.length; index++) {
const row = csvToRowArray[index].split(',');
this.csvArr.push(new CsvArray(parseInt(row[0], 10), row[1]));
}
console.log(this.csvArr);
},
(error) => {
console.log(error);
}
);
}
}
export class CsvArray {
id: number;
text: string;
constructor(id: number, text: string) {
this.id = id;
this.text = text;
}
}
I created a stackblitz so that you can check my implementation.
How to filter the columns? Here I need a date filter column, When we click the header of the date, it automatically sorts the column.
Actually, they asked like this "Need to sort by date and stick the 6 newest on this one as well"
Can you please help with this?
Here is the code I used.
You need to use a helper function in order to parse the string to Date object.
Then you can apply the sorting
Check the code snippet.
I store the Date object into another property in order not to break anything. You can of course replace it with the current property and format it accordingly inside your table cell.
function parseStr(str) {
let parts =str.substring(0, 10).split('.');
return new Date(parts[2], parts[1] - 1, parts[0]);
}
let array = [
{
"number":"5421787",
"amount":1391.74,
"duedate":"28.08.2020 00.00.00",
"voucherdate":"14.08.2020 00.00.00",
"invoicelink":"",
"status":"Betalt"
},
{
"number":"5499047",
"amount":499.0,
"duedate":"29.09.2020 00.00.00",
"voucherdate":"15.09.2020 00.00.00",
"invoicelink":"",
"status":"Betalt"
},
{
"number":"5574780",
"amount":499.0,
"duedate":"29.10.2020 00.00.00",
"voucherdate":"15.10.2020 00.00.00",
"invoicelink":"",
"status":"Betalt"
}
]
console.log(array.map(obj => ({ ...obj, voucherdate2: parseStr(obj.voucherdate)})).sort((a,b) => b.voucherdate2.getTime() - a.voucherdate2.getTime()))
So in your code
const invoiceData = useSelector((state) => state.userReducer.invoices)
let sortableInvoiceData = invoiceData.map(obj => ({ ...obj, voucherdate2: parseStr(obj.voucherdate)})).sort((a,b) => b.voucherdate2.getTime() - a.voucherdate2.getTime())
const renderInvoices = sortableInvoiceData.map((invoice, index) => {
I know that firestore doesn't support full text search and it giving us solution to use third party services. However I found a simple solution to simple "full text search" and I think this might help others who doesn't want to use third party services as me for such a simple task.
I'm trying to search for company name which is saved in firestore collection under my companyName which can be in any format for example "My Awesome Company". When adding new company with companyName or updating a value in companyName I'm also saving searchName with it which is the same value as company name but in lower case without spaces
searchName: removeSpace(companyName).toLowerCase()
removeSpace is my simple custom function which remove all spaces from a text
export const removeSpace = (string) => {
return string.replace(/\s/g, '');
}
That turns our company name to myawesomecompany which is saved in searchName
Now I've got a firestore function to search for company which indexing through searchName and returning companyName. Minumum search value is a searched value without last character and maximum search value is a searched value with added "zzzzzzzzzzzzzzzzzzzzzzzz" transformed to lower case. That means if you search for My Aw then min value will be mya and max value will be myawzzzzzzzzzzzzzzzzzzzzzzz
exports.handler = ((data) => {
const searchValue = data.value.replace(/\s/g, '').toLowerCase()
const minName = searchValue.substr(0, searchName.length-1)
const maxName = searchValue + "zzzzzzzzzzzzzzzzzzzzzzzz"
let list = []
const newRef = db.collection("user").where("profile.searchName", ">=", minName).where("profile.searchName", "<=", maxName)
return newRef.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
list.push({ name: doc.data().profile.companyName})
})
return list
})
})
I didn't have time to fully test it but so far it works without any problems. Please let me know if you spot anything wrong with it. Now the question is
Is "z" character the highest value character in firestore or is there any other more decent way to add into the search value maximum amount without adding "zzzzzzzzzzzzz"?
I like your decision to preprocess the text so that it can be queried, but you could provide for a more flexible search by storing lowercase keywords with the users and searching those. In other words, transform:
"My Awesome Company"
to...
{ my: true, awesome: true, company: true }
...and test against that.
When adding/updating the property:
// save keywords on the user
let keywords = {}
companyName.split(' ').forEach(word => keywords[word.toLowerCase()] = true)
When querying:
let searchKeywords = userInputString.split(' ').map(word => word.toLowerCase())
let collection = db.collection("user")
searchKeywords.forEach(keyword => {
collection = collection.where(`keywords.${keyword}` , '==' , true);
});
With a little modification of previous answer I have made another simple text search. I'm saving keyword to an array instead of saving it in object like this
nameIndex: textIndexToArray(companyName)
where textIndexToArray is my custom function
export const textIndexToArray = (str) => {
const string = str.trim().replace(/ +(?= )/g,'')
let arr = []
for (let i = 0; i < string.trim().length; i++) {
arr.push(string.substr(0,i+1).toLowerCase());
}
return arr
}
which transfer a text into array. For example
"My Company"
will return
[m, my, my , my c, my co, my com, my comp, my compa, my compan, my company]
with nameIndex saved in firestore we can simply query the data thorough nameIndex and return companyName
exports.handler = ((data) => {
const searchValue = data.value.toLowerCase()
let list = []
const newRef = db.collection("user").where("nameIndex", "array-contains", searchValue)
return newRef.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
list.push({ name: doc.data().companyName, })
})
return list
})
})
I want to get multiple parameters such as Array of String and integer in spray,for example:
http://localhost:8088/sWrite?name=book1&name=book2&name=book3&number=5&TF=false
I use below code:
path("sWrite") {
get {
parameters('name.as[Array[String]], 'number.as[Int], 'TF.as[Boolean]) {
(name, number, TF) =>
complete {
"ok"
}
}
}
}
But it's lead to this error:
[error] too many arguments for method parameters: (pdm spray.routing.directives.ParamDefMagnet)pdm.Out
How can i get array of string and integer in spray??
Should be used from paramString:
path("sWrite") {
def paramString(param: (String, String)): String = s"""${param._1} = '${param._2}'"""
parameterSeq {
params =>
complete {
val sr = params.toArray
val sw = tRest.seqWrite(Seq(sr(0)._2, sr(1)._2, sr(2)._2), sr(3)._2.toInt, sr(4)._2.toBoolean)
"Sequence Write Successful"
}
}
Here's a custom directive to extract multiple query parameters with the same name into a Seq[String]:
def multiParameter(param: Symbol): Directive1[Seq[String]] = {
extract(_.request.uri.query.collect {
case (key, value) if key == param.name => value
}).flatMap {
case Nil => reject(MissingQueryParamRejection(param.name))
case x => provide(x)
}
}
It doesn't support all the bells and whistles that the parameter family of directives do, like optionality, defaults, and type conversion. But you can maybe do some of that by hand if you need to.
Anyway, using this directive to solve the original problem looks like this:
path("sWrite") {
get {
multiParameter('name) { name => // is a Seq, but you can do name.toArray
parameters('number.as[Int], 'TF.as[Boolean]) { (number, TF) =>
complete { "ok" }
}
}
}
}