I would like to know if there is any simple way in logstash to copy all the elements of one field array to another. Put another way, is there a way to copy all the nested fields of a top-level field to another top-level field?
I'm having this problem because I'm dealing with a nonstandard multiline log format which is not necessarily thread-safe. So in order to make sure I get all the data I need and in the right order, I'm currently on planning on throwing every relevant value of each type into an array, and then sorting them out later. The problem I'm having is that logstash doesn't seem to support too much array manipulation. If you think my approach just won't work at all in logstash, any suggestions as to what I could do instead would be greatly appreciated.
Thanks!
As this question almost shows: LogStash: How to make a copy of the #timestamp field while maintaining the same time format? you can use the ruby filter to copy an array.
filter {
ruby {
code => "event['another_top_level_field'] = event['top_level_field']"
}
}
Related
I'm making a scheduling app, and storing all the scheduled things in firebase with arrays. When I try to schedule something with the same string value, it fails and doesn't add it to the array. I don't know if this is something in swift I can edit, or if it's a firebase setting.
If it's something in swift, here's the code updating the array:
doc.updateData([
"Instructor": FieldValue.arrayUnion(["\(scheduleinstructor)"])
])
If it's something in firebase, could someone please explain a way around this or a simple fix I overlooked?
According to the documentation on adding items to an array:
arrayUnion() adds elements to an array but only elements not already present
So the fact that the duplicate entry is not added is by design. If you want to allow that, you'll have to:
Read the document with the array from the databae.
Extract the array from the document into your application code.
Add the item to the array.
Write the entire modified array back to the database.
I've finally started to understand a lot of info regarding FireStore, but I'm wondering if I can get some assistance.
If I had a setup similar to or like this:
races
Android
name: Android
size: medium
stats <---- this is the map
str: 10
sex: 12.... (more values)
How would I parse this? I am looking to make specific TextViews apply values found in the database so that I can simply update the database and my app will populate those values so that hard coding and code updating won't be nearly as troublesome in the future.
I currently use something like this:
val androidRef = db.collection("races").document("Android")
androidRef.get().addOnSuccessListener { document ->
if (document != null) {
oneOfTheTextViews.text = document.getString("str")
} else {
}
The issue is currently I can only seem to access from collection (races) / document (android) / then a single field (I have "str" set as a single field, not part of a map or array)
What would the best practice be to do this? Should I not nest them at all? And if I can reference said nesting/mapping/array, what functions need to be called? (To be clear, I am not asking only whether or not it is possible - the reference guides and documents allude to such - but what property/class/method/etc needs to be called in order to access only one of those values or point to one of those values?).
Second question: Is there a way to get a list of document names? If I have several races, and simply want to make a spinner or recycler view based on document names as part of a collection, can I read that to the app?
What would the best practice be to do this?
If you want to get the value of your str property which is nested within your stats map, please change the following line of code:
oneOfTheTextViews.text = document.getString("str")
to
oneOfTheTextViews.text = document.getString("stats.str")
If your str property is a number and not a String, then instead of the above line of code please use this one:
oneOfTheTextViews.text = document.getLong("stats.str")
Should I not nest them at all?
No, you can nest as many properties as you want within a Map.
Is there a way to get a list of document names?
Yes, simply iterate the collection and get the document ids using getId() function.
I fetch the data from a server and some items have a specific attribute others don't. I need to sort data according to this specific attribute and I am using sortBy package but of course it doesn't work properly because when it tries to sort data and doesn't find the attribute, it is broken.
myItems.sort(sortBy('specificAttr'))
Basically, what I did (think of inside of a loop):
if(!myItems.specificAttr) {myItems.speficificAttr = 0);
I know it doesn't make sense at all, but I don't know what I can do.
Do you have any advice with code examples?
Using lodash's sortBy
Lodash handles this case out of the box. If it can't find the attribute, it gets pushed to the end of the sorted array.
var users = [
{ 'age': 48 },
{ 'user': 'barney', 'age': 36 },
{ 'user': 'fred', 'age': 40 },
{ 'user': 'barney', 'age': 34 }
];
_.sortBy(users, [(o) => { return o.user; }]);
// => objects for barney, barney, fred, and no-name in that order
_.sortBy(users, ['user']);
// => objects for barney, barney, fred, and no-name in that order
If you're using some other library's sortBy, check the documentation to see if there's an optional second argument that would allow you to customize the attribute getter function.
Rolling your own
If you're implementing your own sorting algorithm, simply do a check like if myItem.specificAttr to check if the attribute exists while you're doing the actual sorting (rather than prior sorting, as you described in your example).
I am trying to append/remove items from an array inside of a Firestore Document but every time the entire array is replaced instead of the new value being appended. I have tried both of the following:
batch.setData(["favorites": [user.uid]], forDocument: bookRef, options: SetOptions.merge())
batch.updateData(["favorites": [user.uid]], forDocument: bookRef)
I know that instead of an array I can use an object/dictionary but that would mean storing additional data that is irrelevant (such as the key), all I need is the ID's stored inside the array. Is this something that is currently possible in Firestore?
Update elements in an array
If your document contains an array field, you can use arrayUnion() and arrayRemove() to add and remove elements. arrayUnion() adds elements to an array but only elements not already present. arrayRemove() removes all instances of each given element.
let washingtonRef = db.collection("cities").document("DC")
// Atomically add a new region to the "regions" array field.
washingtonRef.updateData([
"regions": FieldValue.arrayUnion(["greater_virginia"])
])
// Atomically remove a region from the "regions" array field.
washingtonRef.updateData([
"regions": FieldValue.arrayRemove(["east_coast"])
])
See documentation here
Actually, nowadays it is possible. With latest updates db.collection.updateData
method actually appends new item to array instead of replacing it.
Example usage can be found in Firebase documentation.
If you need to do it manually, you can use
FieldValue.arrayUnion([user.uid])
Nope. This isn't possible.
Arrays tend to be problematic in an environment like Cloud Firestore where many clients could theoretically append or remove elements from an array at the same time -- if instructions arrive in a slightly different order, you could end up with out-of-bounds errors, corrupted data, or just a really bad time. So you either need to use a dictionary (where you can specify individual keys) or replace the entire array.
I have two data arrays:
$arrnames=('a','b','c')
$arrtypes=('x','y','z')
and I have a lookup array:
$arrlookup=('y','z')
I need to return the elements in $arrnames where the matching element in $arrtypes is contained in $arrlookup. I've been playing with foreach looping over $arrnames and looking at the same element in $arrtypes by index but it's getting sloppy and I know there has to be a better way.
I've seen some similar questions, but not anything that strikes me as specifically answering this problem. Being a Powershell very newbie I'm still learning how to adapt examples to my needs. Any help is greatly appreciated.
My code (technique used as applied to above example - actual code is using complex arrays derived from an XML file and has a lot of debugging code so copy/paste is impractical)
foreach ($a in $arrnames) {
$t=$arrtypes[$arrnames.IndexOf($a)]
if ($arrlookup.Contains($t)) {
$arrresult+=$a
}
)
$arrresult should contain the members of $arrnames that have a type (from $arrtypes) that is in $arrlookup.
Is there a way to use object methods, filtering and the pipeline to simply extract the elements without a foreach loop
Edit - here's the actual code that creates the actual arrays - $builds is an XML document:
$names=$builds.builds.project.name
$types=$builds.builds.project.type
The lookup table is known:
$FXCopTypes=#('batch','component','web')
The XML file I also have control over, but I don't see any way to simplify it more than implementing the hash table code but with the above arrays.
I think you need to change you input inorder to be able to do anything different here. What you are asking for is $arrnames and $arrtypes to really be a hashtable. That way you can access the values using keys.
As it stands I would do this to create the hashtable. The second loop shows how to return each matching value.
$hash = #{}
for($index = 0; $index -lt $arrtypes.Count; $index++){
$hash.($arrtypes[$index]) = $arrnames[$index]
}
$arrresult = $arrlookup | ForEach-Object{
$hash[$_]
}
This would return
b
c
If you could get your input to create that hash table then it reduces the need to rebuild it. Also if you know the lookup before hand as well you can filter it then and just have the output you want.