I am trying to divide a nested array using RDD in Spark. For example, there is a TextFile that contains four sentences, like this:
"he is good", "she is good", "I am good", "we are good"
I used val arr = sc.textFile("filename").map(_.split(" ")) command and got this:
Array[Array[String]] = Array(Array(he, is, good),
Array(she, is, good),
... )
I want to use each array elements (i.e. Array(he, is, good)) but I don't know how to divide this. How can I divide this?
It is unclear what you mean by 'divided', but typically in functional programming languages, when you want to do anything with each element of a collection (or 'iterable'), you can use the map function. map converts each element based on a function you passed it. For example, in a worksheet you can do this:
val sentences = Array(Array("he", "is", "good"),
Array("she", "is", "very", "good"))
def yodaize(sentence: Array[String]): Array[String] = {
val reversed = sentence.reverse
println("Yoda says, '%s'".format(reversed.mkString(" ")))
reversed
}
yodaize(Array("I", "am", "small"))
val yodaSentences = sentences.map(yodaize)
The function yodaize does two things: it reverses the sentence passed to it and as a side effect prints out the reversed sentence. The worksheet output of the above is:
sentences: Array[Array[String]] = [[Ljava.lang.String;#faffecf
yodaize: yodaize[](val sentence: Array[String]) => Array[String]
Yoda says, 'small am I'
res0: Array[String] = [Ljava.lang.String;#4bf1c779
Yoda says, 'good is he'
Yoda says, 'good very is she'
yodaSentences: Array[Array[String]] = [[Ljava.lang.String;#40a19a85
It's hard to see directly here, but yodaSentences is the original array with each sub-array reversed:
Array(Array("good", "is", "he"),
Array("good", "very", "is", "she"))
With map you can pass in any function. It can directly convert the element or have a side effect. In this manner functional languages can deal with each element without really ever needing to 'divide' them. Note that other functions like flatMap, foldLeft and filter can be used to perform other sorts of permutations on your collection.
You can use map and foreach both for Arrays and RDDs:
yourReference.map(arrayOfStrings => arrayOfStrings.mkString)
or
yourReference.foreach(arrayOfStrings => println(Arrays.toString(arrayOfStrings)))
Related
I know how to iterate through an array and I know how to create hash keys and values. I do not know how to create an array for the value and pass it multiple elements. My desired value for hash below is:
{'abc' => [1, 2, 3] , 'def' => [4,5,6,7]}
How would I achieve this hash, while iterating through array a and b below using each?
a = [1,2,3,4,5,6,7]
c = [1,2,3]
b = ['abc', 'def']
hash = {}
From your guidelines given in the comment:
While iterating through array a, if the element of iteration is included in array c, it is passed to the array value within key 'abc'. Otherwise, it is passed to other array value in key 'def'
You can do this:
hash = {}
hash['abc'] = a.select { |x| c.include?(x) }
hash['def'] = a.reject{ |x| c.include?(x) }
See Enumerable#select and Enumerable#reject. Also can take a look at Enumerable#partition which would be another good choice here, where you want to split an array into two arrays based on some condition:
in_a, not_in_a = a.partition { |x| c.include?(x) }
hash = { 'abc' => in_a, 'def' => not_in_a }
You can also do it with regular each if these fancy enumerable methods are bit too much for you:
hash = { 'abc' => [], 'def' => [] }
a.each do |x|
if c.include?(x)
hash['abc'].push(x)
else
hash['def'].push(x)
end
end
Unfortunately this question turned out not to be as interesting as I was hoping. I was hoping that the problem was this:
Knowing the hash key and a value, how can I make sure the key's value is an array and that the given value is appended to that?
For instance, start with h being {}. I have a key name :k and a value 1. I want h[:k], if it doesn't already exist, to be [1]. But if it does already exist, then it's an array and I want to append 1 to that array; for instance, if h[:k] is already [3,2], now it should be [3,2,1].
I can think of various ways to ensure that, but one possibility is this:
(hash[key] ||= []) << value
To see that this works, let's make it a method:
def add_to_hash_array_value(hash:, key:, value:)
(hash[key] ||= []) << value
end
Now I'll just call that a bunch of times:
h = {}
add_to_hash_array_value(hash:h, key:"abc", value:1)
add_to_hash_array_value(hash:h, key:"abc", value:2)
add_to_hash_array_value(hash:h, key:"def", value:4)
add_to_hash_array_value(hash:h, key:"def", value:5)
add_to_hash_array_value(hash:h, key:"abc", value:3)
puts h #=> {"abc"=>[1, 2, 3], "def"=>[4, 5]}
We got the right answer.
This is nice because suppose I have some way of knowing, given a value, what key it should be appended to. I can just repeatedly apply that decision-making process.
However, the trouble with trying to apply that to the original question is that the original question seems not to know exactly how to decide, given a value, what key it should be appended to.
I have been trying to randomize the values in an ordered array (ex:[0,1,2,3]) in Godot. There is supposed to be a shuffle() method for arrays, but it seems to be broken and always returns "null". I have found a workaround that uses a Fisher-Yates shuffle, but the resulting array is considered "unsorted" by the engine, and therefore when I try to use methods such as bsearch() to find a value by it's position, the results are unreliable at best.
My solution was to create a dictionary, comprised of an array containing the random values I have obtained, merged with a second array of equal length with (sorted) numbers (in numerical order) which I can then use as keys to access specific array positions when needed.
Question made simple...
In GDScript, how would you take 2 arrays..
ex: ARRAY1 = [0,1,2,3]
ARRAY2 = [a,b,c,d]
..and merge them to form a dictionary that looks like this:
MergedDictionary = {0:a, 1:b, 2:c, 3:d}
Any help would be greatly appreciated.
Godot does not support "zip" methodology for merging arrays such as Python does, so I am stuck merging them manually. However... there is little to no documentation about how to do this in GDScript, despite my many hours of searching.
Try this:
var a = [1, 2, 3]
var b = ["a", "b", "c"]
var c = {}
if a.size() == b.size():
var i = 0
for element in a:
c[element] = b[i]
i += 1
print("Dictionary c: ", c)
If you want to add elements to a dictionary, you can assign values to the keys like existing keys.
I was reading over some resources about Swift 5's String Interpolation and was trying it out in a Playground. I successfully combined two separate Strings but I am confused about how to combine an Array of Strings.
Here's what I did ...
String(stringInterpolation: {
let name = String()
var combo = String.StringInterpolation(literalCapacity: 7, interpolationCount: 1)
combo.appendLiteral("Sting One, ")
combo.appendInterpolation(name)
combo.appendLiteral("String Two")
print(combo)
return combo
}())
How would you do something like this with an Array of Strings?
It’s unclear what this is supposed to have to do with interpolation. Perhaps there’s a misunderstanding of what string interpolation is? If the goal is to join an array of strings into a single comma-separated string, just go ahead and join the array of strings into a single string:
let arr = ["Manny", "Moe", "Jack"]
let s = arr.joined(separator: ", ")
s // "Manny, Moe, Jack”
If the point is that the array element type is unknown but is string-representable, map thru String(describing:) as you go:
let arr = [1,2,3]
let s = arr.map{String(describing:$0)}.joined(separator: ", ")
s // "1, 2, 3”
I have two arrays, the first array contains field name, type and id.
arr1 = [
{
"n" => "cat",
"t" => 0,
"id" => "42WTd5"
},
{
"n" => "dog",
"t" => 0,
"id" => "1tM5T0"
}
]
Second array contains field, id and value.
arr2 = [
{
"42WTd5"=>"meow",
"1tM5T0"=>"woof"
}
]
How can I join them by id to produce the following result.
cat: meow
dog: woof
Any help is appreciated.
I think you want your result to be a Hash, in which case this would do the job:
def match_animals_to_sounds(animal_array, sound_array)
sounds = sound_array.first
animal_array.map { |animal| [animal['n'], sounds[animal['id']]] }.to_h
end
>> match_animals_to_sounds(arr1, arr2)
=> {"cat"=>"meow", "dog"=>"woof"}
Your arr2 is unusual in that it is an Array of a single element. I'm just calling #first on it to pull out the Hash inside. If you expect some version of this Array to have more than one element in the future, you'll need to rethink the first line of this method.
The second line is standard Ruby Array manipulation. The first part maps each animal to a new Array of two-element Arrays containing each animal's name and sound. At the end, #to_h converts this array of two-element arrays to a single Hash, which is much more useful than an array of strings. I don't know what you intended in your question, but this is probably what you want.
If you prefer to work with Symbols, you can change the second line of the method to:
animal_array.map { |animal| [animal['n'].to_sym, sounds[animal['id']].to_sym] }.to_h
In which case you will get:
>> match_animals_to_sounds(arr1, arr2)
=> {:cat=>:meow, :dog=>:woof}
This is a way to do it.
sounds = arr2[0]
results = arr1.map do |animal|
"#{animal["n"]}: #{sounds[animal["id"]]}"
end
puts results
# => cat: meow
# => dog: woof
Seems like the second array should just be a hash instead. There's no point creating an array if there's only one element in it and that number won't change.
pointless one-liner (don't use this)
puts arr1.map { |x| "#{x["n"]}: #{arr2[0][x["id"]]}" }
You can also get the join result by following code
arr1.collect{ |a| {a["n"] => arr2[0][a["id"]]} }
I'm very new to Ruby so please go easy on me. I have this small function that doesn't want to perform an intersection command. If I go into irb and enter the arrays then set the intersection command like: third_array = array1 & array2, third_array returns the common element. But when I run this snippet through irb, it just returns [ ]. Any suggestions?
class String
define_method(:antigrams) do |word2|
array1 = []
array2 = []
array1.push(self.split(""))
array2.push(word2.split(""))
third_array = array1 & array2
third_array
end
end
After looking at what you have, I think your code boils down to this:
class String
def antigrams(word)
self.chars & word.chars
end
end
"flurry".antigrams("flagrant")
# => ["f", "l", "r"]
If you're calling split('') on a word that's effectively the same as chars, though a lot less efficient. Another mistake was pushing a whole array into an array, which creates a nested array of the form [ [ 'f', 'l', ... ] ]. Since the two resulting array-of-arrays have nothing in common, their inner arrays are different, the & operation returns an empty array.
What you meant was to concatenate the one array to the other, something that can be done with += for example.
Whenever you're curious what's happening, either use irb to try out chunks of code, or p to debug at different points in your method.