How do I compute element-wise sum of the Arrays?
val a = new Array[Int](5)
val b = new Array[Int](5)
// assign values
// desired output: Array -> [a(0)+b(0), a(1)+b(1), a(2)+b(2), a(3)+b(3), a(4)+b(4)]
a.zip(b).flatMap(_._1+_._2)
missing parameter type for expanded function
Try:
a.zip(b).map { case (x, y) => x + y }
When you use an underscore as a placeholder in a function definition, it can only appear once (for each function argument position, that is, but in this case flatMap takes a Function1, so there's only one). If you need to refer to an argument more than once, you can't use the placeholder syntax—you'll need to give the argument a name.
As the other answers point out, you can use .map { case (x, y) => x + y } or the tuple accessor version, but it's also worth noting that if you want to avoid a bunch of tuple allocations in an intermediate collection, you can write the following:
scala> (a, b).zipped.map(_ + _)
res5: Array[Int] = Array(0, 0, 0, 0, 0)
Here zipped is a method that's available on pairs of collections that has a special map that takes a Function2, which means the only tuple that gets created is the (a, b) pair. The extra efficiency probably doesn't matter much in most cases, but the fact that you can pass a Function2 instead of a function from pairs means the syntax is often a little nicer as well.
// one D Array
val x = Array(1, 2, 3, 40, 55)
val x1 = Array(1, 2, 3, 40, 55)
x.indices.map(i=>x(i)+ x(i) )
// TWo D Array
val x1= Array((3,5), (5,7))
val x = Array((1,2), (3,4))
x.indices.map(i=>( x(i)._1 + x1(i)._1, x(i)._2 + x1(i)._2))
Related
I have an array of elements (numbers in my case)
var myArray= Array(1, 2, 3, 4, 5, 6)
//myArray: Array[Int] = Array(1, 2, 3, 4, 5, 6)
and I want to obtain a tuple than contain all of them:
var whatIwant= (1,2,3,4,5,6)
//whatIwant: (Int, Int, Int, Int, Int, Int) = (1,2,3,4,5,6)
I tried the following code but it doesn't work:
var tuple = myArray(0)
for (e <- myArray)
{
var tuple = tuple :+ e
}
//error: recursive variable tuple needs type
The trivial answer is this:
val whatIwant =
myArray match {
case Array(a, b, c, d, e, f) => (a, b, c, d, e, f)
case _ => (0, 0, 0, 0, 0, 0)
}
If you want to support different numbers of element in myArray then you are in for a world of pain because you will lose all the type information associated with a tuple.
If you are using Spark you should probably use its mechanisms to generate the data you want directly rather than converting to Array first.
The number of elements of a tuple is not infinitely superimposed. In earlier versions, there were only 22 at most. Scala treats tuples with different numbers of elements as different classes. So you can't add elements like a list.
Apart from utilizing metaprogramming techniques such as reflection, tuple objects may only be generated explicitly.
Having an Array of Int, and a function without parameters:
scala> val a = new Array[Int](5)
a: Array[Int] = Array(0, 0, 0, 0, 0)
scala> def f(): Int = 1
f: ()Int
I want to apply the function f() to the array with map() or transform(). I tried the following approaches.
First approach
scala> a.map(f)
<console>:14: error: type mismatch;
found : () => Int
required: Int => ?
a.map(f)
^
It fails, which I don't really understand why.
Second approach
scala> a.map(x => f)
res1: Array[Int] = Array(1, 1, 1, 1, 1)
This one works. However, I'm declaring a parameter x that I don't use in the right side of =>. It seems that anonymous functions need at least one parameter.
Is it correct not to use x?
Is it even a bad functional style?
Practical example
To give an example on why one would use that. Imagine I have an array that at some moment I want to mutate to have random values:
val a = new Array[Int](5)
// ...
a.transform(x => random())
Try using underscore for ignored argument like so
a.map(_ => f)
which outputs
res0: Array[Int] = Array(1, 1, 1, 1, 1)
Just for the sake of playing with transformations:
def to1[A, B](f: () => B): A => B = (_: A) => f()
val a = new Array[Int](5)
def f(): Int = 1
a.map(to1(f))
Regarding your example, consider using fill:
val b = Array.fill[Int](a.size)(myArgLessFunc())
I have defined two arrays:
a=np.array([[2,3,4],[5,6,7],[8,9,10]])
b=np.array([-1,-2])
and created a third one:
x=np.asarray([[x - a/2, x + a/2] for x in b])
Now, I have defined two variables
u,v = x[:,0], x[:,1]
My question is extremely simple: is there a way to define those variables without the comma, using only array operations? If I write
u,v = x[:,]
the ordering comes out wrong.
If x is 2D:
u, v = x.T
If x is ND:
u, v = np.swapaxes(x, 0, 1)
To confirm:
>>> np.all(u == x[:, 0])
True
>>> np.all(v == x[:, 1])
True
I was trying to compare integer wrapped arrays in two different columns and give the ratings as string:
import org.apache.spark.sql.Row
import org.apache.spark.sql.functions._
import scala.collection.mutable.WrappedArray
The DataFrame data has column A and B with wrapped array I would like to compare:
val data = Seq(
(Seq(1,2,3),Seq(4,5,6),Seq(7,8,9)),
(Seq(1,1,3),Seq(6,5,7),Seq(11,9,8))
).toDF("A","B","C")
And here is how it looks like:
data: org.apache.spark.sql.DataFrame = [A: array<int>, B: array<int> ... 1 more field]
+---------+---------+----------+
| A| B| C|
+---------+---------+----------+
|[1, 2, 3]|[4, 5, 6]| [7, 8, 9]|
|[1, 1, 3]|[6, 5, 7]|[11, 9, 8]|
+---------+---------+----------+
Then here is the the user define function which I would like to compare each elements in paired arrays in column A and B per row and give the ratings with simple logics. For example if A(1) > B(1) then D(1) is "Top". So as first row with column D, I hope to have ["Top", "Top", "Top"]
def myToChar(num1: Seq[Int], num2: Seq[Int]): Seq[String] = {
val twozipped = num1.zip(num2)
for ((x,y) <- num1.zip(num2)) {
if (x > y) "Top"
if (x < y) "Well"
if (x == y) "Good"
}}
val udfToChar = udf(myToChar(_: Seq[Int], _: Seq[Int]))
val ouput = data.withColumn("D",udfToChar($"A",$"B"))
However, I kept getting the <console>:45: error: type mismatch; error information. Not sure if my udf() type definition is wrong and appreciate any guidance to correct my mistake.
Your myToChar definition is declared to return a Seq[String] - but its implementation doesn't - it returns Unit, because a for expression (without a yield clause) has Unit type.
You can fix this by fixing the implementation of the function:
Replace the for with a map operation
Replace the last if with an else, otherwise the mapping function also returns Unit for inputs that adhere to none of the if conditions (unlike with pattern matching, the compiler can't conclude that your if conditions are exhaustive - it must assume there's also a possibility none of them would hold true)
So - a correct implementation would be:
def myToChar(num1: Seq[Int], num2: Seq[Int]): Seq[String] = {
num1.zip(num2).map { case (x, y) =>
if (x > y) "Top"
if (x < y) "Well"
else "Good"
}
}
Or alternatively using pattern matching with guards:
def myToChar(num1: Seq[Int], num2: Seq[Int]): Seq[String] = {
num1.zip(num2).map {
case (x, y) if x > y => "Top"
case (x, y) if x < y => "Well"
case _ => "Good"
}
}
I have two arrays of strings, say
A = ('abc', 'joia', 'abas8', '09ma09', 'oiam0')
and
B = ('gfdg', '89jkjj', '09ma09', 'asda', '45645ghf', 'dgfdg', 'yui345gd', '6456ds', '456dfs3', 'abas8', 'sfgds').
What I want to do is simply to count the number of elements of every string in A that appears in B (if any). For example, the resulted array here should be: C = (0, 0, 1, 1, 0). How can I do that?
try this:
A.map( x => B.count(y => y == x)))
You can do it how idursun suggested, it will work, but may be not efficient as if you'll prepare intersection first. If B is much bigger than A it will give massive speedup. 'intersect' method has better 'big-O' complexity then doing linear search for each element of A in B.
val A = Array("abc", "joia", "abas8", "09ma09", "oiam0")
val B = Array("gfdg", "89jkjj", "09ma09", "asda", "45645ghf", "dgfdg", "yui345gd", "6456ds", "456dfs3", "abas8", "sfgds")
val intersectCounts: Map[String, Int] =
A.intersect(B).map(s => s -> B.count(_ == s)).toMap
val count = A.map(intersectCounts.getOrElse(_, 0))
println(count.toSeq)
Result
(0, 0, 1, 1, 0)
Use a foldLeft construction as the yield off of each element of A:
val A = List("a","b")
val B = List("b","b")
val C = for (a <- A)
yield B.foldLeft(0) { case (totalc : Int, w : String) =>
totalc + (if (w == a) 1 else 0)
}
And the result:
C: List[Int] = List(0, 2)