I want to map an array in every 3 elements, output is many [k,v]pairs, for example:
input: array(1,2,3,4,5,6,7,8,9,7,12,11)
output: (1 => 2,3) (4 => 5,6)(7 => 8,9) (7 => 12,11)
And I also want to reduce these pairs by keys, for example ,if I want to collect the data with key=7, then the output will be (7=> 8,9,12,11).
Thanks a lot.
I think what you need is following
val input = Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 7, 12, 11)
val output = input.toSeq.grouped(3)
.map(g => (g.head, g.tail)).toList
.groupBy(_._1)
.mapValues(l => l.flatMap(_._2))
Result would be
Map(4 -> List(5, 6), 7 -> List(8, 9, 12, 11), 1 -> List(2, 3))
Try this:
res0 = list.grouped(3).map {x => (x(0), List(x(1),x(2)))}.toList
// you must dump your converted data format into your storage eg hdfs.
// And not the entire thing in the form of array. Transform in form of
// (key,value) and dump in hdfs. That will save a lot of computation.
res1 = sc.parallelize(res0)
res2 = res1.reduceByKey(_++_).collect
But I am not sure how much scalable this solution would be.
EDIT
val res1 = sc.parallelize(arr)
// (1,2,3,4,5,6,7,8,9,7,12,11)
val res2 = res1.zipWithIndex.map(x._2/3,List(x._1))
// (1,0),(2,1),...(12,10),(11,11) -> (0,1),(0,2),(0,3),(1,4),(1,5),(1,6)
val res3 = res2.reduceByKey(_++_).map(_._2)
//(0,List(1,2,3)),(1,List(4,5,6)) -> List(1,2,3),List(4,5,6)
val res4 = res3.map(x => x match {
case x1::xs => (x1,xs)
}).reduceByKey(_++_)
//List(1,2,3) - > (1,List(2,3)) -> reduceByKey
//(1,List(2,3)),(4,List(5,6)),(7,List(8,9,12,11))
Related
I am trying to rotate an array from a particular position using array reversal method.
Input array: [1,2,3,4,5,6,7]
d = 3
Output array: [5,6,7,1,2,3,4]
To achieve this I thought of working on the array in three steps.
Step1: Reverse the array from starting position until d => [4,3,2,1,5,6,7]
Step2: Reverse the array from d till the end of the array => [4,3,2,1,7,6,5]
Step3: Reverse the complete array from Step2 => [5,6,7,1,2,3,4]
I haven't followed any functional programming pattern as I want to check the algorithm step by step.
val arr = Array[Int](1, 2, 3, 4, 5, 6, 7)
def reverseAlgo(brr: Array[Int], start: Int, end: Int): Unit = {
var temp = 0
for(i <- start until end/2) {
temp = brr(i)
brr(i) = brr(end-i-1)
brr(end-i-1) = temp
}
brr.foreach(println)
}
Step1 is working fine:
reverseAlgo(arr, 0, 3)
Output:
3
2
1
4
5
6
7
But Step2 is not producing the required output:
reverseAlgo(arr, 3, 7)
Output:
3
2
1
4
5
6
7
As you see, the output of the array should be: 3,2,1,7,6,5,4
Since the output from Step2 is incorrect, the final output is also wrong.
Step3:
reverseAlgo(arr, 0, arr.length)
Output:
7
6
5
4
1
2
3
Could anyone let me know what is the mistake I am doing here ?
Why not just something as simple as this?
import scala.collection.immutable.ArraySeq
import scala.reflect.ClassTag
def rotate[T : ClassTag](arr: ArraySeq[T])(pos: Int): ArraySeq[T] = {
val length = arr.length
ArraySeq.tabulate[T](n = length) { i =>
arr((i + 1 + pos) % length)
}
}
Which can be used like this:
rotate(arr = ArraySeq(1, 2, 3, 4, 5, 6, 7))(pos = 3)
// res: ArraySeq[Int] = ArraySeq(, 5, 6, 7, 1, 2, 3, 4)
You can see the code running here.
Your code will only work when the range starts at zero.
for(i <- start until end/2) {
temp = brr(i)
brr(i) = brr(end-i-1)
brr(end-i-1) = temp
}
Should be something like:
for(i <- 0 until (end-start)/2) {
temp = brr(start+i)
brr(start+i) = brr(end-i-1)
brr(end-i-1) = temp
}
With this change your code works.
Mutation is to be avoided but, if you must, recursion is still useful.
def reversePart[A](arr: Array[A], start: Int, end: Int): Unit = {
def loop(a:Int, b:Int): Unit =
if (a < b) {
val temp = arr(a)
arr(a) = arr(b)
arr(b) = temp
loop(a+1, b-1)
}
loop(start max 0, end min arr.length-1)
}
val test = Array(1, 2, 3, 4, 5, 6, 7)
reversePart(test, 0, 3) //Array(4, 3, 2, 1, 5, 6, 7)
reversePart(test, 4, 7) //Array(4, 3, 2, 1, 7, 6, 5)
reversePart(test, -1, 99) //Array(5, 6, 7, 1, 2, 3, 4)
I realize this doesn't directly answer your question, but for reference and for readers interested in a slightly different approach, one possibility is to implement this as a view.
In this example, Rotate implements the logic, while IndexedSeqViewRotate adds the rotate method as an extension to any IndexedSeqView as long as it's in scope.
In the tests, I materialized the views into Vectors to take advantage of the equality, but of course you can materialize them into an Array as well.
import scala.collection.IndexedSeqView
import scala.collection.IndexedSeqView.SomeIndexedSeqOps
final class Rotate[A](underlying: SomeIndexedSeqOps[A], n: Int) extends IndexedSeqView[A] {
#inline private def rotateIndex(i: Int): Int = ((i - n) % length + length) % length
override def apply(i: Int): A = underlying(rotateIndex(i))
override lazy val length: Int = underlying.length
}
final implicit class IndexedSeqViewRotate[A](val underlying: IndexedSeqView[A]) extends AnyVal {
def rotate(n: Int): IndexedSeqView[A] = new Rotate(underlying, n)
}
assert(Array().view.rotate(7).to(Vector) == Vector.empty)
assert(Array(1,2,3,4,5,6,7).view.rotate(7).to(Vector) == Vector(1,2,3,4,5,6,7))
assert(Array(1,2,3,4,5,6,7).view.rotate(0).to(Vector) == Vector(1,2,3,4,5,6,7))
assert(Array(1,2,3,4,5,6,7).view.rotate(1).to(Vector) == Vector(7,1,2,3,4,5,6))
assert(Array(1,2,3,4,5,6,7).view.rotate(3).to(Vector) == Vector(5,6,7,1,2,3,4))
assert(Array(1,2,3,4,5,6,7).view.rotate(-1).to(Vector) == Vector(2,3,4,5,6,7,1))
You can play around with this code here on Scastie.
Assume that I have an array like 1 2 3 4 5, I want to rotate it to the left by n and get a new one.
For example the 2 rotation of the above array will result in 3 4 5 1 2. I didn't found any extension function to do that.
You can use built-in java Collections.rotate method, but you need to convert your array to list firstly:
val arr = intArrayOf(1, 2, 3, 4, 5)
val list = arr.toList()
Collections.rotate(list, -2)
println(list.toIntArray().joinToString())
Outputs
3, 4, 5, 1, 2
I interpret "get a new one" to mean that the extension function should return a new array instance, like so (boundary checks omitted, sliceArray is an stdlib function) :
fun <T> Array<T>.rotate(n: Int) =
let { sliceArray(n until size) + sliceArray(0 until n) }
Example
arrayOf(1, 2, 3, 4, 5).rotate(1)
.also { println(it.joinToString()) } // 2, 3, 4, 5, 1
Another extension function, by slicing the array in 2 parts left and right and reassembling it to right + left:
fun <T> Array<T>.leftShift(d: Int) {
val n = d % this.size // just in case
if (n == 0) return // no need to shift
val left = this.copyOfRange(0, n)
val right = this.copyOfRange(n, this.size)
System.arraycopy(right, 0, this, 0, right.size)
System.arraycopy(left, 0, this, right.size, left.size)
}
so this:
val a = arrayOf(1, 2, 3, 4, 5, 6, 7)
a.leftShift(2)
a.forEach { print(" " + it) }
will print
3 4 5 6 7 1 2
Simple solution:
fun <T> Array<T>.rotateLeft(n: Int) = drop(n) + take(n)
fun <T> Array<T>.rotateRight(n: Int) = takeLast(n) + dropLast(n)
The limitation is that n must be less than or equal to the length of the array.
Alternatively, you can use Collections.rotate(...) as follows.
import java.util.Collections
fun <T> Array<T>.rotate(distance: Int) =
toList().also { // toList() is a deep copy to avoid changing the original array.
Collections.rotate(it, distance)
}
fun main() {
val xs = arrayOf(1, 2, 3, 4, 5)
val ys = xs.rotate(-2)
xs.forEach { print("$it ") } // 1 2 3 4 5
println(ys) // [3, 4, 5, 1, 2]
}
For the record, you can use the regular Array constructor to build a new array:
inline fun <reified T> Array<T>.rotate(n: Int) = Array(size) { this[(it + n) % size] }
The element at index it in the source array is copied in the destination array at the new index (it + n) % size to perform the rotation.
It is a bit slower than copying the array by chunks.
I am trying to find the common strings in a map and an array to output the respective values(from map, values here is Map[key -> value]) in Scala, I'm trying to not use any loops. Example:
Input:
Array("Ash","Garcia","Mac") Map("Ash" -> 5, "Mac" -> 4, "Lucas" -> 3)
Output:
Array(5,4)
The output is an array with 5 and 4 because Ash and Mac are common in both the data structures
What constitutes a loop?
def common(arr: Array[String], m: Map[String,Int]): Array[Int] =
arr flatMap m.get
Usage:
common(Array("Ash","Garcia","Mac")
,Map("Ash" -> 5, "Mac" -> 4, "Lucas" -> 3))
// res0: Array[Int] = Array(5, 4)
This is the most elegant solution, I think, but the results may not fit your requirements if there are duplicates in the array.
yourArray.collect(yourMap) // Array(5,4)
Use .filter to find the matching entries only, then get the value of your filtered map.
Given
scala> val names = Array("Ash","Garcia","Mac")
names: Array[String] = Array(Ash, Garcia, Mac)
scala> val nameToNumber = Map("Ash" -> 5, "Mac" -> 4, "Lucas" -> 3)
nameToNumber: scala.collection.immutable.Map[String,Int] = Map(Ash -> 5, Mac -> 4, Lucas -> 3)
.filter.map
scala> nameToNumber.filter(x => names.contains(x._1)).map(_._2)
res3: scala.collection.immutable.Iterable[Int] = List(5, 4)
Alternatively, you can use collect,
scala> nameToNumber.collect{case kv if names.contains(kv._1) => kv._2}
res6: scala.collection.immutable.Iterable[Int] = List(5, 4)
Your complexity here is O(n2)
Quite easy for scala elegant syntax：
val a = Array("Ash","Garcia","Mac")
val m = Map("Ash" -> 5, "Mac" -> 4, "Lucas" -> 3)
println(m.filter { case (k, v) => a.contains(k)}.map { case (k, v) => v}.toArray)
Here is the solution！
I am new to Scala and this is the first time I'm using it. I want to read in a textfile with with two columns of numbers and store each column items in a separate list or array that will have to be cast as integer. For example the textfile looks like this:
1 2
2 3
3 4
4 5
1 6
6 7
7 8
8 9
6 10
I want to separate the two columns so that each column is stored in its on list or array.
Lets say you named the file as "columns", this would be a solution:
val lines = Source.fromFile("columns").getLines()
/* gets an Iterator[String] which interfaces a collection of all the lines in the file*/
val linesAsArraysOfInts = lines.map(line => line.split(" ").map(_.toInt))
/* Here you transform (map) any line to arrays of Int, so you will get as a result an Interator[Array[Int]] */
val pair: (List[Int], List[Int]) = linesAsArraysOfInts.foldLeft((List[Int](), List[Int]()))((acc, cur) => (cur(0) :: acc._1, cur(1) :: acc._2))
/* The foldLeft method on iterators, allows you to propagate an operation from left to right on the Iterator starting from an initial value and changing this value over the propagation process. In this case you start with two empty Lists stored as Tuple an on each step you prepend the first element of the array to the first List, and the second element to the second List. At the end you will have to Lists with the columns in reverse order*/
val leftList: List[Int] = pair._1.reverse
val rightList: List[Int] = pair._2.reverse
//finally you apply reverse to the lists and it's done :)
Here is one possible way of doing this:
val file: String = ??? // path to .txt in String format
val source = Source.fromFile(file)
scala> val columnsTogether = source.getLines.map { line =>
val nums = line.split(" ") // creating an array of just the 'numbers'
(nums.head, nums.last) // assumes every line has TWO numbers only
}.toList
columnsTogether: List[(String, String)] = List((1,2), (2,3), (3,4), (4,5), (1,6), (6,7), (7,8), (8,9), (6,10))
scala> columnsTogether.map(_._1.toInt)
res0: List[Int] = List(1, 2, 3, 4, 1, 6, 7, 8, 6)
scala> columnsTogether.map(_._2.toInt)
res1: List[Int] = List(2, 3, 4, 5, 6, 7, 8, 9, 10)
I have two array like:
val one = Array(1, 2, 3, 4)
val two = Array(4, 5, 6, 7)
var three = one zip two map{case(a, b) => a * b}
It's ok.
But I have a multidimensional Array and a one-dimensional array now, like this:
val mulOne = Array(Array(1, 2, 3, 4),Array(5, 6, 7, 8),Array(9, 10, 11, 12))
val one_arr = Array(1, 2, 3, 4)
I would like to multiplication them, how can I do this in scala?
You could use:
val tmpArr = mulOne.map(_ zip one_arr).map(_.map{case(a,b) => a*b})
This would give you Array(Array(1*1, 2*2, 3*3, 4*4), Array(5*1, 6*2, 7*3, 8*4), Array(9*1, 10*2, 11*3, 12*4)).
Here mulOne.map(_ zip one_arr) maps each internal array of mulOne with corresponding element of one_arr to create pairs like: Array(Array((1,1), (2,2), (3,3), (4,4)), ..) (Note: I have used placeholder syntax). In the next step .map(_.map{case(a,b) => a*b}) multiplies each elements of pair to give
output like: Array(Array(1, 4, 9, 16),..)
Then you could use:
tmpArr.map(_.reduce(_ + _))
to get sum of all internal Arrays to get Array(30, 70, 110)
Try this
mulOne.map{x => (x, one_arr)}.map { case(one, two) => one zip two map{case(a, b) => a * b} }
mulOne.map{x => (x, one_arr)} => for every array inside mulOne, create a tuple with content of one_arr.
.map { case(one, two) => one zip two map{case(a, b) => a * b} } is basically the operation that you performed in your first example on every tuple that were created in the first step.
Using a for comprehension like this,
val prods =
for { xs <- mulOne
zx = one_arr zip xs
(a,b) <- zx
} yield a*b
and so
prods.sum
delivers the final result.