incredible implicit Array conversion in scala - arrays

According to Scaladoc, there is no method named map in Array class, but there is an implicit function implicit def intArrayOps (xs: Array[Int]): ArrayOps[Int] defined in scala.Predef. So you can apply map on Array(1,2,3,4) if you like. But What I am confused about is that the map result is of type Array[Int], not ArrayOps[Int]. Here is my test:
scala> val array = Array(1,2,3,4)
array: Array[Int] = Array(1, 2, 3, 4)
scala> array.map(x => x)
res18: Array[Int] = Array(1, 2, 3, 4)
scala> res18.isInstanceOf[Array[Int]]
res19: Boolean = true
scala> res18.isInstanceOf[scala.collection.mutable.ArrayOps[Int]]
warning: there wre 1 unchecked warnings; re-run with -unchecked for details
res20: Boolean = false

It indeed returns an array, as intended and as is convenient, there is no reason you would need an ArrayOps, it is intended only to provide extra methods to arrays. The doc is wrong.
The routine is actually not implemented in ArrayOps. As most collection methods, it is inherited from TraversableLike. And you see two map methods in the doc:
def map [B] (f: (T) ⇒ B): ArrayOps[B]
def map [B, That] (f: (T) ⇒ B)(implicit bf: CanBuildFrom[Array[T], B, That]): That
Only the second one exists (inherited from TraversableLike). It is intended to allow implementation of map in just one place (traversable like) while allways giving the best possible behavior. For instance, a String is a of Seq[Char], if you map with a function from character to character, you get a String, but if you map from collection to say Int, the result cannot be a String and it will just be a Seq. This is explained in much detail in the paper fighting the bit rot with types.
However, this makes for a very complex signature, which does not reflect the simplicity of using the method, and makes very poor documentation most of the time (you normally would have to chase to which CanBuildFrom in implicit scope would work). This was discussed in this most famous scala question of stack overflow. So the tool scaladoc was extended so that a simpler entry, corresponding to intended usage, may appear. If you look at the source of GenTraversableLike, where the routine is introduced, you will see the following in the scaladoc for map (and a similar one in many methods)
#usecase def map[B](f: A => B): $Coll[B]
Subtypes add in their doc #define Coll <className>, and map (among others) appears with the simplified signature, marked [Use case]. In the source of ArrayOps, there is a #define Coll ArrayOps where it should be Array.

You can use the REPL with the -Xprint:typer option to see what's going on. Here is the output of the map method, reformatted for easier reading:
$ scala -Xprint:typer
scala> Array(1,2,3,4).map(x => x)
[[syntax trees at end of typer]]// Scala source: <console>
// some lines deleted
private[this] val res0: Array[Int] =
scala.this.Predef.intArrayOps(scala.Array.apply(1, 2, 3, 4))
.map[Int, Array[Int]]
(( (x: Int) => x ))
(scala.this.Array.canBuildFrom[Int](reflect.this.Manifest.Int));
So simplifying for package names here is what happens:
intArrayOps(Array(1,2,3,4)) // converts to ArrayOps
.map[Int, Array[Int]] // calls map with parameter lists below
((x:Int) => x) // pass identity function as fisrt param
(Array.canBuildFrom[Int]// pass builder for Array[Int] as second param
(Manifest.Int)) // pass class manifest for Int
So there is indeed a conversion to ArrayOps (first line). It returns ArrayOps[Int].
The ArrayOps.map[Int, Array[Int]] method is then called on it. Then as didierd explain, the original signature for map - not the simplified signature - indicates that the return type inferred will be Array[Int]

Related

Haskell Array Pattern in a function

Hi total Haskell beginner here: What does the pattern in a function for an array look like ? For example: I simply want to add +1 to the first element in my array
> a = array (1,10) ((1,1) : [(i,( i * 2)) | i <- [2..10]])
My first thought was:
> arraytest :: Array (Int,Int) Int -> Array (Int,Int) Int
> arraytest (array (mn,mx) (a,b):xs) = (array (mn,mx) (a,b+1):xs)
I hope you understand my problem :)
You can't pattern match on arrays because the data declaration in the Data.Array.IArray module for the Array type doesn't have any of its data constructors exposed. This is a common practice in Haskell because it allows the author to update the internal representation of their data type without making a breaking change for users of their module.
The only way to use an Array, therefore, is to use the functions provided by the module. To access the first value in an array, you can use a combination of bounds and (!), or take the first key/value pair from assocs. Then you can use (//) to make an update to the array.
arraytest arr = arr // [(index, value + 1)]
where
index = fst (bounds arr)
value = arr ! index
If you choose to use assocs, you can pattern match on its result:
arraytest arr = arr // [(index, value + 1)]
where
(index, value) = head (assocs arr) -- `head` will crash if the array is empty
Or you can make use of the Functor instances for lists and tuples:
arraytest arr = arr // take 1 (fmap (fmap (+1)) (assocs arr))
You will probably quickly notice, though, that the array package is lacking a lot of convenience functions. All of the solutions above are fairly verbose compared to how the operation would be implemented in other languages.
To fix this, we have the lens package (and its cousins), which add a ton of convenience functions to Haskell and make packages like array much more bearable. This package has a fairly steep learning curve, but it's used very commonly and is definitely worth learning.
import Control.Lens
arraytest arr = arr & ix (fst (bounds arr)) +~ 1
If you squint your eyes, you can almost see how it says arr[0] += 1, but we still haven't sacrificed any of the benefits of immutability.
This is more like an extended comment to #4castle's answer. You cannot pattern match on an Array because its implementation is hidden; you must use its public API to work with them. However, you can use the public API to define such a pattern (with the appropriate language extensions):
{-# LANGUAGE PatternSynonyms, ViewPatterns #-}
-- PatternSynonyms: Define patterns without actually defining types
-- ViewPatterns: Construct patterns that apply functions as well as match subpatterns
import Control.Arrow((&&&)) -- solely to dodge an ugly lambda; inline if you wish
pattern Array :: Ix i => (i, i) -> [(i, e)] -> Array i e
-- the type signature hints that this is the array function but bidirectional
pattern Array bounds' assocs' <- ((bounds &&& assocs) -> (bounds', assocs'))
-- When matching against Array bounds' assocs', apply bounds &&& assocs to the
-- incoming array, and match the resulting tuple to (bounds', assocs')
where Array = array
-- Using Array in an expression is the same as just using array
arraytest (Array bs ((i,x):xs)) = Array bs ((i,x+1):xs)
I'm fairly sure that the conversions to and from [] make this absolutely abysmal for performance.

Type Mismatch in scala, Array[Int] and Array[Option[Int]]

First, I am new to Scala, So apologies if the following question is too simple.
I have written the following code to the find the values of the keys that I supply in an array from the map.
def stringToCountMap(inputArray: Array[String], inputMap:Map[String,Int]) : Array[Int] = {
return inputArray.map(x => inputMap.get(x))
}
I got the following error,
type mismatch;
found : Array[Option[Int]]
required: Array[Int]
return inputArray.map(x => inputMap.get(x))
Question:
1) Can anyone explain what is Option[Int]?
2) What is my mistake here ?
Thanks in advance.
Option is Scala's option type (also called a nullable type). It represents a case where the value may not exist.
Consider a map that doesn't contain a requested key. How would you handle a request for the key? One option is to result in an error, such as by throwing an exception. Another is to return a special value that indicates there is no value. Map.get does the latter, using Option as the special type and None as the value. This means the return type of Map.get isn't the value type of the map (Int), but Option applied to the value type (Option[Int]).
To correct the type declaration, change the return type:
def stringToCountMap(inputArray: Array[String], inputMap:Map[String,Int]) : Array[Option[Int]] = {
inputArray.map(x => inputMap.get(x))
}
You can leave out the return type of stringToCountMap and let type inference handle it:
def stringToCountMap(inputArray: Array[String], inputMap:Map[String,Int]) = {
inputArray.map(x => inputMap.get(x))
}
As a consequence, missing keys from the input map get carried through:
scala> stringToCountMap(Array("a", "def"), Map("a" -> 1, "bc" -> 2))
res0: Array[Option[Int]] = Array(Some(1), None)
Option[T] is a wrapper around a value of type T. Its purpose is to prevent NullPointerException, that you might know from Java. A value of type Option[T] might either be None, which, as the name implies is an object that represents nothing, or it might be Some(x: T), which represents an existing value.
inputMap(x) returns an Option[Int], since you have no guarantee that the x key exists in inputMap. If it does, it returns Some(value: Int), else it returns None.
Calling stringToCountMap(Array("a", "b", "c"), Map("a" -> 1, "c" -> 2)) results in Array(Some(1), None, Some(2))
If you want an Array[Int] instead, you might do something like inputArray.map(x => inputMap.getOrElse(x, 0)).get. The getOrElse method has two parameters, where the first one is the key, and the second one is the default value. inputArray.map(x => inputMap.get(x).getOrElse(0)) has the same effect, since calling getOrElse(value) on an Option either unwraps the Some object, or returns the default value.
Now, stringToCountMap(Array("a", "b", "c"), Map("a" -> 1, "c" -> 2)) results in Array(1, 0, 2).
You might also want to omit the keys missing in the input array. In that case, you might do inputArray.flatMap(x => inputMap.get(x)). flatMap is a function similar to map, but it returns strictly, as the name implies, flat collections. For example, calling flatMap(x => x) on an Array[Array[Int]] would return an Array of all the values in the 2D array in a single row.
Here, Option is a collection, as well. If it is of type Some, it contains a single value, if it is None, it is an empty collection. Thus, in the resulting array you would only have the values of the keys present in the map, and the keys not present in the map are skipped.
Now, stringToCountMap(Array("a", "b", "c"), Map("a" -> 1, "c" -> 2)) results in Array(1, 2).
First, there is no need to use 'return' in scala to return any value.
def stringToCountMap(inputArray: Array[String], inputMap:Map[String,Int]) = {
inputArray.map(x => inputMap.get(x))
}
When you get the value of key from Map, It returns result in Option.
For example:
scala> val map = Map(1-> "a",2 -> "b")
map: scala.collection.immutable.Map[Int,String] = Map(1 -> a, 2 -> b)
scala> map.get(1)
res0: Option[String] = Some(a)
scala> map.get(3)
res1: Option[String] = None
When you try to get the value of key, which does not exist. In java, you have encountered with NullPointerException. So when there is no value, it returns None to avoid exception.
For more info refer
In your method, you have given return type as Array[Int] but function returns Array[Option[Int]] that's why it throws compilation error.

Using the apply / (array subscripting) method in a builder pattern

Given a trivial function returning an array:
scala> def methodReturnsArray() = { Array(1.0, 2.0) }
methodReturnsArray: ()Array[Double]
We can go ahead and invoke the function:
scala> val myarr = methodReturnsArray
myarr: Array[Double] = Array(1.0, 2.0)
scala> myarr(0)
res21: Double = 1.0
However, it is not possible to use the apply ( / array index semantics) directly:
scala> methodReturnsArray(0)
<console>:53: error: too many arguments for method methodReturnsArray: ()Array[Double]
methodReturnsArray(0)
^
The request is to explain why that were not possible as given. Secondarily: is there some way to get an "inline" invocation. I.e. not requiring to separate out the steps: s
(a) invoke the method and
(b) access the specific element of the array (via apply())
on separate statements.
Aha! Seconds after posting this I arrived at the answer: need to include the parens on the function call: i.e methodReturnsArray () (0) :
scala> methodReturnsArray()(0)
res22: Double = 1.0

Function defined with generic types

I searched for a while and I was not able to find a clear and satisfying answer to this question.
Suppose I have this little function to compute the mean value of an Array:
def meanofArray(s:Array[Double]) : Double = s.sum/s.length
the problem is that is Array has to be Double.
If for example I have this Array:
val x = Array( 1 , 2 , 3 , 4 )
and make the call :
println(meanofArray(x))
I get the error found : Array[Int]
required: Array[Double]
How can I implement meanofArray so as to be able to accept any type of Int , Long , Float and Double with no errors ?
My thought was Generic types :
def meanofArray(s:Array[T]) : Double = s.sum/s.length
but I could not get it working.
Any other idea is welcomed !
The issue you are running into demonstrates the beauty of type abstraction.
Based solely on the signature of the function you provided
def meanofArray(s:Array[T]) : T
we know quite a bit about what this function can and can not do, as long as it does not do shady things like reflection of course. It can not do things like sort the Array, find the sum, etc. Based on some unfortunate characteristics of Scala, there are some things which we can not be sure it does not do, like use equals or toString for some T.
You need to declare some additional information about the type T in the function signature.
def meanofArray[T](s: Array[T])(implicit ev: Numeric[T]): Double
Here we are declaring that T must have a definition of Numeric[T] defined and in scope at call sites. Numeric[T] provides the definitions of functions required in order to do the calculations you need. We are now declaring that this function may do more things like sort the array or find the sum.
And the implementation you are looking for...
def meanofArray[T](s: Array[T])(implicit ev: Numeric[T]): Double =
s.view.map(ev.toDouble).sum / s.length
EDIT: Map s to view of Array[Double] to decrease likelihood of overflow. Taken from #Eastsun's answer. Taking Numeric[T] implicitly is strongly preferred over implicit view from T => Double in my opinion though.
Here is what you want(The parameter's type of mean is Seq[A] so that you can use it as mean(List(1, 2, 3)) or mean("ABCDEFG")):
scala> def mean[A](arr: Seq[A])(implicit view: A => Double): Double =
(arr map view sum) / arr.size
mean: [A](arr: Seq[A])(implicit view: A => Double)Double
scala> mean(Array(1, 2, 3, 4))
res0: Double = 2.5
Compare with #drstevens 's solution:
scala> def meanofArray[T](s: Array[T])(implicit ev: Numeric[T]): Double =
| ev.toDouble(s.sum) / s.length
meanofArray: [T](s: Array[T])(implicit ev: scala.math.Numeric[T])Double
scala> meanofArray(Array[Byte](111, 111, 111, 111))
res3: Double = -17.0 //Wrong!!!
scala> mean((Array[Byte](111, 111, 111, 111)))
res4: Double = 111.0 //Right !!!
This is the cleanest solution i could come up for your question.
def mean[A <% Double : Numeric](seq:Seq[A]) = seq.sum/seq.length
Here, Seq.sum needs an evidence of A conforms to be a Numeric type. (: in the type parameter)
And that division to work, the result of seq.sum should be viewable as a Double. Altogether this results a result mean of Double type

How do I create a heterogeneous Array in Scala?

In javascript, we can do:
["a string", 10, {x : 1}, function() {}].push("another value");
What is the Scala equivalent?
Arrays in Scala are very much homogeneous. This is because Scala is a statically typed language. If you really need pseudo-heterogeneous features, you need to use an immutable data structure that is parametrized covariantly (most immutable data structures are). List is the canonical example there, but Vector is also an option. Then you can do something like this:
Vector("a string", 10, Map("x" -> 1), ()=>()) + "another value"
The result will be of type Vector[Any]. Not very useful in terms of static typing, but everything will be in there as promised.
Incidentally, the "literal syntax" for arrays in Scala is as follows:
Array(1, 2, 3, 4) // => Array[Int] containing [1, 2, 3, 4]
See also: More info on persistent vectors
Scala will choose the most specific Array element type which can hold all values, in this case it needs the most general type Any which is a supertype of every other type:
Array("a string", 10, new { val x = 1 }, () => ()) :+ "another value"
The resulting array will be of type Array[Any].
Scala might get the ability for a "heterogeneous" list soon:
HList in Scala
Personally, I would probably use tuples, as herom mentions in a comment.
scala> ("a string", 10, (1), () => {})
res1: (java.lang.String, Int, Int, () => Unit) = (a string,10,1,<function0>)
But you cannot append to such structures easily.
The HList mentioned by ePharaoh is "made for this" but I would probably stay clear of it myself. It's heavy on type programming and therefore may carry surprising loads with it (i.e. creating a lot of classes when compiled). Just be careful. A HList of the above (needs MetaScala library) would be (not proven since I don't use MetaScala):
scala> "a string" :: 10 :: (1) :: () => {} :: HNil
You can append etc. (well, at least prepend) to such a list, and it will know the types. Prepending creates a new type that has the old type as the tail.
Then there's one approach not mentioned yet. Classes (especially case classes) are very light on Scala and you can make them as one-liners:
scala> case class MyThing( str: String, int: Int, x: Int, f: () => Unit )
defined class MyThing
scala> MyThing( "a string", 10, 1, ()=>{} )
res2: MyThing = MyThing(a string,10,1,<function0>)
Of course, this will not handle appending either.

Resources