implicit conversion of an array of arrays - arrays

I have a case class called Point defined as follows:
case class Point(x: Double, y: Double)
and a functions that takes an array of points:
def f(coords: Array[Point]) ...
I want to be able to implictly pass an array of double arrays to my function. To achieve this I have defined the following two implicit functions:
implicit def arrayToPoint(a: Array[Double]) = new Point(a(0), a(1))
implicit def arraysToPoints(a: Array[Array[Double]]) = a map(p => Point(p(0), p(1)))
My questions is is there any way I can achieve this with just a single implict conversion function to simplify matters?
And as a related question what would be the best approach if I wanted to be able to pass an Array of Ints instead of Doubles?
Regards
Des

Your method arraysToPoints is superfluous. You could use a view bound on the array argument for your method f and add the conversion to the companion object of Point, like so:
object Point {
implicit def arrayToPoint[A](a: Array[A])(implicit view: A => Double): Point =
Point(a(0), a(1))
}
case class Point(x: Double, y: Double)
def f[P](coords: Array[P])(implicit view: P => Point): Unit = coords.foreach { p =>
println(p: Point)
}
f(Array(Point(1, 2), Point(2, 3)))
f(Array(Array(1.0, 2.0), Array(3.0, 4.0)))
f(Array(Array(1, 2), Array(3, 4)))
In order to allow both arrays of Int and Double to be covered, I have used a second view bound on the arrayToPoint method. Otherwise you would need two separate conversion methods for Array[Int] and Array[Double].
You can read this definition of f as, "take an array of elements of a type P which can be viewed as type Point". One spot where the compiler looks for such views is the companion object of the target type, thus object Point. This is a good place for implicit methods.
The second possibility would be to use the magnet pattern. Instead of converting point by point with a view in f, you would create a single wrapper object at once. This is a bit prettier and for large arrays minimises the penalty on direct Array[Double] arguments (because you instantiate the wrapper once, but then do not need to call the view function any more). arrayToPoint is used whenever the array element type A once again can be viewed as a Double. This is true for Double itself of course, but also for Int which can be seen as a Double though what Scala calls numeric widening (e.g., you can say val x: Double = 33 and the integer 33 is implicitly "widened" to a double).
object Points {
implicit def direct(a: Array[Point]): Points =
new Points {
val peer = a
}
implicit def indirect[A](a: Array[Array[A]])(implicit view: A => Double): Points =
new Points {
lazy val peer = a.map { c => Point(c(0), c(1)) }
}
}
trait Points {
def peer: Array[Point]
}
def f(coords: Points): Unit = coords.peer.foreach(println)
This looks in the companion object for an implicit method from the argument type to the special magnet type Points. I use lazy val for the non-direct arrays so that we might save the actual conversion action if the peer method is not called.

Related

Is the return type of this method a slice or a borrowed array?

I am working through the Rust book, specifically the "Departments" exercise in section 8.3.
I've become a little confused by what is being returned from a method I've created.
#[derive(Debug, PartialEq, Eq, Hash)]
enum Department {
Sales,
Engineering,
}
#[derive(Debug)]
struct Company {
users: HashMap<Department, Vec<String>>,
}
impl Company {
fn add_user_to_department(&mut self, department: Department, employee: String) {
let users = self.users.entry(department).or_insert(Vec::new());
users.push(employee);
}
fn get_users_in_department(&self, department: Department) -> &[String] {
match self.users.get(&department) {
Some(u) => &u,
None => &[]
}
}
}
At first I thought the method get_users_in_department is returning a slice of the Vec (or in the case of None a static literal).
But in my match statement I am not using the slice syntax, I am returning a borrowed reference to the Vec.
match self.users.get(&department) {
Some(u) => &u,
None => &[]
}
If I was returning a slice wouldn't I have to return Some(u) => &u[..],
I guess my confusion boils down to what the return type of get_users_in_department is. Is it returning a reference to an array of String, or a slice of an array of String. Are these two types interchangeable due to some sort of coercion? If it's possible to differentiate between the two, how and when would I want to return one or the other?
From Type Coercions in The Rust Reference:
Type coercions are implicit operations that change the type of a value. They happen automatically at specific locations and are highly restricted in what types actually coerce.
Coercion is allowed between the following types:
&T or &mut T to &U if T implements Deref<Target = U>.
This kind of type coercion is called a deref coercion. Relevant example: Vec<T> implements Deref<Target = [T]> so &Vec<T> can be coerced into &[T].
A coercion can only occur at certain coercion sites in a program; these are typically places where the desired type is explicit or can be derived by propagation from explicit types (without type inference). Possible coercion sites are:
Function results—either the final line of a block if it is not semicolon-terminated or any expression in a return statement.
Example:
// no type coercion
fn vec_to_vec<T>(vec: &Vec<T>) -> &Vec<T> {
vec
}
// &Vec<T> coerced to &[T] via deref coercion
fn vec_to_slice<T>(vec: &Vec<T>) -> &[T] {
vec
}
playground
To answer your question more directly, the type of &u inside get_users_in_department is &Vec<String> but it's being coerced into &[String] by the compiler for you.

Does ccall really convert arguments passed by pointer?

Considering a dynamic library with this native function that returns the sum of all even (32-bit unsigned) numbers in an array:
uint32_t sum_of_even(const uint32_t *numbers, size_t length);
The implementation of the function above was written in Rust as below, and packaged into a C dynamic library.
use libc::size_t;
use std::slice;
#[no_mangle]
pub extern "C" fn sum_of_even(n: *const u32, len: size_t) -> u32 {
let numbers = unsafe {
assert!(!n.is_null());
slice::from_raw_parts(n, len as usize)
};
numbers
.iter()
.filter(|&v| v % 2 == 0)
.sum()
}
I wrote the following Julia (v1.0.1) wrapper function:
lib = Libdl.dlopen(libname)
sumofeven_sym = Libdl.dlsym(lib, :sum_of_even)
sumofeven(a) = ccall(
sumofeven_sym,
UInt32,
(Ptr{UInt32}, Csize_t),
a, length(a)
)
The documentation states multiple times that arguments in ccall are converted to become compatible with the C function prototype (emphasis mine):
Each argvalue to the ccall will be converted to the corresponding argtype, by automatic insertion of calls to unsafe_convert(argtype, cconvert(argtype, argvalue)). (See also the documentation for unsafe_convert and cconvert for further details.) In most cases, this simply results in a call to convert(argtype, argvalue).
And moreover, that when passing an Array{T} by Ptr{U} to a C function, the call is invalidated if the two types T and U are different, since no reinterpret cast is added (section Bits Types):
When an array is passed to C as a Ptr{T} argument, it is not reinterpret-cast: Julia requires that the element type of the array matches T, and the address of the first element is passed.
Therefore, if an Array contains data in the wrong format, it will have to be explicitly converted using a call such as trunc(Int32, a).
However, this is seemingly not the case. If I deliberately pass an array with another type element:
println(sumofeven(Float32[1, 2, 3, 4, 5, 6]))
The program calls the C function with the array passed directly, without converting the values nor complaining about the different element types, resulting in either senseless output or a segmentation fault.
If I redefine the function to accept a Ref{UInt32} instead of a Ptr{UInt32}, I am prevented from calling it with the array of floats:
ERROR: LoadError: MethodError: Cannot `convert` an object of type Array{Float32,1} to an object of type UInt32
Closest candidates are:
convert(::Type{T<:Number}, !Matched::T<:Number) where T<:Number at number.jl:6
convert(::Type{T<:Number}, !Matched::Number) where T<:Number at number.jl:7
convert(::Type{T<:Integer}, !Matched::Ptr) where T<:Integer at pointer.jl:23
...
However, Ref was not designed for arrays.
Making the example work with Ptr{UInt32} requires me to either specify Array{UInt32} as the type of input a (static enforcement), or convert the array first for a more flexible function.
sumofeven(a:: Array{UInt32}) = ccall( # ← either this
sumofeven_sym,
UInt32,
(Ptr{UInt32}, Csize_t),
convert(Array{UInt32}, a), # ← or this
length(a))
With that, I still feel that there is a gap in my reasoning. What is the documentation really suggesting when it says that an array passed to C as a Ptr{T} is not reinterpret-cast? Why is Julia letting me pass an array of different element types without any explicit conversion?
This turned out to be either a bug in the core library or a very misguided documentation, depending on the perspective (issue #29850). The behavior of the function unsafe_convert changed from version 0.4 to 0.5, in a way that makes it more flexible than what is currently suggested.
According to this commit, unsafe_convert changed from this:
unsafe_convert(::Type{Ptr{Void}}, a::Array) = ccall(:jl_array_ptr, Ptr{Void}, (Any,), a)
To this:
unsafe_convert{S,T}(::Type{Ptr{S}}, a::AbstractArray{T}) = convert(Ptr{S}, unsafe_convert(Ptr{T}, a))
For arrays, this relaxed implementation will enable a transformation from an array of T to a pointer of another type S. In practice, unsafe_convert(cconvert(array)) will reinterpret-cast the array's base pointer, as in the C++ nomenclature. We are left with a dangerously reinterpreted array across the FFI boundary.
The key takeaway is that one needs to take extra care when passing arrays to C functions, as the element type of an array in a C-call function parameter is not statically enforced. Use type signatures and/or explicit conversions where applicable.

Cannot pass any array type to a function which takes [Any] as parameter

I have an array of type [String]
let names = ["Joffrey", "Cersei", "Mountain", "Hound"]
I have a function which takes an array of [Any] type.
func printItems(items: [Any]){
for item in items {
print(item)
}
}
Now when I call the function with names as parameters,
printItems(names)
I get an error Cannot invoke 'printItems' with an argument list of type '([String])'.
Any is just a typealias for a protocol which all types implicitly conform to.
Thoughts?
This is a surprising limitation of Swift. You can't cast an array to type [Any] so you can't pass it to a function taking type [Any]. You can use map to cast each item of the array:
printItems(names.map {$0 as Any})
But, the right way to do this in Swift is to use Generics:
func printItems<T>(items: [T]) {
for item in items {
print(item)
}
}
let names = ["Joffrey", "Cersei", "Mountain", "Hound"]
let numbers = [3.1416, 2.71818, 1.4142, 1.618034]
printItems(names) // This now works
printItems(numbers) // This works too
While every type conforms Any, this is not the same as it being a universal implicit superclass that all types inherit from.
When you cast a type to a protocol, you create a new value with a different structure. So for a string to be of type Any, it needs to be physically transformed from the String representation:
sizeof(String) // 24 bytes (on 64-bit, anyway)
to the Any representation:
sizeof(Any) // 32 bytes, includes some meta data
// about what the type really is
Since value types are held directly in the array, the array would be a very different shape so under the hood the compiler would have to do the equivalent of this:
names.map { $0 as Any } // create a new array, with the Any versions
Swift could perhaps automate this process for you (it does if you pass a single variable into a function that takes Any). But personally I’m glad it doesn’t, I’d rather this be more explicit – suppose your array was huge, this would be a lot of processing happening implicitly under the hood.
This is different from when you have an array of reference types, all of which are pointers to the actual data and so all the same size, and which need no transformation when upcasting:
class C { }
class D: C { }
let d = D()
let c: C = d
unsafeBitCast(d, UnsafePointer<Void>.self) // these value will
unsafeBitCast(c, UnsafePointer<Void>.self) // be the same
So saying “this array of [D] is really an array of [C]” is just a matter of the compiler agreeing the types can be substituted, no data transformation needs to take place:
// so this works fine,
// no runtime transformation needed:
func f(cs: [C]) { }
let ds = [D(),D()]
f(ds)
But protocols still are different from superclass references when used with classes:
protocol P { }
extension C: P { }
sizeofValue(C()) // 8 bytes (just a pointer)
sizeofValue(C() as P) // 40 bytes
func g(ps: [P]) { }
g(ds) // won’t compile, needs transformation
You need to explicitly put in your let declaration that you
are declaring an Any array and not a specific type array.
With your current declaration, Swift will evaluate it to [String] array.
and not as [Any] array.
To fix your problem just do the following:
let names : [Any] = ["Joffrey", "Cersei", "Mountain", "Hound"]
If you want to retain your let declaration, use AnyObject
in your printItems function and it will be accepted by Swift.
func printItems(items: [AnyObject]){
for item in items {
print(item)
}
}

Array of Arrays as Iterable of Iterables

Let's consider the following definition to add up all elements in a nested Iterable structure of Ints:
def add(xss : Iterable[Iterable[Int]]) : Int = xss.map(_.sum).sum
However, evaluating following expression yields a type error:
scala> add(Array(Array(1,2,3)))
<console>:9: error: type mismatch;
found : Array[Array[Int]]
required: Iterable[Iterable[Int]]
add(Array(Array(1,2,3)))
^
The function works as expected with other Iterables (like Lists). How can I avoid that error? What's the rationale for it? Guess that is something to do with Arrays being native from Java, but don't know the specifics details in this case.
Thanks
It doesn't work because Scala would need to use 2 implicit conversion in a row to go from Array[Array[Int]] to Iterable[Iterable[Int]], which it (purposefully) doesn't do.
You could specify the outer Array's type :
scala> add(Array[Iterable[Int]](Array(1,2,3)))
res4: Int = 6
or transform its elements to Iterable[Int] (thus bypassing an implicit conversion) :
scala> add(Array(Array(1,2,3)).map(_.toIterable))
res5: Int = 6
The problem comes from the fact that Scala's Array[T] is just a representation for Java's T[]. What makes Array[T] behave like an usual Scala collection is an implicit conversion in Predef.
From Array's documentation :
Two implicit conversions exist in scala.Predef that are frequently
applied to arrays: a conversion to mutable.ArrayOps
and a conversion to mutable.WrappedArray (a subtype of scala.collection.Seq). Both types make available many of the standard
operations found in the Scala collections API. The conversion to
ArrayOps is temporary, as all operations defined on ArrayOps return an
Array, while the conversion to WrappedArray is permanent as all
operations return a WrappedArray.
The conversion to ArrayOps takes priority over the conversion to
WrappedArray.
Your intuition is correct. See the Array type signature:
final class Array[T] extends java.io.Serializable with java.lang.Cloneable
contrast that with Seq type signature:
trait Seq[+A] extends Iterable[A] with collection.Seq[A] ...
As you can see Array does not relate to the Iterable[A] trait.
You can fix this by calling toIterable on the instance:
scala> add(Array(Array(1,2,3).toIterable).toIterable)
res1: Int = 6

Why are Arrays invariant, but Lists covariant?

E.g. why does
val list:List[Any] = List[Int](1,2,3)
work, but
val arr:Array[Any] = Array[Int](1,2,3)
fails (because arrays are invariant). What is the desired effect behind this design decision?
Because it would break type-safety otherwise.
If not, you would be able to do something like this:
val arr:Array[Int] = Array[Int](1,2,3)
val arr2:Array[Any] = arr
arr2(0) = 2.54
and the compiler can't catch it.
On the other hand, lists are immutable, so you can't add something that is not Int
This is because lists are immutable and arrays are mutable.
The normal answer to give is that mutability combined with covariance would break type safety. For collections, this can be taken as a fundamental truth. But the theory actually applies to any generic type, not just collections like List and Array, and we don't have to try and reason about mutability at all.
The real answer has to do with the way function types interact with subtyping. The short story is if a type parameter is used as a return type, it is covariant. On the other hand, if a type parameter is used as an argument type, it is contravariant. If it is used both as a return type and as an argument type, it is invariant.
Let's look at the documentation for Array[T]. The two obvious methods to look at are for the ones for lookup and update:
def apply(i: Int): T
def update(i: Int, x: T): Unit
In the first method T is a return type, while in the second T is an argument type. The rules of variance dictate that T must therefore be invariant.
We can compare the documentation for List[A] to see why it is covariant. Confusingly, we would find these methods, which are analogous to the methods for Array[T]:
def apply(n: Int): A
def ::(x: A): List[A]
Since A is used as both a return type and as an argument type, we would expect A to be invariant just like T is for Array[T]. However, unlike with Array[T], the documentation is lying to us about the type of ::. The lie is good enough for most calls to this method, but isn't good enough to decide the variance of A. If we expand the documentation for this method and click on "Full Signature", we are shown the truth:
def ::[B >: A](x: B): List[B]
So A does not actually appear as an argument type. Instead, B (which can be any supertype of A) is the argument type. This does not place any restriction on A, so it really can be covariant. Any method on List[A] which has A as an argument type is a similar lie (we can tell because these methods are marked as [use case]).
The difference is that Lists are immutable while Arrays are mutable.
To understand why mutability determines variance, consider making a mutable version of List - let's call it MutableList. We'll also make use of some example types: a base class Animal and 2 subclasses named Cat and Dog.
trait Animal {
def makeSound: String
}
class Cat extends Animal {
def makeSound = "meow"
def jump = // ...
}
class Dog extends Animal {
def makeSound = "bark"
}
Notice that Cat has one more method (jump) than Dog.
Then, define a function that accepts a mutable list of animals and modifies the list:
def mindlessFunc(xs: MutableList[Animal]) = {
xs += new Dog()
}
Now, horrible things will happen if you pass a list of cats into the function:
val cats = MutableList[Cat](cat1, cat2)
val horror = mindlessFunc(cats)
If we were using a careless programming language, this will be ignored during compilation. Nevertheless, our world will not collapse if we only access the list of cats using the following code:
cats.foreach(c => c.makeSound)
But if we do this:
cats.foreach(c => c.jump)
A runtime error will occur. With Scala, writing such code is prevented, because the compiler will complain.

Resources