I was looking into multi-dimensional Arrays in Scala and came across some easy ways to create multi-dimensional Arrays. Namely:
val my3DimensionalArray = Array.ofDim[Int](3,4,5)
//array with dimensions 3 x 4 x 5
Or even
val myFilledArray = Array.fill[Int](3,4,5)(0)
//Array of same dimension filled with 0's
However this only works for 1 - 5 dimensional Arrays:
val my6DimensionalArray = Array.ofDim[Int](3,3,3,3,3,3) //Error
So how do people usually deal with creating higher dimensinoal Arrays? Is this left to 3rd party libraries to implement, or are there other data structures that Scala encourages us to use instead of high dimensional Arrays?
// create array of 5-dim-array => 6 dim array
Array.tabulate(3)(_ => Array.ofDim[Int](3,3,3,3,3))
Array.ofDim implementation uses this this pattern. see https://github.com/scala/scala/blob/v2.11.6/src/library/scala/Array.scala#L216-234
If you really want arbitrary numbers of dimensions, you typically use a single flat array with a second array that indexes by dimension to get to the element you want. So, for instance,
class MultiArray(dims: Array[Int]) {
private val array = new Array[Double](dims.product)
private def arrayIndex(index: Array[Int]) = {
var i = index.length - 1
var k = index(i)
var n = 1
while (i > 0) {
n *= dims(i)
k += n * index(i-1)
i -= 1
}
k
}
def apply(index: Array[Int]) = array(arrayIndex(index))
def update(index: Array[Int], value: Double) {
array(arrayIndex(index)) = value
}
}
would be a start. There are various mathematics libraries that do this sort of thing (IIRC Apache Commons Math does I can't quickly find a Java mathematics library that does, but ImgLib2 uses similar techniques for image data (they do local chunking also)). It's not really a generally useful thing to do, which is why you tend to find it in maths libraries.
Related
Teaching myself swift, so complete noob here, but I'm far into a project and just know there must be an easier way to achieve something.
I have a 2D array:
var shopArray = [
["theme":"default","price":0,"owned":true,"active":true,"image":UIImage(named: "defaultImage")!,"title":"BUY NOW"],
["theme":"red","price":1000,"owned":false,"active":false,"image":UIImage(named: "redImage")!,"title":"BUY NOW"],
["theme":"blue","price":2000,"owned":false,"active":false,"image":UIImage(named: "blueImage")!,"title":"BUY NOW"],
["theme":"pool","price":3000,"owned":true,"active":false,"image":UIImage(named: "blueImage")!,"title":"BUY NOW"],
["theme":"line","price":4000,"owned":false,"active":false,"image":UIImage(named: "lineImage")!,"title":"BUY NOW"],
["theme":"neon","price":5000,"owned":false,"active":false,"image":UIImage(named: "lineImage")!,"title":"BUY NOW"]]
Where I simply want to create a function that runs and search for all the "owned" keys and make them all "false".
How do you search and replace in Arrays / 2D Arrays. More specifiaclly, what should the func look like?
Thank you!
You don't have a 2D array, you have an Array of Dictionaries.
You can set all of the values for the owned keys by iterating the indices of the Array and updating the values:
shopArray.indices.forEach { shopArray[$0]["owned"] = false }
That is the functional way to do it. You could also do the same operation with a for loop:
for idx in shopArray.indices {
shopArray[idx]["owned"] = false
}
You could do something like this to loopthrough the array replacing the approriate element.
var i = 0
for x in shopArray {
var y = x
y["owned"] = false
shopArray.remove(at: i)
shopArray.insert(y, at: i)
i = i + 1
}
or you could use a while loop to do the same with less code lines.
var y = 0
while y < shopArray.count {
shopArray[y].updateValue(false, forKey: "owned")
y += 1
}
There is proably somthing doable with .contains, but I'm not sure you need that toachive the result you mention above. Play around in a play ground in xcode and try a few different options without doing anything that might cause issues in your project.
I have been porting over an algorithm I've been using in Java (Android) to Swift (iOS), and have run into some issues with speed on the Swift version.
The basic idea is there are objects with depths (comment tree), and I can hide and show replies from the dataset by matching against a list of hidden objects. Below is a visualization
Top
- Reply 1
- - Reply 2
- - Reply 3
- Reply 4
and after hiding from the dataset
Top
- Reply 1
- Reply 4
The relevant methods I've converted from Java are as follows
//Gets the "real" position of the index provided in the "position" variable. The comments array contains all the used data, and the hidden array is an array of strings that represent items in the dataset that should be skipped over.
func getRealPosition(position: Int)-> Int{
let hElements = getHiddenCountUpTo(location: position)
var diff = 0
var i = 0
while i < hElements {
diff += 1
if(comments.count > position + diff && hidden.contains(comments[(position + diff)].getId())){
i -= 1
}
i += 1
}
return position + diff
}
func getHiddenCountUpTo(location: Int) -> Int{
var count = 0
var i = 0
repeat {
if (comments.count > i && hidden.contains(comments[i].getId())) {
count += 1
}
i += 1
} while(i <= location && i < comments.count)
return count
}
This is used with a UITableViewController to display comments as a tree.
In Java, using array.contains was quick enough to not cause any lag, but the Swift version calls the getRealPosition function many times when calling heightForRowAt and when populating the cell, leading to increasing lag as more comment ids are added to the "hidden" array.
Is there any way I can improve on the speed of the array "contains" lookups (possibly with a different type of collection)? I did profiling on the application and "contains" was the method that took up the most time.
Thank you
Both Java and Swift have to go through all elements contained in the array. This gets slower and slower as the array gets larger.
There is no a priori reason for Java to fare better, as they both use the exact same algorithm. However, strings are implemented very differently in each language, so that could make string comparisons more expensive in Swift.
In any case, if string comparison slows you down, then you must avoid it.
Easy fix: use a Set
If you want a simple performance boost, you can replace an array of strings with a set of strings. A set in Swift is implemented with a hash table, meaning that you have expected constant time query. In practice, this means that for large sets, you will see better performance.
var hiddenset Set<String> = {}
for item in hidden {
strset.insert(item)
}
For best performance: use a BitSet
But you should be able to do a whole lot better than even a set can do. Let us look at your code
hidden.contains(comments[i].getId()))
If you are always accessing hidden in this manner, then it means that what you have is a map from integers (i) to Boolean values (true or false).
Then you should do the following...
import Bitset;
let hidden = Bitset ();
// replace hidden.append(comments[i].getId())) by this:
hidden.add(i)
// replace hidden.contains(comments[i].getId())) by this:
hidden.contains(i)
Then your code will really fly!
To use a fast BitSet implementation in Swift, include the following in Package.swift (it is free software):
import PackageDescription
let package = Package(
name: "fun",
dependencies: [
.Package(url: "https://github.com/lemire/SwiftBitset.git", majorVersion: 0)
]
)
i think you need the realPosition to link from a tap on a row in the tableview to the source array?
1) make a second array with data only for the tableViewDataSource
copy all visible elements to this new array. create a special ViewModel as class or better struct which only has the nessesary data to display in the tableview. save in this new ViewModel the realdataposition also as value. now you have a backlink to the source array
2) then populate this TableView only from the new datasource
3) look more into the functional programming in swift - there you can nicer go over arrays for example:
var array1 = ["a", "b", "c", "d", "e"]
let array2 = ["a", "c", "d"]
array1 = array1.filter { !array2.contains($0) }
or in your case:
let newArray = comments.filter{ !hidden.contains($0.getId()) }
or enumerated to create the viewmodel
struct CommentViewModel {
var id: Int
var text: String
var realPosition: Int
}
let visibleComments: [CommentViewModel] = comments
.enumerated()
.map { (index, element) in
return CommentViewModel(id: element.getId(), text: element.getText(), realPosition: index)
}
.filter{ !hidden.contains($0.id) }
I want to create a 2D list that can have elements of variable lengths inside, for example, if I have a 10x10 list in MATLAB, I can
define it with:
z = cell(10,10)
and start assigning some elements by doing this:
z{2}{3} = ones(3,1)
z{1}{1} = zeros(100,1)
z{1}{2} = []
z{1}{3} = randn(20,1)
...
What is the optimal way to define such empty 2D list in torch? Moreover, is there a way to exploit the tensor structure to do this?
In python, I can do something along this to define an empty 10x10 2D list:
z = [[None for j in range(10)] for i in range(10)]
My best guess for torch is doing something like
z = torch.Tensor(10,10)
for i=1,10 do
for j=1,10 do
z[{{i},{j}}] = torch.Tensor()
end
end
but, this does not work, and defining a tensor inside a tensor seems like a bad idea ...
This is a follow up to the question asked here (however in the link it is asked in python): Create 2D lists in python with variable length indexed vectors
From the documentation I've read, tensors only support primitive numeric data types. You won't be able to use tensor for your intended usage. Leverage tables.
local function makeMatrix(initialVal, ...)
local isfunc = type(initialVal) == "function"
local dimtable = {...}
local function helper(depth)
if depth == 0 then
return isfunc and initialVal() or initialVal
else
local plane = {}
for i = 1, dimtable[depth] do
plane[i] = helper(depth-1)
end
return plane
end
end
return helper(#dimtable)
end
p = makeMatrix(0, 2, 3, 5) -- makes 3D matrix of size 2x3x5 with all elements initialized to 0
makeMatrix(torch.Tensor, m ,n)
Answer from Torch's Google Group forums. Agreeing that tables is the solution:
z = {}
for i=1,10 do
z[i] = {}
for j=1,10 do
z[i][j] = torch.Tensor()
end
end
What would be a most efficient method of merging two neighboring indicies in Scala? What I have in mind a nasty while loops with copying.
For example, there's a buffer array A, with length N. The new array need be generated such that for A(i) = A(i) + A(i+1), where i < N
For example, merging and summing the second and third element, and generate a new array.
ArrayBuffer(1,2,4,3) => ArrayBuffer(1,6,3)
UPDATE:
I think I come up with some solution, but doesn't like it much. Any suggestion to improve would be highly appreciated.
scala> val i = 1
i: Int = 1
scala> ArrayBuffer(1,2,4,3).zipWithIndex.foldLeft(ArrayBuffer[Int]())( (k,v)=> if(v._2==i+1){ k(k.length-1) =(k.last+v._1);k; }else k+= v._1 )
The simplest way to get neighbors is to use sliding method.
a.sliding(2, 1).map(_.sum)
where the first argument is a size and the second one is step.
If you want to keep the first and the last element intact something like this should work:
a.head +: a.drop(1).dropRight(1).sliding(2, 1).map(_.sum).toArray :+ a.last
If you want to avoid copying and array on append/prepend you can rewrite it as follows:
val aa = a.sliding(2, 1).map(_.sum).toArray
aa(0) = a.head
aa(aa.size - 1) = a
or use ListBuffer which provides constant time prepend and append.
It should be also possible to use Iterators:
val middle: Iterator[Int] = a.drop(1).dropRight(1).sliding(2, 1).map(_.sum)
(Iterator(a.head) ++ middle ++ Iterator(a.last)).toArray // or toBuffer
Can anybody optimize following statement in Scala:
// maybe large
val someArray = Array(9, 1, 6, 2, 1, 9, 4, 5, 1, 6, 5, 0, 6)
// output a sorted list which contains unique element from the array without 0
val newList=(someArray filter (_>0)).toList.distinct.sort((e1, e2) => (e1 > e2))
Since the performance is critical, is there a better way?
Thank you.
This simple line is one of the fastest codes so far:
someArray.toList.filter (_ > 0).sortWith (_ > _).distinct
but the clear winner so far is - due to my measurement - Jed Wesley-Smith. Maybe if Rex' code is fixed, it looks different.
Typical disclaimer 1 + 2:
I modified the codes to accept an Array and return an List.
Typical benchmark considerations:
This was random data, equally distributed. For 1 Million elements, I created an Array of 1 Million ints between 0 and 1 Million. So with more or less zeros, and more or less duplicates, it might vary.
It might depend on the machine etc.. I used a single core CPU, Intel-Linux-32bit, jdk-1.6, scala 2.9.0.1
Here is the underlying benchcoat-code and the concrete code to produce the graph (gnuplot). Y-axis: time in seconds. X-axis: 100 000 to 1 000 000 elements in Array.
update:
After finding the problem with Rex' code, his code is as fast as Jed's code, but the last operation is a transformation of his Array to a List (to fullfill my benchmark-interface). Using a var result = List [Int], and result = someArray (i) :: result speeds his code up, so that it is about twice as fast as the Jed-Code.
Another, maybe interesting, finding is: If I rearrange my code in the order of filter/sort/distinct (fsd) => (dsf, dfs, fsd, ...), all 6 possibilities don't differ significantly.
I haven't measured, but I'm with Duncan, sort in place then use something like:
util.Sorting.quickSort(array)
array.foldRight(List.empty[Int]){
case (a, b) =>
if (!b.isEmpty && b(0) == a)
b
else
a :: b
}
In theory this should be pretty efficient.
Without benchmarking I can't be sure, but I imagine the following is pretty efficient:
val list = collection.SortedSet(someArray.filter(_>0) :_*).toList
Also try adding .par after someArray in your version. It's not guaranteed to be quicker, bit it might be. You should run a benchmark and experiment.
sort is deprecated. Use .sortWith(_ > _) instead.
Boxing primitives is going to give you a 10-30x performance penalty. Therefore if you really are performance limited, you're going to want to work off of raw primitive arrays:
def arrayDistinctInts(someArray: Array[Int]) = {
java.util.Arrays.sort(someArray)
var overzero = 0
var ndiff = 0
var last = 0
var i = 0
while (i < someArray.length) {
if (someArray(i)<=0) overzero = i+1
else if (someArray(i)>last) {
last = someArray(i)
ndiff += 1
}
i += 1
}
val result = new Array[Int](ndiff)
var j = 0
i = overzero
last = 0
while (i < someArray.length) {
if (someArray(i) > last) {
result(j) = someArray(i)
last = someArray(i)
j += 1
}
i += 1
}
result
}
You can get slightly better than this if you're careful (and be warned, I typed this off the top of my head; I might have typoed something, but this is the style to use), but if you find the existing version too slow, this should be at least 5x faster and possibly a lot more.
Edit (in addition to fixing up the previous code so it actually works):
If you insist on ending with a list, then you can build the list as you go. You could do this recursively, but I don't think in this case it's any clearer than the iterative version, so:
def listDistinctInts(someArray: Array[Int]): List[Int] = {
if (someArray.length == 0 || someArray(someArray.length-1) <= 0) List[Int]()
else {
java.util.Arrays.sort(someArray)
var last = someArray(someArray.length-1)
var list = last :: Nil
var i = someArray.length-2
while (i >= 0) {
if (someArray(i) < last) {
last = someArray(i)
if (last <= 0) return list;
list = last :: list
}
i -= 1
}
list
}
}
Also, if you may not destroy the original array by sorting, you are by far best off if you duplicate the array and destroy the copy (array copies of primitives are really fast).
And keep in mind that there are special-case solutions that are far faster yet depending on the nature of the data. For example, if you know that you have a long array, but the numbers will be in a small range (e.g. -100 to 100), then you can use a bitset to track which ones you've encountered.
For efficiency, depending on your value of large:
val a = someArray.toSet.filter(_>0).toArray
java.util.Arrays.sort(a) // quicksort, mutable data structures bad :-)
res15: Array[Int] = Array(1, 2, 4, 5, 6, 9)
Note that this does the sort using qsort on an unboxed array.
I'm not in a position to measure, but some more suggestions...
Sorting the array in place before converting to a list might well be more efficient, and you might look at removing dups from the sorted list manually, as they will be grouped together. The cost of removing 0's before or after the sort will also depend on their ratio to the other entries.
How about adding everything to a sorted set?
val a = scala.collection.immutable.SortedSet(someArray filter (0 !=): _*)
Of course, you should benchmark the code to check what is faster, and, more importantly, that this is truly a hot spot.