Sorting Arrays and Finding Changes - Swift - arrays

Question

Okay, so I have a stream that is receiving an array at a fast speed constantly. Here's what I want to do...
If the array is the same, don't do anything. If it is different, make a new array with nil as every value except the changed ones.
Example:
Incoming array 1: [1,1,1,1]
Incoming array 2: [1,1,2,1]
I want to create: [nil, nil, 2, nil]. Only marking the changes.
I made something that worked, I just don't think it's efficient. Is it the best way to do it?
var storedArray = [Int](count: 10, repeatedValue: 0) //array for comparing
func incomingArray(data: [Int]) {
if data == storedArray {return} //do nothing if its the same
var tempArray = [Int?](count: 10, repeatedValue: nil) //nil array
for index in 0...9 {
if storedArray[index] != data[index] {
tempArray[index] = data[index] //replace the temp array index
}
}
//send the completed tempArray to do work ....
storedArray = incomingArray //save the stored as the current data
}
So the above code works. It's just not efficient. Any better ideas for this?
thanks
UPDATE 1:
I have mistaken in the original post. Instead of Int. They are UInt8.

Answer 1

If you’re concerned about performance, the first thing to look for is hidden loops. Here’s one:
if data == storedArray {return}
This is presumably here for intended efficiency – if the two arrays are equal, don’t bother doing anything. But really, this might be self-defeating. That comparison isn’t constant time – it loops over the elements and compares them. Since you’re going to loop over them later anyway, that probably doesn’t give you much.
You could argue it saves you allocating a new array, but this then leads to the next question which is do you really need to create an array with all those nil values? Why not instead generate an array of the indices into the array that are different? That way, the recipient of your differences will only have to loop over the differences (maybe only a couple) rather than the whole array.
It probably makes sense to factor out the array diffing from the processing and storage. Here’s a function that takes two arrays and returns an array of indices where they differ:
func differences<T: Equatable>(lhs: [T], rhs: [T]) -> [Int] {
// indexedPairs is a sequence of (index, (left-hand val, right-hand val))
let indexedPairs = enumerate(zip(lhs,rhs))
// the lazy may or may not help here, benchmark to find out...
return lazy(indexedPairs).filter { (index, pair) in
// only return different pairs
pair.0 != pair.1
}.map {
// only return the index not the values
$0.0
}.array
}
Note this is a pure function – that is, it takes inputs and produces a result without referencing any external state. This makes it easier to test and debug as a standalone function.
You could then rewrite your original function in terms of it:
func incomingArray(data: [Int]) {
let diffs = differences(storedArray, data)
if !diffs.isEmpty {
// send new data and diff indices for further processing
// then overwrite the old array
storedArray = data
}
}
Update
Benchmarking suggests the filter/map version performs horribly, compared to a simple loop, so here’s a version of differences that just uses for…in:
func differences<T: Equatable>(lhs: [T], rhs: [T]) -> [Int] {
var diffs: [Int] = []
// still using zip, since this guards against the two
// arrays being of different sizes - doesn’t seem to
// impact performance
for (i,(l,r)) in zip(indices(lhs),zip(lhs,rhs)) {
if l != r { diffs.append(i) }
}
return diffs
}
Some quick tests suggests this version gets a significant speedup if the input is large and the # of differences small, but performs identically if the arrays are mostly different.

Answer 2

Here are some ideas to make this code faster:
1) Instead of using an array of Int?, use a plain Int and instead of marking elements as nil, mark them as some special integer value. I don't know what that value is, maybe 0 is fine, or -1, or Int.max.
Update: The above change gives me a ~ 10% performance increase
2) Recycle your result array. So that you can skip the following code:
var tempArray = [Int?](count: 10, repeatedValue: nil)
Or maybe better, let the caller pass it in via an inout parameter so that you don't have to worry about ownership of it.
Update: The above change gives me a ~ 50% performance increase
Here is the code for all the versions suggested in this question:
import UIKit
import XCTest
var storedArray1 = [Int?](count: 10, repeatedValue: 0) //array for comparing
func processIncomingArray1(data: [Int]) {
var tempArray = [Int?](count: 10, repeatedValue: nil) //nil array
for index in 0...9 {
if storedArray1[index] != data[index] {
tempArray[index] = data[index] //replace the temp array index
}
}
storedArray1 = tempArray
}
var storedArray2 = [Int](count: 10, repeatedValue: 0)
func processIncomingArray2(data: [Int]) {
var tempArray = [Int](count: 10, repeatedValue: Int.max)
for index in 0...9 {
if storedArray2[index] != data[index] {
tempArray[index] = data[index]
}
}
storedArray2 = tempArray
}
var storedArray3 = [Int](count: 10, repeatedValue: Int.max)
func processIncomingArray3(data: [Int], inout result: [Int]) {
for index in 0...9 {
if result[index] != data[index] {
result[index] = data[index]
}
}
}
// Given two sequences, return a sequence of 2-tuples (pairs)
public func zip<A: SequenceType, B: SequenceType>(a: A, b: B)
-> ZipSequence<A, B>
{
return ZipSequence(a, b)
}
// Lazy sequence of tuples created from values from two other sequences
public struct ZipSequence<A: SequenceType, B: SequenceType>: SequenceType {
private var a: A
private var b: B
public init (_ a: A, _ b: B) {
self.a = a
self.b = b
}
public func generate() -> ZipGenerator<A.Generator, B.Generator> {
return ZipGenerator(a.generate(), b.generate())
}
}
// Generator that creates tuples of values from two other generators
public struct ZipGenerator<A: GeneratorType, B: GeneratorType>: GeneratorType {
private var a: A
private var b: B
public init(_ a: A, _ b: B) {
self.a = a
self.b = b
}
mutating public func next() -> (A.Element, B.Element)? {
switch (a.next(), b.next()) {
case let (.Some(aValue), .Some(bValue)):
return (aValue, bValue)
default:
return nil
}
}
}
func differences<T: Equatable>(lhs: [T], rhs: [T]) -> [Int] {
// indexedPairs is a sequence of (index, (left-hand val, right-hand val))
let indexedPairs = enumerate(zip(lhs,rhs))
// the lazy may or may not help here, benchmark to find out...
return lazy(indexedPairs).filter { (index, pair) in
// only return different pairs
pair.0 != pair.1
}.map {
// only return the index not the values
$0.0
}.array
}
var storedArray4 = [Int](count: 10, repeatedValue: Int.max)
func processIncomingArray4(data: [Int]) {
let diffs = differences(storedArray4, data)
if !diffs.isEmpty {
// send new data and diff indices for further processing
// then overwrite the old array
storedArray4 = data
}
}
func differences5<T: Equatable>(lhs: [T], rhs: [T]) -> [Int] {
var diffs: [Int] = []
// still using zip, since this guards against the two
// arrays being of different sizes - doesn’t seem to
// impact performance
for (i,(l,r)) in zip(indices(lhs),zip(lhs,rhs)) {
if l != r { diffs.append(i) }
}
return diffs
}
var storedArray5 = [Int](count: 10, repeatedValue: Int.max)
func processIncomingArray5(data: [Int]) {
let diffs = differences5(storedArray4, data)
if !diffs.isEmpty {
// send new data and diff indices for further processing
// then overwrite the old array
storedArray5 = data
}
}
class StackOverflowTests: XCTestCase {
func testPerformanceExample1() {
var data = [1,2,3,4,5,6,7,8,9,10]
self.measureBlock() {
for i in 1...100000 {
processIncomingArray1(data)
}
}
}
func testPerformanceExample2() {
var data = [1,2,3,4,5,6,7,8,9,10]
self.measureBlock() {
for i in 1...100000 {
processIncomingArray2(data)
}
}
}
func testPerformanceExample3() {
var data = [1,2,3,4,5,6,7,8,9,10]
self.measureBlock() {
for i in 1...100000 {
processIncomingArray3(data, &storedArray3)
}
}
}
func testPerformanceExample4() {
var data = [1,2,3,4,5,6,7,8,9,10]
self.measureBlock() {
for i in 1...100000 {
processIncomingArray4(data)
}
}
}
func testPerformanceExample5() {
var data = [1,2,3,4,5,6,7,8,9,10]
self.measureBlock() {
for i in 1...100000 {
processIncomingArray5(data)
}
}
}
}

Answer 3

I think I have the best answer. Instead of the whole empty nil array. I made my temp array a bool value. Then if the value changes, mark its index as true.
So here's a example.
Incoming array 1: [1,1,1,1]
Incoming array 2: [1,1,2,1]
I then export the full array plus a bool array of: [false, false, true, false].
Then I just check if theres a change and pull the value. It turned out to work much faster then the other answers. I also recycled the temp array to speed it up. My guess is that since it's value can only be true or false, it's a lot faster then NIL/UInt8.
Thanks for the help guys. Let me know if any other ideas come up.