This question already has answers here:
Processing vec in parallel: how to do safely, or without using unstable features?
(2 answers)
Closed 7 years ago.
I'm novice to Rust (v1.0.0) and thread-programming.
I try to calculate elements of b-array using a-array. Each element of the b-array can be calculated independently of the others (parallel).
extern crate rand;
use rand::Rng;
use std::io;
use std::thread;
use std::sync::{Arc, Mutex};
fn main() {
let mut a : [u32; 10] = [0; 10];
let mut b = Arc::new(Mutex::new([0; 10]));
let mut rng = rand::thread_rng();
for x in 0..9 {
a[x] = (rng.gen::<u32>() % 1000) + 1;
};
for x in 0..4 {
let b = b.clone();
thread::spawn(move || { let mut b = b.lock().unwrap();
for y in 0..4 {
b[x] += a[y] * a[y*2+1];
b[x+5] += a[y+1] * a[y*2];
}
});
};
thread::sleep_ms(1000);
for x in 0..a.len() {
println!("a({0})={1}, b({0})={2}", x, a[x], b[x]);
};
}
Can you help me:
if I use: let mut b = Arc::new(Mutex::new([u32; 10] = [0; 10])); -> I get error unresolved name 'u32'. Did you mean 'a'? How can I set the type of array element ?
thread::sleep_ms(1000) - It is so rudely. How can I check that all thread is finished?
How can I get back my calculated b[i] and/or gather thread-calculated b-arrays in the final one ? Now I've got error: cannot index a value of type 'alloc::arc::Arc<std::sync::mutex::Mutex<[u32; 10]>>'
Can I use only one b-array in memory and send into thread (using pointers) to calculating two elements of b-array?
Thank for solutions.
Working code is (I've modified it for show problem):
extern crate rand;
use rand::Rng;
use std::thread;
use std::sync::{Arc, Mutex};
fn main() {
let mut a : [u32; 10000] = [0; 10000];
let b = Arc::new(Mutex::new([0u32; 10]));
let mut rng = rand::thread_rng();
for x in 0..10000 {
a[x] = (rng.gen::<u32>() % 10) + 1;
};
for x in 0..5 {
let b = b.clone();
thread::spawn(move || { let mut b = b.lock().unwrap();
println!("thread {} started", x);
for y in 0..5000 {
b[x] += a[y] * a[y*2+1];
b[x+5] += a[y+1] * a[y*2];
};
b[x] += a[x];
b[x+5] -= a[x];
println!("thread {} finished", x);
});
};
thread::sleep_ms(1000);
for x in 0..10 {
println!("b({0})={1}", x, b.lock().unwrap()[x]);
};
}
The output is:
thread 1 started
thread 1 finished
thread 3 started
thread 3 finished
thread 0 started
thread 0 finished
thread 2 started
thread 2 finished
thread 4 started
thread 4 finished
b(0)=149482
...
b(9)=149065
Threads are processed step-by-step.
Note that the clone() method on the Arc object does not "clone" the array, simply it increments the reference counter of the Arc.
I think you are asking for a general strategy to process data in parallel in Rust. Your code lock the b array in each thread, so you have no parallel processing.
To do real parallel processing you would need a mutable access to the array without a lock on the entire array but you cannot do that in safe Rust.
To do that you have to use some sort of unsafe mechanism, such raw pointers.
This is a simple example to process a (non mutable) input vector into a (mutable) output vector concurrently:
use std::thread;
use std::sync::Arc;
fn main() {
let input = Arc::new([1u32, 2, 3, 4]);
let output = Arc::new([0; 4]);
let mut handles = Vec::new();
for t in 0..4 {
let inp = input.clone();
let out = output.clone();
let handle = thread::spawn(move || unsafe {
let p = (out.as_ptr() as *mut u32).offset(t as isize);
*p = inp[t] + (t as u32 + 1);
});
handles.push(handle);
}
for h in handles {
h.join().unwrap();
}
println!("{:?}", output);
}
You still need to use Arc to pass data into the threads and to have a proper lifetime management.
Then inside the thread you need to get a mutable pointer to the data (out.as_ptr() as *mut u32), then the item processed in that thread using the offset method.
Related
I'm trying to return an array iterator of an array in an Option from the closure passed to a filter_map so I can flatten it afterwards.
Unfortunately, rustc produces the following error:
cannot return value referencing local variable `res`
returns a value referencing data owned by the current function
main.rs(3, 5): returns a value referencing data owned by the current function
main.rs(3, 10): `res` is borrowed here
for the minimal example:
fn demo<'a>() -> Option<impl Iterator + 'a> {
let res = [1,2];
Some(res.into_iter())
}
Though the complete code I'm trying to make work is this:
fn generate_next<'a>(prev: &'a [u32]) -> impl Iterator + 'a {
let mut counter = 1_u32;
prev.windows(2).filter_map(move |window| {
if window[0] == window[1] {
counter+=1;
None
} else {
let res = [counter, window[0]];
counter=1;
Some(res.into_iter())
}
}).flatten()
}
Both produce the same error for the Some(...) part.
If I understand correctly, the code should work, because the into_iter() method consumes the array and produces an iterator from it. Some should then take ownership of the iterator via a move. Why does rustc think that I'm borrowing res here?
I'm also open to other ways to implement the generate_next function.
Calling into_iter() on an array produces the same result as calling iter(), i.e. an iterator over references. This is an unfortunate gotcha within the Rust standard library.
You can accomplish what you want by using std::iter::once to create iterators of counter and window[0] and then chain them together:
fn generate_next<'a>(prev: &'a [u32]) -> impl Iterator + 'a {
let mut counter = 1_u32;
prev.windows(2)
.filter_map(move |window| {
if window[0] == window[1] {
counter += 1;
None
} else {
let counter_iter = std::iter::once(counter);
let window_iter = std::iter::once(window[0]);
counter = 1;
Some(counter_iter.chain(window_iter))
}
})
.flatten()
}
playground
This question is motivated by this CodinGame puzzle.
I am implementing a basic pathfinding algorithm using Dijkstra's method. It uses a boundary HashMap and a finished HashMap to hold pathfinding-related node info. In a particular loop, I find the highest-valued node in boundary, remove the node, add the node to finished, and add/update the node's neighbors' info in boundary.
Attempting to mutate boundary while looping over it is making Rust's borrow checker queasy, but the logic of the loop seems sound to me. How do I rewrite it so that the compiler shares my confidence? (Or fix the errors I'm missing, if that's the issue.)
Code:
On Rust Playground here
use std::io;
use std::collections::{HashSet, HashMap};
use std::cmp::Ordering;
use std::cell::RefCell;
struct NodeInfo {
nbrs: HashSet<i32>,
gwlinks: i32,
}
#[derive(PartialEq,PartialOrd)]
struct PFInfo {
avg: f32,
cum: i32,
dist: i32,
prev: Option<i32>,
}
impl Eq for PFInfo {}
impl Ord for PFInfo {
fn cmp(&self, other: &PFInfo) -> Ordering {
match self.partial_cmp(other) {
Some(ord) => ord,
None => Ordering::Equal
}
}
}
type Graph = HashMap<i32, RefCell<NodeInfo>>;
type PFGraph = HashMap<i32, PFInfo>;
// Find the path that passes the most gateway links per distance traveled,
// starting at a given node. This is meant to simulate the behavior of an
// "agent" which traverses the graph in the puzzle mentioned above.
fn generate_path(si: &i32, graph: &Graph) -> Vec<i32> {
let n = graph.len();
let mut boundary = PFGraph::with_capacity(n);
let mut finished = PFGraph::with_capacity(n);
boundary.insert( si.clone(),
PFInfo {
avg: 0.,
cum: graph.get(&si).unwrap().borrow().gwlinks,
dist: 0,
prev: None } );
// Keep grabbing the key corresponding the highest value until `boundary` is
// empty
while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1) {
// Move the node from `boundary` to `finished`
let val = boundary.remove(&currid).unwrap();
finished.insert(currid.clone(), val);
// Add or update all adjacent nodes that are not in `finished`
for nbrid in graph.get(&currid).unwrap()
.borrow()
.nbrs.iter()
.filter(|x| !finished.contains_key(x)) {
let currval = finished.get(&currid).unwrap();
let prev = Some(currid.clone());
let dist = currval.dist + 1;
let cum = currval.cum + graph.get(nbrid).unwrap().borrow().gwlinks;
let avg = cum as f32 / dist as f32;
boundary.insert(
nbrid.clone(),
PFInfo {
avg: avg,
cum: cum,
dist: dist,
prev: prev,
}
);
}
}
let mut path = Vec::new();
let mut currid = finished.iter().max_by_key(|x| x.1).unwrap().0.clone();
path.push(currid.clone());
while let Some(previd) = finished.get(&currid).unwrap().prev {
path.push(previd.clone());
currid = previd.clone();
}
path.reverse();
path
}
macro_rules! parse_input {
($x:expr, $t:ident) => ($x.trim().parse::<$t>().unwrap())
}
#[test]
fn test_generate_path() {
let mut inputs = "8 13 2
6 2
7 3
6 3
5 3
3 4
7 1
2 0
0 1
0 3
1 3
2 3
7 4
6 5
4
5".lines();
let header = inputs.next().unwrap().split_whitespace().collect::<Vec<_>>();
let n = parse_input!(header[0], i32); // the total number of nodes in the level, including the gateways
let l = parse_input!(header[1], i32); // the number of links
let e = parse_input!(header[2], i32); // the number of exit gateways
let mut graph = Graph::with_capacity(n as usize);
for node in 0..n {
graph.insert(node, RefCell::new(NodeInfo{ nbrs: HashSet::new(), gwlinks: 0 }));
}
let graph = graph;
for _ in 0..l as usize {
let link = inputs.next().unwrap();
let nodes = link.split(" ").collect::<Vec<_>>();
let n1 = parse_input!(nodes[0], i32); // N1 and N2 defines a link between these nodes
let n2 = parse_input!(nodes[1], i32);
graph.get(&n1).unwrap().borrow_mut().nbrs.insert(n2);
graph.get(&n2).unwrap().borrow_mut().nbrs.insert(n1);
}
let mut gateways = HashSet::new();
for _ in 0..e as usize {
let ei = parse_input!(inputs.next().unwrap(), i32); // the index of a gateway node
gateways.insert(ei);
}
let gateways = gateways;
for gwid in &gateways {
for gwnbr in &graph.get(gwid).unwrap().borrow().nbrs {
(&graph).get(&gwnbr).unwrap().borrow_mut().gwlinks += 1;
}
}
assert_eq!(generate_path(&0, &graph), vec![0, 3]);
}
Errors:
rustc 1.18.0 (03fc9d622 2017-06-06)
error[E0502]: cannot borrow `boundary` as mutable because it is also borrowed as immutable
--> <anon>:53:19
|
50 | while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1) {
| -------- immutable borrow occurs here
...
53 | let val = boundary.remove(&currid).unwrap();
| ^^^^^^^^ mutable borrow occurs here
...
76 | }
| - immutable borrow ends here
error[E0502]: cannot borrow `boundary` as mutable because it is also borrowed as immutable
--> <anon>:66:13
|
50 | while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1) {
| -------- immutable borrow occurs here
...
66 | boundary.insert(
| ^^^^^^^^ mutable borrow occurs here
...
76 | }
| - immutable borrow ends here
error: aborting due to 2 previous errors
I found a solution to my issue, and it's somewhat generalizable, which is what I was hoping for. The problem was that an implicit reference created in the while let statement was living to the end of the loop even though it was only needed on that one line. The borrow begins at .iter() and is no longer needed once the referenced value is cloned at the end of the expression.
while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1).clone() {
// ^---where borrow begins ^---where borrow could end
// Move the node from `boundary` to `finished`
let val = boundary.remove(&currid).unwrap();
finished.insert(currid.clone(), val);
...
} // <--- where borrow does end
The trick was moving the binding of currid into the loop. When the value is borrowed in the while let statement, the borrow checker apparently thinks the borrow needs to last throughout the loop. If, instead, the implicit borrow is made in a regular let binding, the borrow checker is smart enough to realize the borrow can be safely discarded at the end of the line.
while !boundary.is_empty() {
let currid = boundary.iter().max_by_key(|x| x.1).unwrap().0.clone();
// ^---where borrow begins ^---where borrow ends
// Move the node from `boundary` to `finished`
let val = boundary.remove(&currid).unwrap();
finished.insert(currid.clone(), val);
...
}
I guess the take away here is that if you need to mutate a structure in a loop that depends on it, put any borrows of the structure inside the loop and keep those borrows as short as possible – for example, by using clone.
This might be one of the situations eventually mitigated by the proposed non-lexical lifetimes.
I've actually googled this extensively, within stackoverflow and elsewhere.
Most questions are about [UInt8] to String or [UInt8] to type_a (not array).
To clarify, I'd like to take an array of type_a. Get its pointer and tell swift to treat the next n iterations of type_b (size_of) as array of type_b.
I've tried variations of https://stackoverflow.com/a/26954091/5276890 which didn't work. A comment there led me to https://stackoverflow.com/a/42255468/5276890.
withMemoryRebound seems like the right way but I couldn't find the right invocation.
Here's a sample code of what I'm doing instead to convert [UInt8] to [UInt32.bigEndian], both to clarify and in case it's useful (not likely)
var intData = [UInt32]()
let M = UInt32(256*256*256)
var m = M
var bigE:UInt32 = 0
for i in 0..<data.count {
bigE += UInt32(data[i]) * m
if m == 1 {
intData.append(bigE)
bigE = 0
m = M
} else {
m = m/256
}
}
<disclaimer+rant>
I have to admit I never could figure out the whole closures+withUnsafe* syntax and mostly used patterns online and modified them. I'd spend the time learning this, just as soon as the language authors decide and settle down on one specific syntax :(
</disclaimer+rant>
Use withUnsafeBufferPointer to get a pointer to the element
storage of the source array.
Use withMemoryRebound to "reinterpret" that pointer as pointing
to elements of the target type.
Use Array(UnsafeBufferPointer(...) to create an array of the
target type.
Example:
let source: [UInt16] = [1, 2, 3, 4]
let dest = source.withUnsafeBufferPointer {
$0.baseAddress!.withMemoryRebound(to: UInt32.self, capacity: 2) {
Array(UnsafeBufferPointer(start: $0, count: 2))
}
}
print(dest) // [131073, 262147]
Or as a generic function:
func convertArray<S, T>(_ source: [S], to: T.Type) -> [T] {
let count = source.count * MemoryLayout<S>.stride/MemoryLayout<T>.stride
return source.withUnsafeBufferPointer {
$0.baseAddress!.withMemoryRebound(to: T.self, capacity: count) {
Array(UnsafeBufferPointer(start: $0, count: count))
}
}
}
Example:
let source: [UInt16] = [1, 2, 3, 4]
let dest = convertArray(source, to: UInt32.self)
print(dest) // [131073, 262147]
If you only need a (temporary) view on the array storage interpreted
in another type then you can avoid the Array creation
and use the UnsafeBufferPointer (which is a Collection and
has array-like methods) without copying the data:
source.withUnsafeBufferPointer {
$0.baseAddress!.withMemoryRebound(to: UInt32.self, capacity: 2) {
let u32bufptr = UnsafeBufferPointer(start: $0, count: 2)
// ... Operate on u32bufptr ...
for elem in u32bufptr { print(elem) }
}
}
fn change(a: &mut i32, b: &mut i32) {
let c = *a;
*a = *b;
*b = c;
}
fn main() {
let mut v = vec![1, 2, 3];
change(&mut v[0], &mut v[1]);
}
When I compile the code above, it has the error:
error[E0499]: cannot borrow `v` as mutable more than once at a time
--> src/main.rs:9:32
|
9 | change(&mut v[0], &mut v[1]);
| - ^ - first borrow ends here
| | |
| | second mutable borrow occurs here
| first mutable borrow occurs here
Why does the compiler prohibit it? v[0] and v[1] occupy different memory positions, so it's not dangerous to use these together. And what should I do if I come across this problem?
You can solve this with split_at_mut():
let mut v = vec![1, 2, 3];
let (a, b) = v.split_at_mut(1); // Returns (&mut [1], &mut [2, 3])
change(&mut a[0], &mut b[0]);
There are uncountably many safe things to do that the compiler unfortunately does not recognize yet. split_at_mut() is just like that, a safe abstraction implemented with an unsafe block internally.
We can do that too, for this problem. The following is something I use in code where I need to separate all three cases anyway (I: Index out of bounds, II: Indices equal, III: Separate indices).
enum Pair<T> {
Both(T, T),
One(T),
None,
}
fn index_twice<T>(slc: &mut [T], a: usize, b: usize) -> Pair<&mut T> {
if a == b {
slc.get_mut(a).map_or(Pair::None, Pair::One)
} else {
if a >= slc.len() || b >= slc.len() {
Pair::None
} else {
// safe because a, b are in bounds and distinct
unsafe {
let ar = &mut *(slc.get_unchecked_mut(a) as *mut _);
let br = &mut *(slc.get_unchecked_mut(b) as *mut _);
Pair::Both(ar, br)
}
}
}
}
Since Rust 1.26, pattern matching can be done on slices. You can use that as long as you don't have huge indices and your indices are known at compile-time.
fn change(a: &mut i32, b: &mut i32) {
let c = *a;
*a = *b;
*b = c;
}
fn main() {
let mut arr = [5, 6, 7, 8];
{
let [ref mut a, _, ref mut b, ..] = arr;
change(a, b);
}
assert_eq!(arr, [7, 6, 5, 8]);
}
The borrow rules of Rust need to be checked at compilation time, that is why something like mutably borrowing a part of a Vec is a very hard problem to solve (if not impossible), and why it is not possible with Rust.
Thus, when you do something like &mut v[i], it will mutably borrow the entire vector.
Imagine I did something like
let guard = something(&mut v[i]);
do_something_else(&mut v[j]);
guard.do_job();
Here, I create an object guard that internally stores a mutable reference to v[i], and will do something with it when I call do_job().
In the meantime, I did something that changed v[j]. guard holds a mutable reference that is supposed to guarantee nothing else can modify v[i]. In this case, all is good, as long as i is different from j; if the two values are equal it is a huge violation of the borrow rules.
As the compiler cannot guarantee that i != j, it is thus forbidden.
This was a simple example, but similar cases are legions, and are why such access mutably borrows the whole container. Plus the fact that the compiler actually does not know enough about the internals of Vec to ensure that this operation is safe even if i != j.
In your precise case, you can have a look at the swap(..) method available on Vec that does the swap you are manually implementing.
On a more generic case, you'll probably need an other container. Possibilities are wrapping all the values of your Vec into a type with interior mutability, such as Cell or RefCell, or even using a completely different container, as #llogiq suggested in his answer with par-vec.
The method [T]::iter_mut() returns an iterator that can yield a mutable reference for each element in the slice. Other collections have an iter_mut method too. These methods often encapsulate unsafe code, but their interface is totally safe.
Here's a general purpose extension trait that adds a method on slices that returns mutable references to two distinct items by index:
pub trait SliceExt {
type Item;
fn get_two_mut(&mut self, index0: usize, index1: usize) -> (&mut Self::Item, &mut Self::Item);
}
impl<T> SliceExt for [T] {
type Item = T;
fn get_two_mut(&mut self, index0: usize, index1: usize) -> (&mut Self::Item, &mut Self::Item) {
match index0.cmp(&index1) {
Ordering::Less => {
let mut iter = self.iter_mut();
let item0 = iter.nth(index0).unwrap();
let item1 = iter.nth(index1 - index0 - 1).unwrap();
(item0, item1)
}
Ordering::Equal => panic!("[T]::get_two_mut(): received same index twice ({})", index0),
Ordering::Greater => {
let mut iter = self.iter_mut();
let item1 = iter.nth(index1).unwrap();
let item0 = iter.nth(index0 - index1 - 1).unwrap();
(item0, item1)
}
}
}
}
On recent nightlies, there is get_many_mut():
#![feature(get_many_mut)]
fn main() {
let mut v = vec![1, 2, 3];
let [a, b] = v
.get_many_mut([0, 1])
.expect("out of bounds or overlapping indices");
change(a, b);
}
Building up on the answer by #bluss you can use split_at_mut() to create a function that can turn mutable borrow of a vector into a vector of mutable borrows of vector elements:
fn borrow_mut_elementwise<'a, T>(v:&'a mut Vec<T>) -> Vec<&'a mut T> {
let mut result:Vec<&mut T> = Vec::new();
let mut current: &mut [T];
let mut rest = &mut v[..];
while rest.len() > 0 {
(current, rest) = rest.split_at_mut(1);
result.push(&mut current[0]);
}
result
}
Then you can use it to get a binding that lets you mutate many items of original Vec at once, even while you are iterating over them (if you access them by index in your loop, not through any iterator):
let mut items = vec![1,2,3];
let mut items_mut = borrow_mut_elementwise(&mut items);
for i in 1..items_mut.len() {
*items_mut[i-1] = *items_mut[i];
}
println!("{:?}", items); // [2, 3, 3]
The problem is that &mut v[…] first mutably borrows v and then gives the mutable reference to the element to the change-function.
This reddit comment has a solution to your problem.
Edit: Thanks for the heads-up, Shepmaster. par-vec is a library that allows to mutably borrow disjunct partitions of a vec.
I publish my daily utils for this to crate.io. Link to the doc.
You may use it like
use arref::array_mut_ref;
let mut arr = vec![1, 2, 3, 4];
let (a, b) = array_mut_ref!(&mut arr, [1, 2]);
assert_eq!(*a, 2);
assert_eq!(*b, 3);
let (a, b, c) = array_mut_ref!(&mut arr, [1, 2, 0]);
assert_eq!(*c, 1);
// ⚠️ The following code will panic. Because we borrow the same element twice.
// let (a, b) = array_mut_ref!(&mut arr, [1, 1]);
It's a simple wrapper around the following code, which is sound. But it requires that the two indexes are different at runtime.
pub fn array_mut_ref<T>(arr: &mut [T], a0: usize, a1: usize) -> (&mut T, &mut T) {
assert!(a0 != a1);
// SAFETY: this is safe because we know a0 != a1
unsafe {
(
&mut *(&mut arr[a0] as *mut _),
&mut *(&mut arr[a1] as *mut _),
)
}
}
Alternatively, you may use a method that won't panic with mut_twice
#[inline]
pub fn mut_twice<T>(arr: &mut [T], a0: usize, a1: usize) -> Result<(&mut T, &mut T), &mut T> {
if a0 == a1 {
Err(&mut arr[a0])
} else {
unsafe {
Ok((
&mut *(&mut arr[a0] as *mut _),
&mut *(&mut arr[a1] as *mut _),
))
}
}
}
I've been manipulating byte arrays in Swift 2.1 lately, and I often find myself writing code like this:
// code to add functions to a [UInt8] object
extension CollectionType where Generator.Element == UInt8 {
func xor(with byte: UInt8) -> [UInt8] {
return map { $0 ^ byte }
}
}
// example usage: [67, 108].xor(with: 0) == [67, 108]
Is there an easy way to parallelize this map call, so that multiple threads can operate on non-overlapping areas of the array at the same time?
I could write code to manually divide the array into sub-arrays and call map on each sub-array in distinct threads.
But I wonder if some framework exists in Swift to do the division automatically, since map is a functional call that can work in a thread-safe environment without side-effects.
Clarifying notes:
The code only needs to work on a [UInt8] object, not necessarily every CollectionType.
The easiest way to perform a loop of calculations in parallel is concurrentPerform (previously called dispatch_apply; see Performing Loop Iterations Concurrently in the Concurrency Programming Guide). But, no, there is no map rendition that will do this for you. You have to do this yourself.
For example, you could write an extension to perform the concurrent tasks:
extension Array {
public func concurrentMap<T>(_ transform: (Element) -> T) -> [T] {
var results = [Int: T](minimumCapacity: count)
let lock = NSLock()
DispatchQueue.concurrentPerform(iterations: count) { index in
let result = transform(self[index])
lock.synchronized {
results[index] = result
}
}
return (0 ..< results.count).compactMap { results[$0] }
}
}
Where
extension NSLocking {
func synchronized<T>(block: () throws -> T) rethrows -> T {
lock()
defer { unlock() }
return try block()
}
}
You can use whatever synchronization mechanism you want (locks, serial queues, reader-writer), but the idea is to perform transform concurrently and then synchronize the update of the collection.
Note:
This will block the thread you call it from (just like the non-concurrent map will), so make sure to dispatch this to a background queue.
One needs to ensure that there is enough work on each thread to justify the inherent overhead of managing all of these threads. (E.g. a simple xor call per loop is not sufficient, and you'll find that it's actually slower than the non-concurrent rendition.) In these cases, make sure you stride (see Improving Loop Code that balances the amount of work per concurrent block). For example, rather than doing 5000 iterations of one extremely simple operation, do 10 iterations of 500 operations per loop. You may have to experiment with suitable striding values.
While I suspect you don't need this discussion, for readers unfamiliar with concurrentPerform (formerly known as dispatch_apply), I'll illustrate its use below. For a more complete discussion on the topic, refer to the links above.
For example, let's consider something far more complicated than a simple xor (because with something that simple, the overhead outweighs any performance gained), such as a naive Fibonacci implementation:
func fibonacci(_ n: Int) -> Int {
if n == 0 || n == 1 {
return n
}
return fibonacci(n - 1) + fibonacci(n - 2)
}
If you had an array of Int values for which you wanted to calculate, rather than:
let results = array.map { fibonacci($0) }
You could:
var results = [Int](count: array.count, repeatedValue: 0)
DispatchQueue.concurrentPerform(iterations: array.count) { index in
let result = self.fibonacci(array[index])
synchronize.update { results[index] = result } // use whatever synchronization mechanism you want
}
Or, if you want a functional rendition, you can use that extension I defined above:
let results = array.concurrentMap { fibonacci($0) }
For Swift 2 rendition, see previous revision of this answer.
My implementation seems to be correct and performs well by comparison with all the others I've seen. Tests and benchmarks are here
extension RandomAccessCollection {
/// Returns `self.map(transform)`, computed in parallel.
///
/// - Requires: `transform` is safe to call from multiple threads.
func concurrentMap<B>(_ transform: (Element) -> B) -> [B] {
let batchSize = 4096 // Tune this
let n = self.count
let batchCount = (n + batchSize - 1) / batchSize
if batchCount < 2 { return self.map(transform) }
return Array(unsafeUninitializedCapacity: n) {
uninitializedMemory, resultCount in
resultCount = n
let baseAddress = uninitializedMemory.baseAddress!
DispatchQueue.concurrentPerform(iterations: batchCount) { b in
let startOffset = b * n / batchCount
let endOffset = (b + 1) * n / batchCount
var sourceIndex = index(self.startIndex, offsetBy: startOffset)
for p in baseAddress+startOffset..<baseAddress+endOffset {
p.initialize(to: transform(self[sourceIndex]))
formIndex(after: &sourceIndex)
}
}
}
}
}
Hope this helps,
-Dave
You can use parMap(), which is parrallel map. You can use activity monitor to check if it's parrallel map.
func map<T: Collection, U>( _ transform: (T.Iterator.Element) -> U, _ xs: T) -> [U] {
return xs.reduce([U](), {$0 + [transform($1)]})
}
public func parMap<T,U>(_ transform: #escaping (T)->U, _ xs: [T]) -> [U] {
let len = xs.count
var results = [U?](repeating: nil, count: len)
let process = { (i: Int) -> Void in results[i] = transform(xs[i]) }
DispatchQueue.concurrentPerform(iterations: len, execute: process)
return map({$0!}, results)
}
func test() {
parMap({_ in Array(1...10000000).reduce(0,+)}, Array(1...10))
}