I'm generating hashes (MD5) of numbers from 1 to N in some threads. According to the first letter of the hash, the number that generates it is stored in an array. E.g, the number 1 results in c4ca4238a0b923820dcc509a6f75849b and the number 2 in c81e728d9d4c2f636f067f89cc14862c, so they are stored in a specific array of hashes that starts with "c".
The problem is that I need to generate them sorted from the lower to the higher. It is very expensive to sort them after the sequence is finished, N can be as huge as 2^40. As I'm using threads the sorting never happens naturally. E.g. One thread can generate the hash of the number 12 (c20ad4d76fe97759aa27a0c99bff6710) and store it on "c" array and other then generates the hash of the number 8 (c9f0f895fb98ab9159f51fd0297e236d) and store it after the number 12 in "c" array.
I can't simply verify the last number on the array because as far as the threads are running they can be very far away from each other.
Is there any solution for this thread problem? Any solution that is faster than order the array after all the threads are finished would be great.
I'm implementing this in C.
Thank you!
Instead of having one array for each prefix (eg. "c"), have one array per thread for each prefix. Each thread inserts only into its own arrays, so it will always insert the numbers in increasing order and the individual thread arrays will remain sorted.
You can then quickly (O(N)) coalesce the arrays at the end of the process, since the individual arrays will all be sorted. This will also speed up the creation process, since you won't need any locking around the arrays.
Since you mentioned pthreads I'm going to assume you're using gcc (this is not necessarily the case but it's probably the case). You can use the __sync_fetch_and_add to get the value for the end of the array and add one to it in one atomic operation. It would go something like the following:
insertAt = __sync_fetch_and_add(&size[hash], 1);
arrayOfInts[insertAt] = val;
The only problem you'll run into is if you need to resize the arrays (not sure if you know the array size beforehand or not). For that you will need a lock (most efficiently one lock per array) that you lock exclusively while reallocating the array and non-exclusively when inserting. Particularly this could be done with the following functions (which assume programmer does not release an unlocked lock):
// Flag 2 indicates exclusive lock
void lockExclusive(int* lock)
{
while(!__sync_bool_compare_and_swap(lock, 0, 2));
}
void releaseExclusive(int* lock)
{
*lock = 0;
}
// Flag 8 indicates locking
// Flag 1 indicates non-exclusive lock
void lockNonExclusive(int* lock, int* nonExclusiveCount)
{
while((__sync_fetch_and_or(lock, 9) & 6) != 0);
__sync_add_and_fetch(nonExclusiveCount, 1);
__sync_and_and_fetch(lock, ~8);
}
// Flag 4 indicates unlocking
void releaseNonExclusive(int* lock, int* nonExclusiveCount)
{
while((__sync_fetch_and_or(lock, 4) & 8) != 0);
if(__sync_sub_and_fetch(nonExclusiveCount) == 0);
__sync_and_and_fetch(lock, ~1);
__sync_and_and_fetch(lock, 4);
}
Related
The bounty expires in 5 days. Answers to this question are eligible for a +100 reputation bounty.
johnbakers is looking for an answer from a reputable source:
Desiring a good understanding of why copy-on-write is not interfering with multithreaded updates to different array indexes and whether this is in fact safe to do from a specification standpoint, as it appears to be.
I see frequent mention that Swift arrays, due to copy-on-write, are not threadsafe, but have found this works, as it updates different and unique elements in an array from different threads simultaneously:
//pixels is [(UInt8, UInt8, UInt8)]
let q = DispatchQueue(label: "processImage", attributes: .concurrent)
q.sync {
DispatchQueue.concurrentPerform(iterations: n) { i in
... do work ...
pixels[i] = ... store result ...
}
}
(simplified version of this function)
If threads never write to the same indexes, does copy-on-write still interfere with this? I'm wondering if this is safe since the array itself is not changing length or memory usage. But it does seem that copy-on-write would prevent the array from staying consistent in such a scenario.
If this is not safe, and since doing parallel computations on images (pixel arrays) or other data stores is a common requirement in parallel computation, what is the best idiom for this? Is it better that each thread have its own array and then they are combined after all threads complete? It seems like additional overhead and the memory juggling from creating and destroying all these arrays doesn't feel right.
Updated answer:
Having thought about this some more, I suppose the main thing is that there's no copy-on-write happening here either way.
COW happens because arrays (and dictionaries, etc) in Swift behave as value types. With value types, if you pass a value to a function you're actually passing a copy of the value. But with array, you really don't want to do that because copying the entire array is a very expensive operation. So Swift will only perform the copy when the new copy is edited.
But in your example, you're not actually passing the array around in the first place, so there's no copy on write happening. The array of pixels exists in some scope, and you set up a DispatchQueue to update the pixel values in place. Copy-on-write doesn't come into play here because you're not copying in the first place.
I see frequent mention that Swift arrays, due to copy-on-write, are not threadsafe
To the best of my knowledge, this is more or less the opposite of the actual situation. Swift arrays are thread-safe because of copy-on-write. If you make an array and pass it to multiple different threads which then edit the array (the local copy of it), it's the thread performing the edits that will make a new copy for its editing; threads only reading data will keep reading from the original memory.
Consider the following contrived example:
import Foundation
/// Replace a random element in the array with a random int
func mutate(array: inout [Int]) {
let idx = Int.random(in: 0..<array.count)
let val = Int.random(in: 1000..<10_000)
array[idx] = val
}
class Foo {
var numbers: [Int]
init(_ numbers: [Int]) {
// No copying here; the local `numbers` property
// will reference the same underlying memory buffer
// as the input array of numbers. The reference count
// of the underlying buffer is increased by one.
self.numbers = numbers
}
func mutateNumbers() {
// Copy on write can happen when we call this function,
// because we are not allowed to edit the underlying
// memory buffer if more than one array references it.
// If we have unique access (refcount is 1), we can safely
// edit the buffer directly.
mutate(array: &self.numbers)
}
}
var numbers = [0, 1, 2, 3, 4, 5]
var foo_instances: [Foo] = []
for _ in 0..<4 {
let t = Thread() {
let f = Foo(numbers)
foo_instances.append(f)
for _ in 0..<5_000_000 {
f.mutateNumbers()
}
}
t.start()
}
for _ in 0..<5_000_000 {
// Copy on write can potentially happen here too,
// because we can get here before the threads have
// started mutating their arrays. If that happens,
// the *original* `numbers` array in the global will
// make a copy of the underlying buffer, point to the
// the new one and decrement the reference count of the
// previous buffer, potentially releasing it.
mutate(array: &numbers)
}
print("Global numbers:", numbers)
for foo in foo_instances {
print(foo.numbers)
}
Copy-on-write can happen when the threads mutate their numbers, and it can happen when the main thread mutates the original array, and but in neither case will it affect any of the data used by the other objects.
Arrays and copy-on-write are both thread-safe. The copying is done by the party responsible for the editing, not the other instances referencing the memory, so two threads will never step on each others toes here.
However, what you're doing isn't triggering copy-on-write in the first place, because the different threads are writing to the array in place. You're not passing the value of the array to the queue. Due to the how the closure works, it's more akin to using the inout keyword on a function. The reference count of the underlying buffer remains 1 but the reference count of the array goes up, because the threads executing the work are all pointing to the same array. This means that COW doesn't come into play at all.
As for this part:
If this is not safe, and since doing parallel computations on images (pixel arrays) or other data stores is a common requirement in parallel computation, what is the best idiom for this?
It depends. If you're simply doing a parallel map function, executing some function on each pixel that depends solely on the value of that pixel, then just doing a concurrentPerform for each pixel seems like it should be fine. But if you want to do something like apply a multi-pixel filter (like a convolution for example), then this approach does not work. You can either divide the pixels into 'buckets' and give each thread a bucket for itself, or you can have a read-only input pixel buffer and an output buffer.
Old answer below:
As far as I can tell, it does actually work fine. This code below runs fine, as best as I can tell. The dumbass recursive Fibonacci function means the latter values in the input array take a bit of time to run. It maxes out using all CPUs in my computer, but eventually only the slowest value to compute remains (the last one), and it drops down to just one thread being used.
As long as you're aware of all the risks of multi-threading (don't read the same data you're writing, etc), it does seem to work.
I suppose you could use withUnsafeMutableBufferPointer on the input array to make sure that there's no overhead from COW or reference counting.
import Foundation
func stupidFib(_ n: Int) -> Int {
guard n > 1 else {
return 1
}
return stupidFib(n-1) + stupidFib(n-2)
}
func parallelMap<T>(over array: inout [T], transform: (T) -> T) {
DispatchQueue.concurrentPerform(iterations: array.count) { idx in
array[idx] = transform(array[idx])
}
}
var data = (0..<50).map{$0} // ([0, 1, 2, 3, ... 49]
parallelMap(over: &data, transform: stupidFib) // uses all CPU cores (sort of)
print(data) // prints first 50 numbers in the fibonacci sequence
Suppose I want to do a parallel sum of an array, here is the pseudo code:
SplitIntoNParts();
int *result = new int[N]; // this is shared among N worker threads
// worker i store its addition result in result[i]
// since worker thread i only accesses result[i], no race condition happens
for(int i = 0; i < N; i++){
// create a worker thread to do addition in part i, and store the result in result[i]
}
// join threads
// return sum of result array
I read from a lecture on parallel programming (page25)
If array elements happen to share a cache line, this leads to false sharing. – Non-shared data in the same cache line so each update invalidates the cache line … in essence “sloshing independent data” back and forth between threads.
It says sharing the result array is false sharing, but I didn't quite get it.
What does 'each update invalidate the cache line' mean?
What does the 'false sharing' result from?
Does it mean that updates to cached array will invalidate the cache?
Or it means if one core updated its cache line, then it has to invalidate all other cores' cache lines? But if this is the case, even if we use a single variable result to store the result (and use locks to synchronize the addition of result) instead of using an array, this situation still can not be avoided (one core modifying result variable still invalidates all other cores' result)
Is it possible to create arrays based of their index as in
int x = 4;
int y = 5;
int someNr = 123;
int foo[x][y] = someNr;
dynamically/on the run, without creating foo[0...3][0...4]?
If not, is there a data structure that allow me to do something similar to this in C?
No.
As written your code make no sense at all. You need foo to be declared somewhere and then you can index into it with foo[x][y] = someNr;. But you cant just make foo spring into existence which is what it looks like you are trying to do.
Either create foo with correct sizes (only you can say what they are) int foo[16][16]; for example or use a different data structure.
In C++ you could do a map<pair<int, int>, int>
Variable Length Arrays
Even if x and y were replaced by constants, you could not initialize the array using the notation shown. You'd need to use:
int fixed[3][4] = { someNr };
or similar (extra braces, perhaps; more values perhaps). You can, however, declare/define variable length arrays (VLA), but you cannot initialize them at all. So, you could write:
int x = 4;
int y = 5;
int someNr = 123;
int foo[x][y];
for (int i = 0; i < x; i++)
{
for (int j = 0; j < y; j++)
foo[i][j] = someNr + i * (x + 1) + j;
}
Obviously, you can't use x and y as indexes without writing (or reading) outside the bounds of the array. The onus is on you to ensure that there is enough space on the stack for the values chosen as the limits on the arrays (it won't be a problem at 3x4; it might be at 300x400 though, and will be at 3000x4000). You can also use dynamic allocation of VLAs to handle bigger matrices.
VLA support is mandatory in C99, optional in C11 and C18, and non-existent in strict C90.
Sparse arrays
If what you want is 'sparse array support', there is no built-in facility in C that will assist you. You have to devise (or find) code that will handle that for you. It can certainly be done; Fortran programmers used to have to do it quite often in the bad old days when megabytes of memory were a luxury and MIPS meant millions of instruction per second and people were happy when their computer could do double-digit MIPS (and the Fortran 90 standard was still years in the future).
You'll need to devise a structure and a set of functions to handle the sparse array. You will probably need to decide whether you have values in every row, or whether you only record the data in some rows. You'll need a function to assign a value to a cell, and another to retrieve the value from a cell. You'll need to think what the value is when there is no explicit entry. (The thinking probably isn't hard. The default value is usually zero, but an infinity or a NaN (not a number) might be appropriate, depending on context.) You'd also need a function to allocate the base structure (would you specify the maximum sizes?) and another to release it.
Most efficient way to create a dynamic index of an array is to create an empty array of the same data type that the array to index is holding.
Let's imagine we are using integers in sake of simplicity. You can then stretch the concept to any other data type.
The ideal index depth will depend on the length of the data to index and will be somewhere close to the length of the data.
Let's say you have 1 million 64 bit integers in the array to index.
First of all you should order the data and eliminate duplicates. That's something easy to achieve by using qsort() (the quick sort C built in function) and some remove duplicate function such as
uint64_t remove_dupes(char *unord_arr, char *ord_arr, uint64_t arr_size)
{
uint64_t i, j=0;
for (i=1;i<arr_size;i++)
{
if ( strcmp(unord_arr[i], unord_arr[i-1]) != 0 ){
strcpy(ord_arr[j],unord_arr[i-1]);
j++;
}
if ( i == arr_size-1 ){
strcpy(ord_arr[j],unord_arr[i]);
j++;
}
}
return j;
}
Adapt the code above to your needs, you should free() the unordered array when the function finishes ordering it to the ordered array. The function above is very fast, it will return zero entries when the array to order contains one element, but that's probably something you can live with.
Once the data is ordered and unique, create an index with a length close to that of the data. It does not need to be of an exact length, although pledging to powers of 10 will make everything easier, in case of integers.
uint64_t* idx = calloc(pow(10, indexdepth), sizeof(uint64_t));
This will create an empty index array.
Then populate the index. Traverse your array to index just once and every time you detect a change in the number of significant figures (same as index depth) to the left add the position where that new number was detected.
If you choose an indexdepth of 2 you will have 10² = 100 possible values in your index, typically going from 0 to 99.
When you detect that some number starts by 10 (103456), you add an entry to the index, let's say that 103456 was detected at position 733, your index entry would be:
index[10] = 733;
Next entry begining by 11 should be added in the next index slot, let's say that first number beginning by 11 is found at position 2023
index[11] = 2023;
And so on.
When you later need to find some number in your original array storing 1 million entries, you don't have to iterate the whole array, you just need to check where in your index the first number starting by the first two significant digits is stored. Entry index[10] tells you where the first number starting by 10 is stored. You can then iterate forward until you find your match.
In my example I employed a small index, thus the average number of iterations that you will need to perform will be 1000000/100 = 10000
If you enlarge your index to somewhere close the length of the data the number of iterations will tend to 1, making any search blazing fast.
What I like to do is to create some simple algorithm that tells me what's the ideal depth of the index after knowing the type and length of the data to index.
Please, note that in the example that I have posed, 64 bit numbers are indexed by their first index depth significant figures, thus 10 and 100001 will be stored in the same index segment. That's not a problem on its own, nonetheless each master has his small book of secrets. Treating numbers as a fixed length hexadecimal string can help keeping a strict numerical order.
You don't have to change the base though, you could consider 10 to be 0000010 to keep it in the 00 index segment and keep base 10 numbers ordered, using different numerical bases is nonetheless trivial in C, which is of great help for this task.
As you make your index depth become larger, the amount of entries per index segment will be reduced
Please, do note that programming, especially lower level like C consists in comprehending the tradeof between CPU cycles and memory use in great part.
Creating the proposed index is a way to reduce the number of CPU cycles required to locate a value at the cost of using more memory as the index becomes larger. This is nonetheless the way to go nowadays, as masive amounts of memory are cheap.
As SSDs' speed become closer to that of RAM, using files to store indexes is to be taken on account. Nevertheless modern OSs tend to load in RAM as much as they can, thus using files would end up in something similar from a performance point of view.
I found something weird: for whatever reason, the array version of this will almost always contain random 0s after the following code runs, where as the pointer version does not.
var a = UnsafeMutablePointer<Int>.allocate(capacity: N)
//var a = [Int](repeating: 0, count: N)
let n = N / iterations
DispatchQueue.concurrentPerform(iterations: iterations) { j in
for i in max(j * n, 1)..<((j + 1) * n) {
a[i] = 1
}
}
for i in max(1, N - (N % n))..<N {
a[i] = 1
}
Is there a particular reason for this? I know that Swift arrays might not be consecutive in memory, but accessing the memory location with respect to each Index just once, from a single thread should not do anything too funny.
Arrays are not thread safe and, although they are bridged to Objective-C objects, they behave as value types with COW (copy on Write) logic. COW on an array will make a copy of the whole array when any element changes and the reference counter is greater than 1 (conceptually, the actual implementation is a bit more subtle).
Your thread that makes changes to the array will trigger a memory copy whenever the main thread happens to reference and element. The main thread, also makes changes so it will cause COW as well. What you end up with is the state of the last modified copy used by either thread. This will randomly leave some of the changes in limbo and explains the "missed" items.
To avoid this you would need to perform all changes in a specific thread and use sync() to ensure that COW on the array is only performed by that thread (this may actually reduce the number of memory copies and give better performance for very large arrays). There will be overhead and potential contention using this approach though. It is the price to pay for thread safeness.
The other way to approach this would be to use an array of objects (referenced types). This makes your array a simple list of pointers that are not actually changed by modifying data in the referenced objects. although, in actual programs, you would need to mind thread safeness within each object instance, there would be far less interference (and overhead) than what you get with arrays of value types.
I have a simple C program which uses a varying number of pthreads to find the first N prime numbers, adding candidates found to be prime to an array. The array is passed as an arg to each thread, with each thread executing nearly all of its code in a critical section. I have no issue with the prime checking, and the printing to screen of prime numbers and the result_number variable as they are found works. However, when N primes are found, and the array is printed, I find that (roughly) every second time the program executes, some (variably from 1 to 5) of the early prime number array elements (generally restricted to those < 17) are printed out as extremely large or negative numbers, with the majority of primes printing fine. Only the code of the thread function is below (not checkPrime or main), as everything else seems to work fine.
Also, if the program is executed with a single thread (i.e. no sharing of the array between multiple threads for updating), this peculiarity never occurs.
result_number, candidate, N are all global vars.
void *primeNums(void *arg) {
pthread_mutex_lock(&mutex);
int *array = (int *) arg;
int is_prime = 0;
int j = 0;
while (result_number <= N) {
candidate++;
is_prime = checkPrime(candidate);
if (is_prime == 1) {
array[result_number] = candidate;
if (result_number == N) {
while (j < N) {
printf("%d\n", array[j]);
j++;
}
}
/* Test verification output; always accurate */
printf("Result number: %d = %d\n", result_number, candidate);
result_number++;
}
}
pthread_mutex_unlock(&mutex);
}
I genuinely do not believe other Qs cover this, as I have looked (and would have preferred finding an answer to writing my own question). There is a chance that I am not searching properly, admittedly.
EDIT: Example unwanted output:
-386877696
3
-395270400
32605
11
13
...
Continues on fine from here.
I have a hunch that if, instead of passing in array and using the global result_number, you pass in array + result_number and loop on a local variable: i = 0; while(i < N) {...}, you may solve your problem. (assuming array + result_number + N - 1 doesn't go out of bounds).
The problem with using global variables to hold the range information for each thread is that the thread function primeNums() modifies some of them. So if you start your first thread (thread #1) with result_number set to the start of the range you want thread #1 to process, thread #1 will keep changing its value while you reset it to give to thread #2. So thread #2 won't be processing the range you want it to process.
I assume you want each thread to process a separate range of indexes in your array? Currently you are passing a pointer to the beginning of the array to the function and using a global variable to hold the index into the array of the chunk you want to have processed by that thread.
To avoid using that global index all you have to do is pass a pointer to an offset into the middle of your array where you want processing to begin. So rather than pass in the value that points to the beginning element (array) pass in a value that points to the element you want to start processing from (array + result_number).
If you do that, inside your function primeNums() it acts as if the pointer you passed in is the beginning of an array (even though its somewhere in the middle) and you can run your loop from 0 to N because you have already added result_number before you called the function.
Having said all that I suspect you are still not going to be able to process this array in parallel (if that's is indeed what you are trying to do) because each thread relies on the candidate being set to the largest value from the previous thread...
To protect 'candidate` from being simultaneously changed by the code that launches the other threads (if you do that) you can take a copy of that variable after you synchronize on your mutex (lock). But, to be honest, I am not sure if this algorithm is going to let you parallelise this processing.