Dependency in Adjacent Elements while Updating 2d array (OpenMP) - c

Question

I have a 2d array, say arr[SIZE][SIZE], which is updated in two for loops in the form:
for(int i = 0; i < SIZE; i++)
for(int j = 0; j < SIZE; j++)
arr[i][j] = new_value();
that I am trying to parallelise using OpenMP.
There are two instances where this occurs, the first is the function new_value_1() which relies on arr[i+1][j] and arr[i][j+1] (the "edge of the array" issue is already taken care of), which I can happily parallelise using the chessboard technique:
for(int l = 0; l < 2; l++)
#pragma omp parallel for private(i, j)
for(int i = 0; i < SIZE; i++)
for(int j = (i + l) % 2; j < SIZE; j += 2)
arr[i][j] = new_value_1();
The issue comes with this second instance, new_value_2(), which relies upon:
arr[i+1][j],
arr[i-1][j],
arr[i][j+1],
arr[i][j-1]
i.e. the adjacent elements in all directions.
Here, there is a dependency on the negative adjacent elements, so arr[0][2] = new_value_2() depends on the already updated value of arr[0][1] which would not be computed until the second l loop.
I was wondering if there was something I was missing in parallelising this way or if the issue is inherent with the way the algorithm works? If the former, any other approaches would be appreciated.

Answer 1

I was wondering if there was something I was missing in parallelising this way or if the issue is inherent with the way the algorithm works?
Yes, you're missing something, but not along the lines you probably hoped. Supposing that the idea is that the parallel version should compute the same result as the serial version, the checkerboard approach does not solve the problem even for the new_value_1() case. Consider this layout of array elements:
xoxoxo
oxoxox
xoxoxo
oxoxox
On the first of the two checkerboard passes, the 'x' elements are updated according to the original values of the 'o' elements -- so far, so good -- but on the second pass, the 'o' elements are updated based on the new values of the 'x' elements. The data dependency is broken, yes, but the overall computation is not the same. Of course, the same applies even more so to the new_value_2() computation. The checkerboarding doesn't really help you.
If the former, any other approaches would be appreciated.
You could do the computation in shells. For example, consider this labeling of the array elements:
0123
1234
2345
3456
All the elements with the same label can be computed in parallel (for both new_value() functions), provided that all those with numerically lesser labels are computed first. That might look something like this:
for(int l = 0; l < (2 * SIZE - 1); l++) {
int iterations = (l < SIZE) ? (l + 1) : (2 * SIZE - (l + l));
int i = (l < SIZE) ? l : (SIZE - 1);
int j = (l < SIZE) ? 0 : (1 + l - SIZE);
#pragma omp parallel for private(i, j, m)
for(int m = 0; m < iterations; m++) {
arr[i--][j++] = new_value_1();
}
}
You won't that way get as much of a benefit from parallelization, but that is an inherent aspect of the way the serial computation works, at least for the new_value_2() case.
For the new_value_1() case, though, you might do a little better by going row by row:
for(int i = 0; i < SIZE; i++)
#pragma omp parallel for private(j)
for(int j = 0; j < SIZE; j++)
arr[i][j] = new_value_1();
Alternatively, for the new_value_1() case only, you could potentially get a good speedup by storing the results in a separate array:
#pragma omp parallel for private(i, j), collapse(2)
for(int i = 0; i < SIZE; i++)
for(int j = 0; j < SIZE; j++)
arr2[i][j] = new_value_1();
If that requires you to copy the result back to the original array afterward then it might well not be worth it, but you could potentially avoid that by flipping back and forth between two arrays: compute from the first into the second, and then the next time around, compute from the second into the first ( if the problem [PSPACE] scaling permits having such an extended in-RAM allocation, i.e. it again comes at a [PTIME] cost, hopefully paid just once ).