how to do memory blocking for this code snippet - c

Question

I have this piece of code and I am trying to optimize it using cache coherence method like temporal and spatial locality with cache blocking. (https://www.intel.com/content/www/us/en/developer/articles/technical/cache-blocking-techniques.html)
void randFunction1(int *arrayb, int dimension)
{
int i, j;
for (i = 0; i < dimension; ++i)
for (j = 0; j < dimension; ++j) {
arrayb[j * dimension+ i] = arrayb[j * dimension+ i] || arrayb[i * dimension+ j];
}
}
This is how I have optimised it but I was told it doesn't seem to make use of the memory blocking techniques.
for (int i = 0; i < dimension; ++i){
int j = i;
for (; j < dimension; ++j)
{
//access 2 times
arrayb[j * dimension+ i] = arrayb[j * dimension+ i] || arrayb[i * dimension+ j];
arrayb[i * dimension+ j] = arrayb[i * dimension+ j] || arrayb[j * dimension + i];
}
}
Could someone tell me how I can make use of the cache blocking (using locality for smaller tiles) for this sample piece of code? Any help is appreciated thank you!

Answer 1

I think you have a fundamental misunderstanding of cache blocking, misunderstood what you were being asked to do, or whoever asked you to do it doesn't understand. I am also hesitant to give you the full answer because this smells of a contrived example for a home work problem.
The idea is to block/tile/window up the data you're operating on, so the data you're operating on stays in the cache as you operate on it. To do this effectively you need to know the size of the cache and the size of the objects. You didn't give us enough details to know these answers but I can make some assumptions to illustrate how you might do this with the above code.
First how arrays are laid out in memory just so we can reference it later. Say dimension is 3.
That means we have a grid layout where i is the first number and j is the second like...
[0,0][0,1][0,2]
[1,0][1,1][1,2]
[2,0][2,1][2,2]
which is really in memory like:
[0,0][0,1][0,2][1,0][1,1][1,2][2,0][2,1][2,2]
We can also treat this like a 1d array where:
[0,0][0,1][0,2][1,0][1,1][1,2][2,0][2,1][2,2]
[ 0 ][ 1 ][ 2 ][ 3 ][ 4 ][ 5 ][ 6 ][ 7 ][ 8 ]
If our cache line could hold say 3 of these guys in there then there would be 3 'blocks'. 0-2, 3-5, and 6-8. If we access them in order, blocking just happens (assuming correct byte alignment of index 0 of the array... but lets keep it simple for now - this is likely taken care of already anyway). That is when we access 0, then 0, 1 and 2 are loaded into the cache. Next we access 1, it's already there. Then 2, already there. Then 3, load 3, 4 and 5 into the cache and so on.
Let's take a look at the original code for a second.
arrayb[j * dimension+ i] = arrayb[j * dimension+ i] || arrayb[i * dimension+ j];
Lets do just a couple of iterations but take out the indexing variables and replace them with their values. I'll use ^ to point to indexes you access and | to show the locations of our imaginary cache lines.
arrayb[0] = arrayb[0] || arrayb[0]
[ 0 ][ 1 ][ 2 ] | [ 3 ][ 4 ][ 5 ] | [ 6 ][ 7 ][ 8 ]
^
arrayb[3] = arrayb[3] || arrayb[1]
[ 0 ][ 1 ][ 2 ] | [ 3 ][ 4 ][ 5 ] | [ 6 ][ 7 ][ 8 ]
^ ^
arrayb[6] = arrayb[6] || arrayb[2]
[ 0 ][ 1 ][ 2 ] | [ 3 ][ 4 ][ 5 ] | [ 6 ][ 7 ][ 8 ]
^ ^
arrayb[1] = arrayb[1] || arrayb[3]
[ 0 ][ 1 ][ 2 ] | [ 3 ][ 4 ][ 5 ] | [ 6 ][ 7 ][ 8 ]
^ ^
So you see other than the first iteration, you cross the cache line every time jumping all over the place.
I think you noticed that the operation you're performing is logical or. That means you do not have to preserve the original order of operations as you go through the loop as your answer will be the same. That is it doesn't matter if you do arrayb[1] = arrayb[1] || arrayb[3] first or arrayb[3] = arrayb[3] | arrayb[1] first.
In your proposed solution you might think you're doing a little better because you noticed the pattern where on the second and fourth iteration we access the same indexes (just flip where we're reading and writing) but you didn't adjust the loops at all, so actually you just did twice the work.
0 = 0 || 0
0 = 0 || 0
3 = 3 || 1
1 = 1 || 3
6 = 6 || 2
2 = 2 || 6
1 = 1 || 3
3 = 3 || 1
4 = 4 || 4
4 = 4 || 4
7 = 7 || 5
5 = 5 || 7
2 = 2 || 6
6 = 6 || 2
5 = 5 || 7
7 = 7 || 5
8 = 8 || 8
8 = 8 || 8
If you fix the double work, you're on your way but you're not really using a blocking strategy. And to be honest, you can't. It's almost like the problem was designed to be not-real-world and purposely cause caching problems. The problem with your example is that you're using a single array that only accesses the same memory locations in pairs (twice). Other than their swap, they're never reused.
You can kind of optimize some of the accesses but you'll always be stuck with a majority collection that crosses boundaries. I think this is what you've been asked to do, but this is not a very good example problem for it. If we keep in mind how the memory in your array is actually being accessed and never really reused then increasing the size of the example makes it really obvious.
Say dimensions was 8 and your cache is big enough to hold 16 items (x86_64 can hold 16 ints in a cacheline). Then the most optimal access grouping would be operations where all indexes fell within 0-15, 16-31, 32-47, or 48-63. There aren't that many of them.
Not crossing a cache line:
0 = 0 || 0
1 = 1 || 8
8 = 8 || 1
9 = 9 || 9
18 = 18 || 18
19 = 19 || 26
26 = 26 || 19
27 = 27 || 27
36 = 36 || 36
37 = 37 || 44
44 = 44 || 37
54 = 54 || 54
55 = 55 || 62
62 = 62 || 55
63 = 63 || 63
Always crossing a cache line:
2 = 2 || 16
3 = 3 || 24
4 = 4 || 32
5 = 5 || 40
6 = 6 || 48
7 = 7 || 56
10 = 10 || 17
11 = 11 || 25
12 = 12 || 33
13 = 13 || 41
14 = 14 || 49
15 = 15 || 57
16 = 16 || 2
17 = 17 || 10
20 = 20 || 34
21 = 21 || 42
22 = 22 || 50
23 = 23 || 58
24 = 24 || 3
25 = 25 || 11
28 = 28 || 35
29 = 29 || 43
30 = 30 || 51
31 = 31 || 59
32 = 32 || 4
33 = 33 || 12
34 = 34 || 20
35 = 35 || 28
38 = 38 || 52
39 = 39 || 60
40 = 40 || 5
41 = 41 || 13
42 = 42 || 21
43 = 43 || 29
45 = 45 || 45
46 = 46 || 53
47 = 47 || 61
48 = 48 || 6
49 = 49 || 14
50 = 50 || 22
51 = 51 || 30
52 = 52 || 38
53 = 53 || 46
56 = 56 || 7
57 = 57 || 15
58 = 58 || 23
59 = 59 || 31
60 = 60 || 39
61 = 61 || 47
This really gets terrible as the number of items out paces the number that'll fit in the cache. You're only hope to save anything at this point is the pattern you noticed where you can do half the memory accesses that while smart, is not blocking/tiling.
The link you provided is similarly bad imo for illustrating cache blocking. It doesn't do a good job of describing what is actually taking place in its loops but at least it tries.
They tile the inner loop to keep memory accesses more local, which I think is what you've been asked to do but given a problem it can't apply to.
It smells like your teacher meant to give you 2 or 3 arrays, but accidentally gave you just one. It's very close to matrix multiplication but missing an inner loop and two other arrays.