Aligning 12-byte struct within a warp - cuda - arrays

Question

Lets say we have a struct of 3 integers which is not aligned:
struct data {
int x;
int y;
int z;
};
I pass an array of this struct to kernel. I'm aware that I should pass struct of array instead of array of struct but this is not important for this question.
32 threads inside a warp, access memory in coalesced manner (i to i + 31) which equals total memory of 384 bytes. 384 bytes is multiple of L1 cache line (128 bytes) which means three memory transaction of 128-byte each.
Now if we have an aligned struct:
struct __align__(16) aligned_data {
int x;
int y;
int z;
};
if access patterns remains the same as previous example, then it would fetch 512 bytes of memory which is 4 memory transaction each requesting 128-byte.
So this means is first example more efficient or second one is still more efficient although it fetches more memory.

Answer 1

The only real way to answer a question is by benchmarking. And if you do, you may not get the same answer depending on your hardware. When I run this:
#define NITER (128)
struct data {
int x;
int y;
int z;
};
struct __align__(16) aligned_data {
int x;
int y;
int z;
};
template<typename T, int niter>
__global__
void kernel(T *in, int *out, int dowrite=0)
{
int tid = threadIdx.x + blockIdx.x * blockDim.x;
int nthreads = blockDim.x * gridDim.x;
int oval = 0;
#pragma unroll
for(int i=0; i<niter; ++i,tid+=nthreads) {
T val = in[tid];
oval += val.x + val.y + val.z;
}
if (dowrite) {
out[tid] = oval;
}
}
template __global__ void kernel<data,NITER>(data *, int*, int);
template __global__ void kernel<aligned_data,NITER>(aligned_data *, int*, int);
int main()
{
const int bs = 512;
const int nb = 32;
const int nvals = bs * nb * NITER;
data *d_; cudaMalloc((void **)&d_, sizeof(data) * size_t(nvals));
aligned_data *ad_; cudaMalloc((void **)&ad_, sizeof(aligned_data) * size_t(nvals));
for(int i=0; i<10; ++i) {
kernel<data,NITER><<<nb, bs>>>(d_, (int *)0, 0);
kernel<aligned_data,NITER><<<nb, bs>>>(ad_, (int *)0, 0);
cudaDeviceSynchronize();
}
cudaDeviceReset();
return 0;
}
I see that the aligned structure version gives overall higher performance on a compute 5.2 capability device:
Time(%) Time Calls Avg Min Max Name
52.71% 2.3995ms 10 239.95us 238.10us 241.79us void kernel<data, int=128>(data*, int*, int)
47.29% 2.1529ms 10 215.29us 214.91us 215.51us void kernel<aligned_data, int=128>(aligned_data*, int*, int)
In this case I would assumed that the roughly 10% improvement is down to the lower number of load instructions which are issued. In the unaligned case the compiler issues three 32 bit loads to fetch the structure, whereas in the aligned case the compiler issues a single 128 bit load to fetch the structure. The reduction in instructions seems to offset the 25% wasted net memory bandwidth. On other hardware with different memory instruction throughput to bandwdith ratios, the result might well be different.