NEON implementation in ARM - arm

Question

I am a beginner in NEON and wanted to optimize the following code but while it compiles and produces the same output as desired , I don't see any improvement. AFAIK NEON is helpful in doing operations on contiguous block of data so I was hoping some improvement in execution time and cycles.What am i doing wrong??
I'm working on gcc on Ubuntu 12.04 with -03 level optimization
normal c implementation
for(i= 0;i<9215;i++)
{
Z[i] = (L[i]>0)?0:1;
}
Neon form
for(i=0;i<9215;i+=4)
{
int32x4_t l_N = vld1q_s32(&L[i]);
uint32x4_t mask_n=vcltq_s32(l_N,zero_N);
int32x4_t z_n = vbslq_s32(mask_n,one_N,zero_N);
vst1q_s32(&Z[i],z_n);
}

Answer 1

Problems:
You are using a very inefficient algorithm for the computation inside the loop
Your routine suffers from heavy pipeline interlocks, instruction by instruction
void isNonNatural(int32_t * pDst, int32_t *pSrc, int n)
{
int32x4_t vec;
const int32x4_t one = vdupq_n_s32(1);
int32_t a;
unsigned int i;
if (n >= 4)
{
n -= 4;
while (1) {
do {
n -= 4;
vec = vld1q_s32(pSrc++);
vec = vqsubq_s32(vec, one);
vec = (int32x4_t) vshrq_n_u32((uint32x4_t) vec, 31);
vst1q_s32(pDst++, vec);
} while (n >= 0);
if (n <= -4) return;
// dealing with residuals
pSrc += n; // rewind pointers
pDst += n;
} // iterate for one last time
}
for (i = 0; i < n; ++i) {
a = *pSrc++;
if (a > 0) a = 0; else a = 1;
*pDst++ = a;
}
}
This function above should be somewhat faster than your implementation.
a saturating subtraction by 1 is done so that 0 becomes -1 while 0x80000000 remains 0x80000000
The elements get shifted by 31 bits so that only the sign bit remains.
I you can live with 0xffffffff instead of 1, you can leave out the typecasting and use vshrq_n_s32 instead. It won't be any faster though.
Pay attention to the residual management.
Programming NEON is like driving a big truck. You shouldn't drive it like a compact car.
While NEON can compute multiple data at once, mostly in a single cycle, it has higher instruction latencies, usually 3~4 cycles. In other words, each and every instruction has to wait that long for the previous one to return the result in the implementation above.
Virtually the only way avoiding this is unrolling, a deep one.
void isNonNatural_unroll(int32_t * pDst, int32_t *pSrc, int n)
{
int32x4_t vec1, vec2, vec3, vec4;
const int32x4_t one = vdupq_n_s32(1);
int32_t a;
unsigned int i;
if (n >= 16)
{
n -= 16;
while (1) {
do {
n -= 16;
vec1 = vld1q_s32(pSrc++);
vec2 = vld1q_s32(pSrc++);
vec3 = vld1q_s32(pSrc++);
vec4 = vld1q_s32(pSrc++);
vec1 = vqsubq_s32(vec1, one);
vec2 = vqsubq_s32(vec2, one);
vec3 = vqsubq_s32(vec3, one);
vec4 = vqsubq_s32(vec4, one);
vec1 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec1, 31);
vec2 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec2, 31);
vec3 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec3, 31);
vec4 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec4, 31);
vst1q_s32(pDst++, vec1);
vst1q_s32(pDst++, vec2);
vst1q_s32(pDst++, vec3);
vst1q_s32(pDst++, vec4);
} while (n >= 0);
if (n <= -16) return;
// dealing with residuals
pSrc += n; // rewind pointers
pDst += n;
} // iterate for one last time
}
if (n & 8)
{
vec1 = vld1q_s32(pSrc++);
vec2 = vld1q_s32(pSrc++);
vec1 = vqsubq_s32(vec1, one);
vec2 = vqsubq_s32(vec2, one);
vec1 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec1, 31);
vec2 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec2, 31);
vst1q_s32(pDst++, vec1);
vst1q_s32(pDst++, vec2);
}
if (n & 4)
{
vec1 = vld1q_s32(pSrc++);
vec1 = vqsubq_s32(vec1, one);
vec1 = (int32x4_t) vshrq_n_u32((uint32x4_t) vec1, 31);
vst1q_s32(pDst++, vec1);
}
n &= 3;
for (i = 0; i < n; ++i) {
a = *pSrc++;
if (a > 0) a = 0; else a = 1;
*pDst++ = a;
}
}
Now this one should be a lot faster than the previous ones since virtually all the latencies are hidden (more than four times as fast), provided the pathetic compilers don't mess it up.