Benchmarking, sequential x parallel program. Sublinear speedup? - c

Question

Update2. Solved! This is memory issue. Some benching about it here:
http://dontpad.com/bench_mem
Update. My goal is to achieve best throughput. All my results are here.
Sequential Results:
https://docs.google.com/spreadsheet/ccc?key=0AjKHxPB2qgJXdE8yQVNHRkRiQ2VzeElIRWwxMWtRcVE&usp=sharing
Parallel Results*:
https://docs.google.com/spreadsheet/ccc?key=0AjKHxPB2qgJXdEhTb2plT09PNEs3ajBvWUlVaWt0ZUE&usp=sharing
multsoma_par1_vN, N determines how data is acessed by each thread.
N: 1 - NTHREADS displacement, 2 - L1 displacement, 3 - L2 displacement, 4 - TAM/NTHREADS
I am having a hard time trying to figure out why my parallel code runs just slighty faster than sequential code.
What I basically do is to loop through a big array (10^8 elements) of a type (int/float/double) and apply the computation: A = A * CONSTANT + B. Where A and B are arrays of same size.
Sequential code only do a single function call.
Parallel version create pthreads and uses the same function as starting function.
I am using gettimeofday(), RDTSC() and more recently getrusage() to measure timings. My main results are expressed by Clocks per Element (CPE).
My processor is an i5-3570K. 4 Cores, no hyper-threading.
The problem is that I can get up to 2.00 CPE under sequential code and when going parallel my best performance was 1.84 CPE. I know that I get an overhead by creating pthreads and calling more timing routines, but I don't think this is the reason for not getting better timings.
I did measured each thread CPE and executed the program with 1, 2, 3 and 4 threads. When creating only one thread, I get the expected result CPE around 2.00 (+ some overhead expressed in miliseconds but overall CPE is not affected at all).
When running with 2 threads or more the main CPE decreases, but each thread CPE increases.
2 threads I get main CPE around 1.9 and each thread to 3.8 ( Why this is not 2.0 ?! )
The same happens to 3 and 4 threads.
4 threads I get main CPE around 1.85 (my best timing) and each thread with 7.0~7.5 CPE.
Using many threads more than avaiable cores(4) I still getting CPE under 2.0 but not better than 1.85 (most times higher due to overhead).
I suspect that maybe context switching could be the limiting factor here. When running with 2 threads I can count 5 to 10 involuntary contexts switch from each thread...
But I am not so sure about this. Are those seemly few context switches enough to almost double my CPE ? I was expecting to atleast get around 1.00 CPE using all my CPU Cores.
I went further on this and analyzed the assembly code for this function. They are identical, except for some extra shifts and adds (4 instructions) at the very beginning of the function and they are out of loops.
In case you want to see some code:
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <sys/time.h>
#include <cpuid.h>
typedef union{
unsigned long long int64;
struct {unsigned int lo, hi;} int32;
} tsc_counter;
#define RDTSC(cpu_c) \
__asm__ __volatile__ ("rdtsc" : \
"=a" ((cpu_c).int32.lo), \
"=d" ((cpu_c).int32.hi) )
#define CNST 5
#define NTHREADS 4
#define L1_SIZE 8096
#define L2_SIZE 72512
typedef int data_t;
data_t * A;
data_t * B;
int tam;
double avg_thread_CPE;
tsc_counter thread_t0[NTHREADS], thread_t1[NTHREADS];
struct timeval thread_sec0[NTHREADS], thread_sec1[NTHREADS];
void fillA_B(int tam){
int i;
for (i=0;i<tam;i++){
A[i]=2; B[i]=2;
}
return;
}
void* multsoma_par4_v4(void *arg){
int w;
int i,j;
int *id = (int *) arg;
int limit = tam-14;
int size = tam/NTHREADS;
int tam2 = ((*id+1)*size);
int limit2 = tam2-14;
gettimeofday(&thread_sec0[*id],NULL);
RDTSC(thread_t0[*id]);
//Mult e Soma
for (i=(*id)*size;i<limit2 && i<limit;i+=15){
A[i] = A[i] * CNST + B[i];
A[i+1] = A[i+1] * CNST + B[i+1];
A[i+2] = A[i+2] * CNST + B[i+2];
A[i+3] = A[i+3] * CNST + B[i+3];
A[i+4] = A[i+4] * CNST + B[i+4];
A[i+5] = A[i+5] * CNST + B[i+5];
A[i+6] = A[i+6] * CNST + B[i+6];
A[i+7] = A[i+7] * CNST + B[i+7];
A[i+8] = A[i+8] * CNST + B[i+8];
A[i+9] = A[i+9] * CNST + B[i+9];
A[i+10] = A[i+10] * CNST + B[i+10];
A[i+11] = A[i+11] * CNST + B[i+11];
A[i+12] = A[i+12] * CNST + B[i+12];
A[i+13] = A[i+13] * CNST + B[i+13];
A[i+14] = A[i+14] * CNST + B[i+14];
}
for (; i<tam2 && i<tam; i++)
A[i] = A[i] * CNST + B[i];
RDTSC(thread_t1[*id]);
gettimeofday(&thread_sec1[*id],NULL);
double CPE, elapsed_time;
CPE = ((double)(thread_t1[*id].int64-thread_t0[*id].int64))/((double)(size));
elapsed_time = (double)(thread_sec1[*id].tv_sec-thread_sec0[*id].tv_sec)*1000;
elapsed_time+= (double)(thread_sec1[*id].tv_usec - thread_sec0[*id].tv_usec)/1000;
//printf("Thread %d workset - %d\n",*id,size);
//printf("CPE Thread %d - %lf\n",*id, CPE);
//printf("Time Thread %d - %lf\n",*id, elapsed_time/1000);
avg_thread_CPE+=CPE;
free(arg);
pthread_exit(NULL);
}
void imprime(int tam){
int i;
int ans = 12;
for (i=0;i<tam;i++){
//printf("%d ",A[i]);
//checking...
if (A[i]!=ans) printf("WA!!\n");
}
printf("\n");
return;
}
int main(int argc, char *argv[]){
tsc_counter t0,t1;
struct timeval sec0,sec1;
pthread_t thread[NTHREADS];
double CPE;
double elapsed_time;
int i;
int* id;
tam = atoi(argv[1]);
A = (data_t*) malloc (tam*sizeof(data_t));
B = (data_t*) malloc (tam*sizeof(data_t));
fillA_B(tam);
avg_thread_CPE = 0;
//Start Computing...
gettimeofday(&sec0,NULL);
RDTSC(t0); //Time Stamp 0
for (i=0;i<NTHREADS;i++){
id = (int*) malloc(sizeof(int));
*id = i;
if (pthread_create(&thread[i], NULL, multsoma_par4_v4, (void*)id)) {
printf("--ERRO: pthread_create()\n"); exit(-1);
}
}
for (i=0; i<NTHREADS; i++) {
if (pthread_join(thread[i], NULL)) {
printf("--ERRO: pthread_join() \n"); exit(-1);
}
}
RDTSC(t1); //Time Stamp 1
gettimeofday(&sec1,NULL);
//End Computing...
imprime(tam);
CPE = ((double)(t1.int64-t0.int64))/((double)(tam)); //diferenca entre Time_Stamps/repeticoes
elapsed_time = (double)(sec1.tv_sec-sec0.tv_sec)*1000;
elapsed_time+= (double)(sec1.tv_usec - sec0.tv_usec)/1000;
printf("Main CPE: %lf\n",CPE);
printf("Avg Thread CPE: %lf\n",avg_thread_CPE/NTHREADS);
printf("Time: %lf\n",elapsed_time/1000);
free(A); free(B);
return 0;
}
I appreciate any help.

Answer 1

After seeing the full code, I rather agree with the guess of #nosid in comments: since the ratio of compute operations to memory loads is low, and the data (about 800M if I am not mistaken) don't fit in cache, the memory bandwidth is likely the limiting factor. The link to the main memory is shared to all cores in a processor, so when its bandwidth is saturated, all memory operations start stalling and take longer time; thus CPE increases.
Also, the following place in your code is a data race:
avg_thread_CPE+=CPE;
as you sum up CPE values calculated on different threads to a single global variable without any synchronization.
Below I left the part of my initial answer, including the "first statement" referred in the comments. I still find it correct, for the definition of CPE as the number of clocks taken by the operations over a single element.
You should not expect the clocks per element (CPE) metric to decrease
due to use of multiple threads. By definition, it's how fast a
single data item is processed, in average. Threading helps to process all data faster (by simultaneous processing on different
cores), so the elapsed wallclock time, i.e. the time to execute the
whole program, should be expected to decrease.