Assertion failure using MPI_Gather - c

Question

I am trying to write MPI C code that repeatedly performs a calculation and saves its outcome into a single array for outputting less frequently. Example code below (the size of var, 200, is sufficient for the number of CPUs in use):
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
int main(int argc, char **argv){
float *phie, *phitemp, var[200];
int time=0, gatherphi=10, gatherfile = 200, j, iter=0, lephie, x;
int nelecrank = 2, size, rank, Tmax = 2000;
FILE *out;
MPI_Init(&argc, &argv) ;
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
lephie = gatherfile/gatherphi; // number of runs of calculation before output
// allocate memory
//printf("%d Before malloc.\n", rank);
if (rank==1) phie=(float *) malloc(nelecrank*size*gatherfile/gatherphi*sizeof(float));
phitemp=(float *) malloc(nelecrank*sizeof(float));
//printf("%d After malloc.\n", rank);
for(j=0;j<200;j++) var[j]=rank;
for(time=0;time<Tmax;time++){
if (!time%gatherphi) {// do calculation
for (j=0;j<nelecrank;j++) { // each processor does the calculation nelecrank times
phitemp[j]=0;
for (x = 0; x<=4; x++) {
phitemp[j]=phitemp[j]+var[rank+j*size];
}
}
} // end of j for loop
printf("iter: %d, %d Before MPI_Gather.\n", iter, rank);
MPI_Gather(phitemp, nelecrank, MPI_FLOAT, phie+iter*nelecrank*size*sizeof(float), nelecrank, MPI_FLOAT, 1, MPI_COMM_WORLD);
iter++;
} // end of gatherphi condition
if (time % gatherfile) { //output result of calculation
iter=0;
if (rank==1) {
out = fopen ("output.txt", "wt+");
if (out == NULL) {
printf("Could not open output file.\n");
exit(1);
}
else printf("Have opened output file.\n");
for (j=0;j<nelecrank*size*lephie;j++) {
fprintf(out,"%f ",*(phie+j*sizeof(float)));
}
fclose(out);
}
} // end of file output
if (rank==1) {
if (phie) free (phie);
}
if (phitemp) free (phitemp);
MPI_Finalize();
return 0;
}
It gives me repeated memory allocation problems until it finally exits. I am not experienced using memory allocation in MPI - can you help?
Many thanks,
Marta

Answer 1

Basically, phie is not big enough.
You malloced nelecrank*size*gatherfile/gatherphi*sizeof(float)=80*sizeof(float) memory for phie.
But, your MPI_Gather usage requires iter*nelecrank*size*sizeof(float)+nelecrank*size*sizeof(float) memory. iter takes a maximum value of nelecrank*Tmax-1, so phie needs to be (nelecrank*Tmax-1)*nelecrank*size*sizeof(float)+nelecrank*size*sizeof(float), which is around 8000*size*sizeof(float) large.
Maybe you want to reset iter=0 in your loop somewhere?