Efficient usage of MPI_Allreduce - c

Question

I have on every processor a list range with numbers. I want to determine the maximum number of every row of these lists range.
The first four lists range for every processor P0-P3. The red list contains the maximum values of each row which every processor after MPI_Allreduce gets.
Here is a working version of my code:
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <mpi.h>
//#define KEY_MAX 100
typedef struct{
int myrank;
int numprocs;
int *range;
} SubDomainKeyTree;
void compRange(SubDomainKeyTree *s, int myrank, int numprocs){
s->myrank = myrank;
s->numprocs = numprocs;
// Allocate memory for (numprocs+1) ranges
s->range = malloc((numprocs+1) * sizeof(int));
// Compute range values
for(int p=0; p<=numprocs; p++){
s->range[p] = rand()%100;
}
for(int p=0; p<s->numprocs; p++){
if(s->myrank == p){
for(int k=0; k<=s->numprocs; k++){
printf("Processor %d: %d random number is %d\n", p, k, s->range[k]);
}
printf("\n");
}
}
}
void compDynRange(SubD *s){
int rangeForAll[s->numprocs+1];
//////////////////////////////////
// This is not really efficient //
//////////////////////////////////
for(int r=0; r<=s->numprocs; r++){
MPI_Allreduce(&s->range[r], &rangeForAll[r], 1, MPI_INT, MPI_MAX, MPI_COMM_WORLD);
}
for(int p=0; p<s->numprocs; p++){
if(s->myrank == p){
for(int k=0; k<=s->numprocs; k++){
s->range[k] = rangeForAll[k];
printf("Processor %d: %d random number after MPI_Allreduce is %d\n", p, k, s->range[k]);
}
printf("\n");
}
}
}
int main(int argc, char **argv){
int nameLen;
char processorName[MPI_MAX_PROCESSOR_NAME];
int myrank; // Rank of processor
int numprocs; // Number of processes
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Get_processor_name(processorName,&nameLen);
MPI_Status status;
time_t t;
srand((unsigned)time(NULL)+myrank*numprocs+nameLen);
SubD s;
compRange(&s, myrank, numprocs);
compDynRange(&s);
MPI_Finalize();
return 0;
}
I use a for-loop which seems highly inefficient to me. Here I compute the maximum value of every row of all lists one after the other.
But can I use MPI_Allreduce without that for-loop?
I already tried that instead of the for-loop which does not work.
MPI_Allreduce(&s->range, &rangeForAll, s->numprocs+1, MPI_INT, MPI_MAX, MPI_COMM_WORLD);
Can someone give me a hint how I can do that?

Answer 1

As already hinted in comment, the error you had in you code was that instead of passing the arrays containing your send and receive buffers, you were passing some pointers to them. I imagine that this error was simply coming fro the change from a single element used initially (like &s->range[r]) which was perfectly correct, to the full array by just removing the indexed access (ie &s->range) which was wrong.
So as explained, using:
MPI_Allreduce(s->range, rangeForAll, s->numprocs+1, MPI_INT, MPI_MAX, MPI_COMM_WORLD)
just does the trick. However, since you want to get the results into the s->range arrays rather than the temporary rangeFarAll ones, you'd better off not defining the later at all, and use the MPI_IN_PLACE keyword as sending parameter and s->range as receiving one. The call becomes:
MPI_Allreduce(MPI_IN_PLACE, s->range, s->numprocs+1, MPI_INT, MPI_MAX, MPI_COMM_WORLD)
and s->range acts both as sending and receiving buffer. Therefore, the final results will all be in the s->range buffers after the call, sparing you the need of doing the copy explicitly.