difficulty understanding MPI scatter and gather in C - c

Question

I am trying to learn to use MPI. Below is my simple program to test MPI scatter and gather. I don't understand how it works and why it produces the result
1 2 3 4 4 5 6 7 8 9 10 11
instead of expected
1 2 3 4 5 6 7 8 9 10 11 12
The documentation and all the examples I can find are too complicated/poorly worded for me to understand. I just want to scatter an array across 3 processes and add one to each value in each process. Alternatively I would be happy to see how a 2D array was sent row by row to each process and each row was processed simply.
int main(int argc, char **argv) {
int rank; // my process ID
int size = 3; // number of processes/nodes
MPI_Status status;
MPI_Init(&argc, &argv); // start MPI
MPI_Comm_size(MPI_COMM_WORLD, &size); // initialize MPI
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
unsigned char inData[12]; // data returned after being "processed"
unsigned char outData[12]; // buffer for receiving data
unsigned long datasize = 12; // size of data to process
unsigned char testData[12]; // data to be processed
if (rank == 0) {
// initialize data
for (int i = 0; i < datasize; i++) {
testData[i] = i;
outData[i] = 0;
inData[i] = 0;
}
}
// scatter the data to the processes
// I am not clear about the numbers sent in and out
MPI_Scatter(&testData, 12, MPI_UNSIGNED_CHAR, &outData,
12, MPI_UNSIGNED_CHAR, 0, MPI_COMM_WORLD);
MPI_Barrier(MPI_COMM_WORLD);
// process data
for (int i = 0; i < 4; i++) { outData[i] = outData[i] + 1; }
MPI_Barrier(MPI_COMM_WORLD);
// gather processed data
MPI_Gather(&outData, 12, MPI_UNSIGNED_CHAR, &inData,
12, MPI_UNSIGNED_CHAR, 0, MPI_COMM_WORLD);
//print processed data from root
if (rank == 0) {
for (int i = 0; i < 12; i++) {
printf("\n%d", inData[i]);
}
MPI_Finalize();
}
return 0;
}

Answer 1

Though your main error is using 12 instead of 4, let's do it step-by-step.
// int size = 3; // number of processes/nodes
int size;
...
MPI_Comm_size(MPI_COMM_WORLD, &size); // initialize MPI
assert(size == 3);
There is no point in setting size to 3. This value will be overwritten by MPI_Comm_size with the actual number of processes. This number is determined by how you run your MPI application (e.g. mpirun -np 3).
//unsigned char outData[12]; // buffer for receiving data
unsigned char outData[4];
We have 12 elements and 3 processes, 4 elements per processes. So, 4 elements are enough for outData.
outData[i] = 0;
inData[i] = 0;
There is no point in zeroing these buffers, they will be overwritten.
// scatter the data to the processes
// I am not clear about the numbers sent in and out
MPI_Scatter(&testData, 4 /*12*/, MPI_UNSIGNED_CHAR, &outData,
4 /*12*/, MPI_UNSIGNED_CHAR, 0, MPI_COMM_WORLD);
We have 4 elements per processes, so the number should be 4, not 12.
MPI_Barrier(MPI_COMM_WORLD);
You don't need barriers here.
MPI_Gather(&outData, 4 /*12*/, MPI_UNSIGNED_CHAR, &inData,
4 /*12*/, MPI_UNSIGNED_CHAR, 0, MPI_COMM_WORLD);
Same story, 4 instead of 12.
MPI_Finalize();
This should be called by all processes.