Send variable length vector with constant header in one message - c

Question

I have n workers, 1 master (rank 0) and need to send messages via MPI from the n workers to the master. The message format is a variable length vector (float *dta) and constant size header struct { int32_t x, int32_t y } dtaHdr.
The master just loops through the incoming results and processes them. It is important for it to be able to associate which dtaHdrgoes with which dta.
I know how to:
Create MPI_Datatype for the constant sized dtaHdr and send it via P2P MPI_Send/MPI_Recv.
Send a variable length vector of any datatype (e.g. MPI_Float) via P2P MPI_Send/MPI_Recv.
The issue is I have no idea how to combine these two approaches.
I know that I could:
Send header first and then the data second in two separate messages.
That has the problem of reordering & interleaving of messages. I need a reliable, easy, and scalable way to associate the header and its data on master. With two messages I can't see a way how to always & simply get both a header and data for incoming message on the master. I.e. two workers could send hdr & data messages to master & they could become interleaved. (I'm not sure what the guarantees for ordering are TBH, even after reading MPI specification).
Encode both the header and the data to a MPI_Byte array and send it esseentially as a binary blob.
Sounds really dirty & breaks some guarantees.
My question is: how do I MPI-idiomatically send one identifiable logical message that contains both a constant sized header of one type and a variable sized vector of some second type.

Answer 1

This program uses MPI_Pack and MPI_Unpack to send two different types in the same message:
#include <mpi.h>
#include <stdio.h>
#define ARRAY_SIZE(array) \
(sizeof(array) / sizeof(array[0]))
struct data_header {
int32_t x;
int32_t y;
};
MPI_Datatype dt_header;
MPI_Datatype dt_vector;
void sendmsg(void) {
struct data_header header = { 1, 2 };
float example[] = { 1.0, 2.0, 3.0, 4.0 };
char buffer[4096];
int position;
MPI_Pack(&header, 1, dt_header, buffer, sizeof(buffer), &position, MPI_COMM_WORLD);
MPI_Pack(example, 1, dt_vector, buffer, sizeof(buffer), &position, MPI_COMM_WORLD);
MPI_Send(buffer, position, MPI_PACKED, 0, 0, MPI_COMM_WORLD);
}
void recvmsg(void) {
struct data_header header;
float example[4];
char buffer[4096];
int position = 0;
MPI_Recv(buffer, sizeof(buffer), MPI_PACKED, 1, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
MPI_Unpack(buffer, sizeof(buffer), &position, &header, 1, dt_header, MPI_COMM_WORLD);
MPI_Unpack(buffer, sizeof(buffer), &position, example, 1, dt_vector, MPI_COMM_WORLD);
printf("x = %d, y = %d\n", header.x, header.y);
for (int index = 0; index < ARRAY_SIZE(example); index++) {
printf("%f ", example[index]);
}
printf("\n");
}
int main(void) {
int world_size;
int world_rank;
MPI_Init(NULL, NULL);
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
MPI_Type_contiguous(2, MPI_INT, &dt_header);
MPI_Type_commit(&dt_header);
MPI_Type_contiguous(4, MPI_FLOAT, &dt_vector);
MPI_Type_commit(&dt_vector);
if (0 == world_rank) {
recvmsg();
}
else {
sendmsg();
}
MPI_Finalize();
return 0;
}
Output
x = 1, y = 2
1.000000 2.000000 3.000000 4.000000
This is really just proof-of-concept code. Hopefully, it will help you find the solution you are looking for.
Note
This code does no error checking and should not be used in a production environment.