MPI_Send works only with statically allocated buffer - c

If I want to define my own type, and use it as a datatype with MPI_Send to take only even rows from a matrix, does that matrix (send buffer) have to be allocated statically?
I seem to have problems when I allocate it dynamically. Is this because addresses need to be successive for data to be sent?

No, memory to be sent with MPI_Send does not have to be statically allocated.
To send array subsets, you likely want to use MPI_Type_indexed. Here is a slightly modified version of the example from the mpi.deino.net article on MPI_Type_indexed, where I have replaced the statically allocated buffer
int buffer[27];
to a dynamically allocated buffer
int* buffer = (int*)malloc(27 * sizeof(int));
I hope it helps:
#include <mpi.h>
#include <stdio.h>
int main(int argc, char *argv[])
{
int rank, size, i;
MPI_Datatype type, type2;
int blocklen[3] = { 2, 3, 1 };
int displacement[3] = { 0, 3, 8 };
int* buffer = (int*)malloc(27 * sizeof(int)); //int buffer[27];
MPI_Status status;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &size);
if (size < 2)
{
printf("Please run with 2 processes.\n");
MPI_Finalize();
return 1;
}
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Type_contiguous(3, MPI_INT, &type2);
MPI_Type_commit(&type2);
MPI_Type_indexed(3, blocklen, displacement, type2, &type);
MPI_Type_commit(&type);
if (rank == 0)
{
for (i=0; i<27; i++)
buffer[i] = i;
MPI_Send(buffer, 1, type, 1, 123, MPI_COMM_WORLD);
}
if (rank == 1)
{
for (i=0; i<27; i++)
buffer[i] = -1;
MPI_Recv(buffer, 1, type, 0, 123, MPI_COMM_WORLD, &status);
for (i=0; i<27; i++)
printf("buffer[%d] = %d\n", i, buffer[i]);
fflush(stdout);
}
MPI_Finalize();
free(buffer);
return 0;
}

Related

Can you send an array within an array using MPI_Send and MPI_Recv?

This is the very basic function of my program, and as such is not necessarily reproducible. However, I was wondering if there is a way to send an array of arrays using MPI? Or is this something that is not possible and I should flatten my array? Any help would be greatly appreciated as I've been struggling with trying to figure this out.
int *individual_topIds;
int **cell_topIds;
cell_topIds = (int**) malloc(sizeof(int*)*25*boxes);
if(rank == 0) {
for (int i = 0; i < boxes; i++) {
individual_topIds = (int*) malloc(sizeof(int)*25);
for(int j = 0; j < cellMatrix[i].numTop; j++){
individual_topIds[j] = cellMatrix[i].aTopIds[j];
}
cell_topIds[i] = individual_topIds;
}
MPI_Send(cell_topIds, boxes*25, MPI_INT, 1, 10, MPI_COMM_WORLD);
}
Then in my rank == 1 section. I have tried send and receive with just boxes, and not boxes*25 as well.
for 1 -> boxes
MPI_Recv(cell_topIds, boxes*25, MPI_INT, 0, 10, MPI_COMM_WORLD, &status);
int *ptop;
ptop = (int*) malloc(sizeof(int)*25);
ptop = cell_topIds[i];
printf("1\n");
for(int j = 0; j < sizeof(&ptop)/sizeof(int); j++){
printf("%d, ", ptop[j]);
}
printf("2\n");
end for i -> boxes
free(ptop);
Edit: Forgot to mention that the output of the print is a seg fault
Caught error: Segmentation fault (signal 11)
This is not a particularly well-worded question.
However, MPI will let you send arrays of arrays if you use a custom type, as below:
#include "mpi.h"
#include <stdio.h>
struct Partstruct
{
char c;
double d[6];
char b[7];
};
int main(int argc, char *argv[])
{
struct Partstruct particle[1000];
int i, j, myrank;
MPI_Status status;
MPI_Datatype Particletype;
MPI_Datatype type[3] = { MPI_CHAR, MPI_DOUBLE, MPI_CHAR };
int blocklen[3] = { 1, 6, 7 };
MPI_Aint disp[3];
MPI_Init(&argc, &argv);
disp[0] = &particle[0].c - &particle[0];
disp[1] = &particle[0].d - &particle[0];
disp[2] = &particle[0].b - &particle[0];
MPI_Type_create_struct(3, blocklen, disp, type, &Particletype);
MPI_Type_commit(&Particletype);
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
if (myrank == 0)
{
MPI_Send(particle, 1000, Particletype, 1, 123, MPI_COMM_WORLD);
}
else if (myrank == 1)
{
MPI_Recv(particle, 1000, Particletype, 0, 123, MPI_COMM_WORLD, &status);
}
MPI_Finalize();
return 0;
}
Alternatively, use a flat array design (this is a good idea for performance reasons as well as easy compatibility with MPI).

How to send a integer array via MPI_Send?

I'm trying to create a program in regular C that divides an integer array equally between any amount of process. For debugging purposes I'm using an integer array with 12 numbers and only 2 process so that the master process will have [1,2,3,4,5,6] and the slave1 will have [7,8,9,10,11,12]. However I'm getting an error saying: MPI_ERR_BUFFER: invalid buffer pointer.
After some research I found out that there is a function that does that (MPI_Scatter). Unfortunately, since I'm learning MPI the implementation is restricted to MPI_Send and MPI_Recv only. Anyway, both MPI_Send and MPI_Recv use a void*, and I'm sending a int* so it should work. Can anyone point out what am I doing wrong? Thank you.
int* create_sub_vec(int begin, int end, int* origin);
void print(int my_rank, int comm_sz, int n_over_p, int* sub_vec);
int main(void){
int comm_sz;
int my_rank;
int vec[12] = {1,2,3,4,5,6,7,8,9,10,11,12};
int* sub_vec = NULL;
int n_over_p;
MPI_Init(NULL, NULL);
MPI_Comm_size(MPI_COMM_WORLD, &comm_sz);
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
n_over_p = 12/comm_sz;
printf("Process %d calcula n_over_p = %d\n", my_rank, n_over_p);
if (my_rank != 0) {
MPI_Recv(sub_vec, n_over_p, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
print(my_rank, comm_sz, n_over_p, sub_vec);
} else {
printf("Distribuindo dados\n");
for (int i = 1; i < comm_sz; i++) {
sub_vec = create_sub_vec(i*n_over_p, (i*n_over_p)+n_over_p, vec);
MPI_Send(sub_vec, n_over_p, MPI_INT, i, 0, MPI_COMM_WORLD);
}
printf("Fim da distribuicao de dados\n");
sub_vec = create_sub_vec(0, n_over_p, vec);
print(my_rank, comm_sz, n_over_p, sub_vec);
}
MPI_Finalize();
return 0;
}
int* create_sub_vec(int begin, int end, int* origin){
int* sub_vec;
int size;
int aux = 0;
size = end - begin;
sub_vec = (int*)malloc(size * sizeof(int));
for (int i = begin; i < end; ++i) {
*(sub_vec+aux) = *(origin+i);
aux += 1;
}
return sub_vec;
}
void print(int my_rank, int comm_sz, int n_over_p, int* sub_vec){
printf("Process %d out of %d received sub_vecotr: [ ", my_rank, comm_sz);
for (int i = 0; i < n_over_p; ++i)
{
printf("%d, ", *(sub_vec+i));
}
printf("]\n");
}
The issue is that sub_vec is not allocated on non zero rank.
It is up to you to do that (e.g. MPI does not allocate the receive buffer).
the receive part should look like
if (my_rank != 0) {
sub_vec = (int *)malloc(n_over_p * sizeof(int));
MPI_Recv(sub_vec, n_over_p, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
}
As you wrote, the natural way is via MPI_Scatter() (and once again, it is up to you to allocate the receive buffer before starting the scatter.

Portability of sending values stored in char array using MPI

If I have a char array that represents, for example, integer value(s), and I used it to send these values via MPI with the appropriate MPI datatype for send and receive operations as follows:
int main(int argc, char* argv[]){
int my_rank; /* rank of process */
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
unsigned char buff[100];
if (my_rank == 0){
int n = 99;
int i;
for(i = 0; i < sizeof(n); i++){
buff[i] = (n >> (8 * i)) & 0xFF;
}
MPI_Send(&buff, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
}
else{
MPI_Recv(&buff, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, NULL);
int n = *(int *)buff;
printf("%d\n", n);
}
/* shut down MPI */
MPI_Finalize();
return 0;
}
Is this portable across machines of different architecture/endiness?
I suspect the only part that is not portable is the conversion from integer value to char array:
int i;
for(i = 0; i < sizeof(n); i++){
buff[i] = (n >> (8 * i)) & 0xFF;
}
But, anyways if not, is there a way to make the above program fully portable with the existence of the char array to store value(s)?
The way you serialize integers into a buffer is not portable. But if you are sending the integer, why not send integers directly, just take endianness into account. Suppose you want to sent 32bits integers:
int n = 99;
int sent = htonl(n);
MPI_Send(&send, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
For receiving:
int n;
int recv;
MPI_Recv(&recv, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, NULL);
n = ntonl(recv);

Basic Matrix operation with dynamical array allocation using MPI

I have already looked for answers about MPI and dynamic allocation, but there is still an error in my code.
I think the pairs send/receive work well. The problem is probably due to the identical part when I want to do some basic operations. I can't specify indices of the array, otherwise I get this error:
[lyomatnuc09:07574] * Process received signal *
[lyomatnuc09:07575] * Process received signal *
[lyomatnuc09:07575] Signal: Segmentation fault (11)
[lyomatnuc09:07575] Signal code: Address not mapped (1)
[lyomatnuc09:07575] Failing at address: 0x60
The basic code that reproduce the error is below :
int **alloc_array(int rows, int cols) {
int *data = (int *)malloc(rows*cols*sizeof(int));
int **array= (int **)malloc(rows*sizeof(int*));
for (int i=0; i<rows; i++)
array[i] = &(data[cols*i]);
return array;
}
int main(int argc, char *argv[])
{
MPI_Init(&argc, &argv); //initialize MPI operations
MPI_Comm_rank(MPI_COMM_WORLD, &rank); //get the rank
MPI_Comm_size(MPI_COMM_WORLD, &size); //get number of processes
MPI_Datatype columntype;
MPI_Type_vector(10, 1, 10, MPI_INT, &columntype);
MPI_Type_commit(&columntype);
start_time = MPI_Wtime();
if (rank == 0)
{
int **A;
A = alloc_array(10,10);
for ( int i =1 ;i<size;i++)
{
MPI_Send(&(A[0][0]), 10*10, MPI_INT, i, 1, MPI_COMM_WORLD);
}
} else if (rank >= 1) {
int **A2;
A2 = alloc_array(10,10);
MPI_Recv(&(A2[0][0]), 10*10, MPI_INT, 0, 1, MPI_COMM_WORLD, &status);
for (int i =0; i<10; i++)
{
for ( int j=0; j<10;i++)
{
A2[i][j]=i*j;//bug here
}
}
}//end slaves task
MPI_Finalize();
return 0;
}

MPI type matching

I was wondering why this program actually works in MPI (openMPI 1.5/1.6. )
#include <stdio.h>
#include <mpi.h>
#define VECTOR_SIZE 100
int main(int argc,char ** argv) {
int A[VECTOR_SIZE];
int sub_size=2;
int count=10;
MPI_Datatype partial_array;
int rank,size;
MPI_Status status;
MPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Comm_size(MPI_COMM_WORLD,&size);
MPI_Type_vector(count, sub_size,
2*sub_size, MPI_INT, &partial_array);
MPI_Type_commit(&partial_array);
if (rank == 0) {
int i;
// server - initialize data and send
for (i = 0; i< VECTOR_SIZE; i++) {
A[i] = i;
}
MPI_Send(&(A[0]), 1, partial_array, 1, 0, MPI_COMM_WORLD);
} else if (rank==1) {
int i;
for (i = 0; i< VECTOR_SIZE; i++) {
A[i] = 0;
}
// vector is composed by 20 MPI_INT elements
MPI_Recv(&(A[0]),20, MPI_INT, 0, 0, MPI_COMM_WORLD, &status);
printf("\n");
for (i = 0; i<VECTOR_SIZE; i++) {
printf("%d ",A[i]);
}
printf("\n");
}
MPI_Finalize();
}
while this other program where Send and Receive primitives are exchanged does not terminate (the receive never completes):
#include <stdio.h>
#include <mpi.h>
#define VECTOR_SIZE 100
int main(int argc,char ** argv) {
int A[VECTOR_SIZE];
int sub_size=2;
int count=10;
MPI_Datatype partial_array;
int rank,size;
MPI_Status status;
MPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Comm_size(MPI_COMM_WORLD,&size);
MPI_Type_vector(count, sub_size,
2*sub_size, MPI_INT, &partial_array);
MPI_Type_commit(&partial_array);
if (rank == 0) {
int i;
// server - initialize data and send
for (i = 0; i< VECTOR_SIZE; i++) {
A[i] = i;
}
MPI_Send(&(A[0]),20, MPI_INT, 0, 0, MPI_COMM_WORLD);
} else if (rank==1) {
int i;
// client - receive data and print
for (i = 0; i< VECTOR_SIZE; i++) {
A[i] = 0;
}
MPI_Recv(&(A[0]), 1, partial_array, 1, 0, MPI_COMM_WORLD, &status);
printf("\n");
for (i = 0; i<VECTOR_SIZE; i++) {
printf("%d ",A[i]);
}
printf("\n");
}
MPI_Finalize();
}
If I understand MPI type mathing rules correctly neither of the two should complete.
Obviously in the second program rank 0 is sending to itself and rank 1 is expecting message also from itself:
MPI_Send(&(A[0]),20, MPI_INT, 0, 0, MPI_COMM_WORLD);
destination rank should be 1, not 0
MPI_Recv(&(A[0]), 1, partial_array, 1, 0, MPI_COMM_WORLD, &status);
source rank should be 0, not 1.
Otherwise you do not understand the MPI type matching correctly. It only states that underlying primitive types in the type maps on both ends should match. You are creating a vector whose type map has 20 primitive integers. If you send one element of this type, your message will actually contain 20 integers. On the receiver side you provide space for at least 20 integers so this is correct. The opposite is also correct.
It is not correct if you send only 10 or 18 integers in the second program since they will not make a complete element of the vector type. Nevertheless, the receive operation will complete but if you call MPI_Get_count() on the status, if will return MPI_UNDEFINED because from the number of received primitive integer elements one cannot construct an integer number of vector elements. It is also not correct to mix primitive types, e.g. send MPI_DOUBLE (or vector, or structure, or whatever other type that has doubles) and receive it as MPI_INT.
Please also note that MPI messages do not carry their type map or type ID with them so most MPI implementations do not check if types match. It is possible to send MPI_FLOAT and receive it as MPI_INT (because both are 4 bytes on most systems) but it is not correct to do so.

Resources