BAD TERMINATION in MPI Bsend - c

i'm trying send packed structure by MPI_Bsend(). Something i'm doing wrong and i cannot find solution.
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "mpi.h"
#define SIZE 10
struct car {
int id;
int vmax;
char marka[SIZE];
char model[SIZE];
};
int main(int argc, char **argv) {
int i;
int rank, size;
double t1, t2;
struct car BMW, BMW2;
BMW.id = 1;
strcpy(BMW.marka, "BMW");
strcpy(BMW.model, "szybki");
BMW.vmax = 199;
MPI_Status status;
MPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Comm_size(MPI_COMM_WORLD,&size);
int rozmiar, packet_size, msg_size, position = 0,tag;
void *bufor;
MPI_Pack_size(2, MPI_INT, MPI_COMM_WORLD, &rozmiar);
packet_size = rozmiar;
MPI_Pack_size(2 * SIZE, MPI_CHAR, MPI_COMM_WORLD, &rozmiar);
packet_size += rozmiar;
msg_size = 2 * packet_size + MPI_BSEND_OVERHEAD;
bufor = (void *)malloc(msg_size);
MPI_Buffer_attach(bufor, msg_size);
t1 = MPI_Wtime();
if (rank == 0) {
tag = 0;
for(i=1;i<size;i++){
MPI_Pack(&BMW.id,1, MPI_INT, bufor, msg_size, &position, MPI_COMM_WORLD);
MPI_Pack(&BMW.vmax,1, MPI_INT, bufor, msg_size, &position, MPI_COMM_WORLD);
MPI_Pack(&BMW.model,SIZE, MPI_CHAR, bufor, msg_size, &position, MPI_COMM_WORLD);
MPI_Pack(&BMW.marka,SIZE, MPI_CHAR, bufor, msg_size, &position, MPI_COMM_WORLD);
MPI_Bsend(bufor,position,MPI_PACKED,i,tag,MPI_COMM_WORLD);
}
} else {
MPI_Recv(bufor,msg_size,MPI_PACKED,0,MPI_ANY_TAG,MPI_COMM_WORLD,&status);
position = 0;
MPI_Unpack(bufor, msg_size, &position, &BMW2.id, 1, MPI_INT, MPI_COMM_WORLD);
MPI_Unpack(bufor, msg_size, &position, &BMW2.vmax, 1, MPI_INT, MPI_COMM_WORLD);
MPI_Unpack(bufor, msg_size, &position, &BMW2.model, SIZE, MPI_CHAR, MPI_COMM_WORLD);
MPI_Unpack(bufor, msg_size, &position, &BMW2.marka, SIZE, MPI_CHAR, MPI_COMM_WORLD);
printf("rank = %d | BMW id: %d, marka: %s, model: %s, vmax: %d \n",rank, BMW2.id, BMW2.marka, BMW2.model, BMW2.vmax);
}
t2 = MPI_Wtime();
MPI_Buffer_detach(&bufor, &msg_size);
MPI_Finalize();
if (i == size)
printf("Elapsed time is %.15f\n", t2 - t1 );
return(0);
}
Error:
====================================================================
BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
PID 25637 RUNNING AT debian
EXIT CODE: 11
================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault
(signal 11)

You are using the buffered mode of MPI incorrectly. The buffer you give to MPI via MPI_Buffer_attach is supposed to be used by MPI internally. Do not use the buffered MPI interface, it is very rarely useful and very difficult to get right.
Just remove the MPI_Buffer_ and use MPI_Send instead of MPI_Bsend and you are on the right track. MPI_Pack can be a bit clumsy, you may want to look insto custom datatypes (MPI_Type_create_struct) instead. If you have a homogeneous system, you can also send the raw bytes of the struct car.

Related

MPI send receive

#include "mpi.h"
#include <stdio.h>
int main(argc,argv)
int argc;
char *argv[]; {
int numtasks, rank, dest, source, rc, count, tag=1;
MPI_Status Stat;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
if (rank%2 == 1) {
dest = (rank+1)%numtasks;
source = (rank-1+numtasks)%numtasks;
rc = MPI_Send(&outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD);
rc = MPI_Recv(&inmsg, 1, MPI_CHAR, source, tag,MPICOMM_WORLD, &Stat);
}
else {
dest = (rank-1+numtasks)%numtasks;
source = (rank+1)%numtasks;
rc = MPI_Recv( &inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat );
rc = MPI_Send( &outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD );
}
rc = MPI_Get_count(&Stat, MPI_CHAR, &count);
printf("Task %d: Received %d char(s) from task %d with tag %d\n",
rank, count, Stat.MPI_SOURCE, Stat.MPI_TAG);
MPI_Finalize();
}
Results:
Task 2: Received 1 char(s) from task 3 with a tag 1
Task 0: Received 1 char(s) from task 1 with a tag 1
Why the task can't identify more than 2 processors?
I want run it on more than 2 processors.
I've updated the mpi program with the ring pattern, i think maybe wrong at the line if(rank%2==1)
May try to better investigate the MPI mechanics:
Do not hesitate to experiment with the code, as it will help you grasp the concept of numtasks, rank-ID and associated run-time operations. Compilation errors will report shortcuts in the above posted code. For more MPI-related practices and insights, one ought also assign and evaluate rc return codes, from the respective MPI_<fun>()-calls
Q: Why the task can't identify more than 2 processors?
A: It can, but the if(){..}else if(){..} code-blocks did not allow others to produce any visible output.
Q: I want to change that 4197005 result into 2, 3 - how?
A: One cannot change an MPI-reported number, but you can change the behaviour of your code ( see below ) and make some more outputs, where feasible about who is doing what, when, how and where it is actually going on. This way we learn to understand the MPI-concepts and validate the actual MPI-code execution.
Feel free to ask more.
#include "mpi.h"
#include <stdio.h>
int main( int argc,
char *argv[]
) {
int numtasks, rank, dest, source, rc, count, tag = 1;
MPI_Status Stat;
printf( "(INF) will MPI_Init()...\n" );
MPI_Init( &argc, &argv );
printf( "(INF) MPI_Init() done.\n" );
MPI_Comm_size( MPI_COMM_WORLD, &numtasks );
MPI_Comm_rank( MPI_COMM_WORLD, &rank );
printf( "(INF) I am MPI[%d of %d]\n",
rank,
numtasks
);
if ( rank == 0 ) {
dest = 1;
source = 1;
rc = MPI_Send( &outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD );
rc = MPI_Recv( &inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat );
printf( "(INF) I am MPI[%d of %d] having just completed an MPI_Send( ->MPI[%d] ) + MPI_Recv( <-MPI[%d] ). Last RetCODE == %d.\n",
rank,
numtasks,
dest,
source,
rc
);
}
else
if ( rank == 1 ) {
dest = 0;
source = 0;
rc = MPI_Recv( &inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat );
rc = MPI_Send( &outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD );
printf( "(INF) I am MPI[%d of %d] having just completed an MPI_Recv( <-MPI[%d] ) + MPI_Send( ->MPI[%d] ). Last RetCODE == %d\n",
rank,
numtasks,
source,
dest,
rc
);
}
else
if ( rank == 2 ) {
dest = 3;
source = 3;
rc = MPI_Recv( &inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat );
rc = MPI_Send( &outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD );
printf( "(INF) I am MPI[%d of %d] having just completed an MPI_Recv( <-MPI[%d] ) + MPI_Send( ->MPI[%d] ). Last RetCODE == %d\n",
rank,
numtasks,
source,
dest,
rc
);
}
else
if ( rank == 3 ) {
dest = 2;
source = 2;
rc = MPI_Send( &outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD );
rc = MPI_Recv( &inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat );
printf( "(INF) I am MPI[%d of %d] having just completed an MPI_Send( ->MPI[%d] ) + MPI_Recv( <-MPI[%d] ). Last RetCODE == %d\n",
rank,
numtasks,
dest,
source,
rc
);
}
else {
printf( "(INF) I am MPI[%d of %d] will NOP...\n",
rank,
numtasks
);
}
rc = MPI_Get_count( &Stat, MPI_CHAR, &count );
/*
Task 3: Received 1 char(s) from task 4197005 with a tag 0
Task 2: Received 1 char(s) from task 4197005 with a tag 0
Task 1: Received 1 char(s) from task 0 with a tag 1
Task 0: Received 1 char(s) from task 1 with a tag 1 */
printf( "Task %d: Received %d char(s) from task %d with tag %d\n",
rank,
count,
Stat.MPI_SOURCE,
Stat.MPI_TAG
);
printf( "(INF) I am MPI[%d of %d]: will MPI_Finalize()...\n",
rank,
numtasks
);
MPI_Finalize();
printf( "(INF) I am MPI[%d of %d]: MPI_Finalize() done, will exit()...\n",
rank,
numtasks
);
}
count and Stat are only initialized on ranks 0 and 1, so you are basically printing garbage on ranks 2 and 3.
Your program is hard-coded to run on 2 tasks (it will crash on 1 task, and print garbage with 3 tasks or more).
I suggest you investigate using MPI_Sendrecv() in order to send and receive deadlock-free in one MPI call.
Not sure what you are trying to achieve, please elaborate and update your question. That being said, you might consider a ring pattern so your program can run on any number of MPI tasks (e.g. send to (rank+1)%numtasks and receive from (rank+numtasks-1)%numtasks)
Per your edited question, the communication pattern can the same for all the MPI ranks as follow
dest = (rank+1)%numtasks;
source = (rank-1+numtasks)%numtasks;
MPI_Sendrecv(&outmsg, 1, MPI_CHAR, dest, tag,
&inmsg, 1, MPI_CHAR, source, tag,
MPI_COMM_WORLD, &Stat);
#include "mpi.h"
#include <stdio.h>
int main (argc,argv)
int argc;
char *argv[]; {
int numtasks, rank, dest, source, rc, count, tag=1;
char inmsg, outmsg='x';
MPI_Status Stat;
MPI_Init (&argc, &argv);
MPI_Comm_size (MPI_COMM_WORLD, &numtasks);
MPI_Comm_rank (MPI_COMM_WORLD, &rank);
if (rank%2 == 0)
{
dest=(rank+1)%numtasks;
source=(rank+1)%numtasks;
rc = MPI_Send(&outmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD);
rc = MPI_Recv(&inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat);
}
else {
dest = (rank-1+numtasks)%numtasks;
source = (rank-1+numtasks)%numtasks;
rc = MPI_Recv (&inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat);
rc = MPI_Send (&outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD);
}
{
rc = MPI_Get_count (&Stat, MPI_CHAR, &count);
printf ("Task %d: Received %d char(s) from task %d with a tag %d \n", rank, count, Stat.MPI_SOURCE, Stat.MPI_TAG);
}
MPI_Finalize();

What should be the value of message in MPI send-receive program?

I have developed a given simple MPI program such that process 0 sends message to process 1 and receives message from process p-1. Following is the code :
In the skeleton given to me ,
char *message;
message= (char*)malloc(msg_size);
is confusing me. To check the correctness of program, I am trying to look value of message that been sent or received. So should it be hexadecimal value?
int main(int argc, char **argv)
{
double startwtime, endwtime;
float elapsed_time, bandwidth;
int my_id, next_id; /* process id-s */
int p; /* number of processes */
char* message; /* storage for the message */
int i, k, max_msgs, msg_size, v;
MPI_Status status; /* return status for receive */
MPI_Init( &argc, &argv );
MPI_Comm_rank( MPI_COMM_WORLD, &my_id );
MPI_Comm_size( MPI_COMM_WORLD, &p );
if (argc < 3)
{
fprintf (stderr, "need msg count and msg size as params\n");
goto EXIT;
}
if ((sscanf (argv[1], "%d", &max_msgs) < 1) ||
(sscanf (argv[2], "%d", &msg_size) < 1))
{
fprintf (stderr, "need msg count and msg size as params\n");
goto EXIT;
}
**message = (char*)malloc (msg_size);**
if (argc > 3) v=1; else v=0; /*are we in verbose mode*/
/* don't start timer until everybody is ok */
MPI_Barrier(MPI_COMM_WORLD);
int t=0;
if( my_id == 0 ) {
startwtime = MPI_Wtime();
// do max_msgs times:
// send message of size msg_size chars to process 1
// receive message of size msg_size chars from process p-1
while(t<max_msgs) {
MPI_Send((char *) message, msg_size, MPI_CHAR, 1 , 0, MPI_COMM_WORLD);
MPI_Recv((char *) message, msg_size, MPI_CHAR, p-1, 0, MPI_COMM_WORLD, &status);
t++;
}
MPI_Barrier(MPI_COMM_WORLD);
endwtime = MPI_Wtime();
elapsed_time = endwtime-startwtime;
bandwidth = 2.0 * max_msgs * msg_size / (elapsed_time);
printf("Number, size of messages: %3d , %3d \n", max_msgs, msg_size);
fflush(stdout);
printf("Wallclock time = %f seconds\n", elapsed_time );
fflush(stdout);
printf("Bandwidth = %f bytes per second\n", bandwidth);
fflush(stdout);
} else if( my_id == p-1 ) {
// do max_msgs times:
// receive message of size msg_size from process to the left
// send message of size msg_size to process to the right (p-1 sends to 0)
while(t<max_msgs) {
MPI_Send((char *) message, msg_size, MPI_CHAR, 0, 0, MPI_COMM_WORLD);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
t++;
}
} else {
while(t<max_msgs) {
MPI_Send((char *) message, msg_size, MPI_CHAR, my_id+1, 0, MPI_COMM_WORLD);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
t++;
}
}
MPI_Barrier(MPI_COMM_WORLD);
EXIT:
MPI_Finalize();
return 0;
}
I am not completely sure if this is what you mean, but I will try.
For what I understand, you want to know what is the message being sent. Well, for the code you provide, memory is assign to the message but any real "readable" message is specify. In this line.
message = (char*)malloc (msg_size);
malloc reserves the memory for the messages, so anyone can write it, however, it doesn't provide any initial value. Sometimes, the memory contains other information previously stored and freed. Then, the message being sent is that "garbage" that is before. This is probably what you call hexadecimal (I hope I understand this right).
The type of value in this case is char (defined as MPI_CHAR in the MPI_Send and MPI_Recv functions). Here you can find more data types for MPI.
I will suggest to assign a value to the message with the with my_id and next_id. So you know who is sending to whom.

MPI program does not work as intended

I have developed one MPI program which sends and receives messages in cycle. The message passing is initiated by process 0: it sends to process 1, and receives from process p-1. The other processes receive from their left neighbor and send to their right neighbor. Notice that (p-1)-s right neighbor is 0.
Following is my code :
#include <stdio.h>
#include "mpi.h"
#include <stdlib.h>
/* cycle
In this program a message is sent around a circle of processes 0 - (p-1), ]
where p-1 sends to 0.
*/
int main(int argc, char **argv)
{
double startwtime, endwtime;
float elapsed_time, bandwidth;
int my_id, next_id; /* process id-s */
int p; /* number of processes */
char* message; /* storage for the message */
int i, k, max_msgs, msg_size, v;
MPI_Status status; /* return status for receive */
MPI_Init( &argc, &argv );
MPI_Comm_rank( MPI_COMM_WORLD, &my_id );
MPI_Comm_size( MPI_COMM_WORLD, &p );
if (argc < 3)
{
fprintf (stderr, "need msg count and msg size as params\n");
goto EXIT;
}
if ((sscanf (argv[1], "%d", &max_msgs) < 1) ||
(sscanf (argv[2], "%d", &msg_size) < 1))
{
fprintf (stderr, "need msg count and msg size as params\n");
goto EXIT;
}
message = (char*)malloc (msg_size);
char yahoo = 'C';
message =&yahoo;
if (argc > 3) v=1; else v=0; /*are we in verbose mode*/
/* don't start timer until everybody is ok */
MPI_Barrier(MPI_COMM_WORLD);
int t=0;
if( my_id == 0 ) {
startwtime = MPI_Wtime();
// do max_msgs times:
// send message of size msg_size chars to process 1
// receive message of size msg_size chars from process p-1
while(t<max_msgs) {
MPI_Send((char *) message, msg_size, MPI_CHAR, my_id+1, 0, MPI_COMM_WORLD);
printf("Message ,source and destination ids:%c,%d,%d \n ",*message, 0 ,my_id+1);
fflush(stdout);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
printf("Message,source and destination ids:%c,%d,%d \n",*message, my_id-1,
0);
fflush(stdout);
t++;
}
MPI_Barrier(MPI_COMM_WORLD);
endwtime = MPI_Wtime();
elapsed_time = endwtime-startwtime;
bandwidth = 2.0 * max_msgs * msg_size / (elapsed_time);
printf("Number, size of messages: %3d , %3d \n", max_msgs, msg_size);
fflush(stdout);
printf("Wallclock time = %f seconds\n", elapsed_time );
fflush(stdout);
printf("Bandwidth = %f bytes per second\n", bandwidth);
fflush(stdout);
} else if( my_id == p-1 ) {
// do max_msgs times:
// receive message of size msg_size from process to the left
// send message of size msg_size to process to the right (p-1 sends to 0)
while(t<max_msgs) {
MPI_Send((char *) message, msg_size, MPI_CHAR, 0, 0, MPI_COMM_WORLD);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
t++;
}
} else {
while(t<max_msgs) {
MPI_Send((char *) message, msg_size, MPI_CHAR, my_id+1, 0, MPI_COMM_WORLD);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
t++;
}
}
MPI_Barrier(MPI_COMM_WORLD);
EXIT:
MPI_Finalize();
return 0;
}
I am unable to get correct source and destination ids. And program after running for 1/2 times goes into infinite loop. What could be wrong here?
A few notes
The first issue with your code is that it has the potential to deadlock with some MPI implementations. Check out these notes for the details.
Leaving the deadlock issue aside, there are two problems with this code. The first involves the
if( my_id == 0 ) {
conditional. You have
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
which sets the the source to receive the message from PE -1 which wont work. You want to receive from p-1.
Also something is not right with the MPI_Barrier calls. I have to think about it for a little bit ... got it. You call MPI_Barrier in the my_id==0 branch but outside the other branches. This will produce the following
PE 0 calls barrier
all conditionals finish
PE 1 to p-1 calls barrier
PE 0 calls barrier again
frozen program
put a barrier inside all the conditionals or none of them.
Working Example (deadlock possible)
#include <stdio.h>
#include "mpi.h"
#include <stdlib.h>
/* cycle
In this program a message is sent around a circle of processes 0 - (p-1), ]
where p-1 sends to 0.
*/
int main(int argc, char **argv)
{
double startwtime, endwtime;
float elapsed_time, bandwidth;
int my_id, next_id; /* process id-s */
int p; /* number of processes */
char* message; /* storage for the message */
int i, k, max_msgs, msg_size, v;
MPI_Status status; /* return status for receive */
MPI_Init( &argc, &argv );
MPI_Comm_rank( MPI_COMM_WORLD, &my_id );
MPI_Comm_size( MPI_COMM_WORLD, &p );
if (argc < 3)
{
fprintf (stderr, "need msg count and msg size as params\n");
goto EXIT;
}
if ((sscanf (argv[1], "%d", &max_msgs) < 1) ||
(sscanf (argv[2], "%d", &msg_size) < 1))
{
fprintf (stderr, "need msg count and msg size as params\n");
goto EXIT;
}
fprintf(stdout, "my_id: %d, max_msgs: %d\n", my_id, max_msgs);
message = (char*)malloc (msg_size);
char yahoo = 'C';
message =&yahoo;
if (argc > 3) v=1; else v=0; /*are we in verbose mode*/
/* don't start timer until everybody is ok */
MPI_Barrier(MPI_COMM_WORLD);
fprintf(stdout, "my_id: %d, barrier 1\n", my_id);
int t=0;
if( my_id == 0 ) {
startwtime = MPI_Wtime();
// do max_msgs times:
// send message of size msg_size chars to process 1
// receive message of size msg_size chars from process p-1
while(t<max_msgs) {
printf("PE %d about to send\n ", my_id);
fflush(stdout);
MPI_Send((char *) message, msg_size, MPI_CHAR, my_id+1, 0, MPI_COMM_WORLD);
printf("PE %d completed send\n ", my_id);
fflush(stdout);
printf("PE %d about to recv\n ", my_id);
fflush(stdout);
MPI_Recv((char *) message, msg_size, MPI_CHAR, p-1, 0, MPI_COMM_WORLD, &status);
printf("PE %d completed recv\n ", my_id);
fflush(stdout);
t++;
}
MPI_Barrier(MPI_COMM_WORLD);
endwtime = MPI_Wtime();
elapsed_time = endwtime-startwtime;
bandwidth = 2.0 * max_msgs * msg_size / (elapsed_time);
printf("Number, size of messages: %3d , %3d \n", max_msgs, msg_size);
fflush(stdout);
printf("Wallclock time = %f seconds\n", elapsed_time );
fflush(stdout);
printf("Bandwidth = %f bytes per second\n", bandwidth);
fflush(stdout);
} else if( my_id == p-1 ) {
// do max_msgs times:
// receive message of size msg_size from process to the left
// send message of size msg_size to process to the right (p-1 sends to 0)
while(t<max_msgs) {
printf("PE %d about to send\n ", my_id);
fflush(stdout);
MPI_Send((char *) message, msg_size, MPI_CHAR, 0, 0, MPI_COMM_WORLD);
printf("PE %d completed send\n ", my_id);
fflush(stdout);
printf("PE %d about to recv\n ", my_id);
fflush(stdout);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
printf("PE %d completed recv\n ", my_id);
fflush(stdout);
t++;
}
MPI_Barrier(MPI_COMM_WORLD);
} else {
while(t<max_msgs) {
printf("*PE %d about to send\n ", my_id);
fflush(stdout);
MPI_Send((char *) message, msg_size, MPI_CHAR, my_id+1, 0, MPI_COMM_WORLD);
printf("*PE %d completed send\n ", my_id);
fflush(stdout);
printf("*PE %d about to recv\n ", my_id);
fflush(stdout);
MPI_Recv((char *) message, msg_size, MPI_CHAR, my_id-1, 0, MPI_COMM_WORLD, &status);
printf("*PE %d completed recv\n ", my_id);
fflush(stdout);
t++;
}
MPI_Barrier(MPI_COMM_WORLD);
}
EXIT:
MPI_Finalize();
return 0;
}
Example Output
> mpirun -n 3 ./a.out 1 1
my_id: 0, max_msgs: 1
my_id: 1, max_msgs: 1
my_id: 2, max_msgs: 1
my_id: 0, barrier 1
PE 0 about to send
PE 0 completed send
my_id: 1, barrier 1
*PE 1 about to send
my_id: 2, barrier 1
PE 2 about to send
PE 2 completed send
PE 2 about to recv
PE 0 about to recv
PE 0 completed recv
*PE 1 completed send
*PE 1 about to recv
*PE 1 completed recv
PE 2 completed recv
Number, size of messages: 1 , 1
Wallclock time = 0.000050 seconds
Bandwidth = 40136.878906 bytes per second

OpenMPI doesn't send data from an array

I am trying to parallelize a grayscale filter for BMP image, my function get stuck when trying to send data from a pixel array.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "mpi.h"
#define MASTER_TO_SLAVE_TAG 1 //tag for messages sent from master to slaves
#define SLAVE_TO_MASTER_TAG 10 //tag for messages sent from slaves to master
#pragma pack(1)
struct image {
struct fileHeader fh;
struct imageHeader ih;
pixel *array;
};
struct fileHeader {
//blablabla...
};
struct imageHeader {
//blablabla...
};
typedef struct
{
unsigned char R;
unsigned char G;
unsigned char B;
}pixel;
void grayScale_Parallel(struct image *im, int size, int rank)
{
int i,j,lum,aux,r;
pixel tmp;
int total_pixels = (*im).ih.width * (*im).ih.height;
int qty = total_pixels/(size-1);
int rest = total_pixels % (size-1);
MPI_Status status;
//printf("\n%d\n", rank);
if(rank == 0)
{
for(i=1; i<size; i++){
j = i*qty - qty;
aux = j;
if(rest != 0 && i==size-1) {qty=qty+rest;} //para distrubuir toda la carga
printf("\nj: %d qty: %d rest: %d\n", j, qty, rest);
//it gets stuck here,it doesn't send the data
MPI_Send(&(*im).array[j], qty*3, MPI_BYTE, i, MASTER_TO_SLAVE_TAG, MPI_COMM_WORLD);
MPI_Send(&aux, 1, MPI_INT, i, MASTER_TO_SLAVE_TAG+1, MPI_COMM_WORLD);
MPI_Send(&qty, 1, MPI_INT, i, MASTER_TO_SLAVE_TAG+2, MPI_COMM_WORLD);
printf("\nSending to node=%d, sender node=%d\n", i, rank);
}
}
else
{
MPI_Recv(&aux, 1, MPI_INT, MPI_ANY_SOURCE, MASTER_TO_SLAVE_TAG+1, MPI_COMM_WORLD,&status);
MPI_Recv(&qty, 1, MPI_INT, MPI_ANY_SOURCE, MASTER_TO_SLAVE_TAG+2, MPI_COMM_WORLD,&status);
pixel *arreglo = (pixel *)calloc(qty, sizeof(pixel));
MPI_Recv(&arreglo[0], qty*3, MPI_BYTE, MPI_ANY_SOURCE, MASTER_TO_SLAVE_TAG, MPI_COMM_WORLD,&status);
//PROCESS RECEIVED PIXELS...
//SEND to ROOT PROCESS
}
if (rank==0){
//RECEIVE DATA FROM ALL PROCESS
}
}
int main(int argc, char *argv[])
{
int rank, size;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Status status;
int op=1;
char filename_toload[50];
int bright_number=0;
struct image image2;
if (rank==0)
{
printf("File to load: \n");
scanf("%s", filename_toload);
loadImage(&image2, filename_toload);
}
while(op != 0)
{
if (rank==0)
{
printf("Welcome to example program!\n\n");
printf("\t1.- GrayScale Parallel Function\n");
printf("\t2.- Call another Function\n");
printf("\t0.- Exit\n\t");
printf("\n\n\tEnter option:");
scanf("%d", &op);
}
//Broadcast the user's choice to all other ranks
MPI_Bcast(&op, 1, MPI_INT, 0, MPI_COMM_WORLD);
switch(op)
{
case 1:
grayScale_Parallel(&image2, size, rank);
MPI_Barrier(MPI_COMM_WORLD);
printf("GrayScale applied successfully!\n\n");
break;
case 2:
function_blabla();
printf("Function called successfully\n\n");
break;
}
}
MPI_Finalize();
return 0;
}
I think the MPI_Send function can't read the array of pixels, but is strange because i can print the pixels.
Any idea?
To elaborate more on Soravux's answer, you should change the order of your MPI_Send calls (note the changed MASTER_TO_SLAVE_TAGs) as follows to avoid deadlocks:
MPI_Send(&aux, 1, MPI_INT, i, MASTER_TO_SLAVE_TAG, MPI_COMM_WORLD);
MPI_Send(&qty, 1, MPI_INT, i, MASTER_TO_SLAVE_TAG+1, MPI_COMM_WORLD);
MPI_Send(&(*im).array[j], qty*3, MPI_BYTE, i, MASTER_TO_SLAVE_TAG+2, MPI_COMM_WORLD);
These calls need to be matched by the following sequence of MPI_Recv calls
MPI_Recv(&aux, 1, MPI_INT, MPI_ANY_SOURCE, MASTER_TO_SLAVE_TAG, MPI_COMM_WORLD,&status);
MPI_Recv(&qty, 1, MPI_INT, MPI_ANY_SOURCE, MASTER_TO_SLAVE_TAG+1, MPI_COMM_WORLD,&status);
pixel *arreglo = (pixel *)calloc(qty, sizeof(pixel));
MPI_Recv(&arreglo[0], qty*3, MPI_BYTE, MPI_ANY_SOURCE, MASTER_TO_SLAVE_TAG+2, MPI_COMM_WORLD,&status);
Hope this answers your question.
The order in which you call MPI_Send and MPI_Recv is important. You must ensure your calls are always in the same order since these functions are blocking. A call to MPI_Send will not return as long as its corresponding (same tag) MPI_Recv counterpart is not executed on the destination. This may cause deadlocks otherwise.

MPI Debugging, Segmentation fault?

EDIT: My question is similar to C, Open MPI: segmentation fault from call to MPI_Finalize(). Segfault does not always happen, especially with low numbers of processes, so it you answer that one instead that would be great, either way . . .
I was hoping to get some help debugging the following code:
int main(){
long* my_local;
long n, s, f;
MPI_Init(NULL, NULL);
MPI_Comm_size(MPI_COMM_WORLD, &comm_sz);
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
if(my_rank == 0){
/* Get size n from user */
printf("Total processes: %d\n", comm_sz);
printf("Number of keys to be sorted? ");
fflush(stdout);
scanf("%ld", &n);
/* Broadcast size n to other processes */
MPI_Bcast(&n, 1, MPI_LONG, 0, MPI_COMM_WORLD);
/* Create n/comm_sz keys
NOTE! some processes will have 1 extra key if
n%comm_sz != 0 */
create_Keys(&my_local, my_rank, comm_sz, n, &s, &f);
}
if(my_rank != 0){
/* Receive n from process 0 */
MPI_Bcast(&n, 1, MPI_LONG, 0, MPI_COMM_WORLD);
/* Create n/comm_sz keys */
create_Keys(&my_local, my_rank, comm_sz, n, &s, &f);
}
/* The offending function, f is a long set to num elements of my_local*/
Odd_Even_Tsort(&my_local, my_rank, f, comm_sz);
printf("Process %d completed the function", my_rank);
MPI_Finalize();
return 0;
}
void Odd_Even_Tsort(long** my_local, int my_rank, long my_size, int comm_sz)
{
long nochange = 1;
long phase = 0;
long complete = 1;
MPI_Status Stat;
long your_size = 1;
long* recv_buf = malloc(sizeof(long)*(my_size+1));
printf("rank %d has size %ld\n", my_rank, my_size);
while (complete!=0){
if((phase%2)==0){
if( ((my_rank%2)==0) && my_rank < comm_sz-1){
/* Send right */
MPI_Send(&my_size, 1, MPI_LONG, my_rank+1, 0, MPI_COMM_WORLD);
MPI_Send(*my_local, my_size, MPI_LONG, my_rank+1, 0, MPI_COMM_WORLD);
MPI_Recv(&your_size, 1, MPI_LONG, my_rank+1, 0, MPI_COMM_WORLD, &Stat);
MPI_Recv(&recv_buf, your_size, MPI_LONG, my_rank+1, 0, MPI_COMM_WORLD, &Stat);
}
if( ((my_rank%2)==1) && my_rank < comm_sz){
/* Send left */
MPI_Recv(&your_size, 1, MPI_LONG, my_rank-1, 0, MPI_COMM_WORLD, &Stat);
MPI_Recv(&recv_buf, your_size, MPI_LONG, my_rank-1, 0, MPI_COMM_WORLD, &Stat);
MPI_Send(&my_size, 1, MPI_LONG, my_rank-1, 0, MPI_COMM_WORLD);
MPI_Send(*my_local, my_size, MPI_LONG, my_rank-1, 0, MPI_COMM_WORLD);
}
}
phase ++;
complete = 0;
}
printf("Done!\n");
fflush(stdout);
}
And the Error I'm getting is:
[ubuntu:04968] *** Process received signal ***
[ubuntu:04968] Signal: Segmentation fault (11)
[ubuntu:04968] Signal code: Address not mapped (1)
[ubuntu:04968] Failing at address: 0xb
--------------------------------------------------------------------------
mpiexec noticed that process rank 1 with PID 4968 on node ubuntu exited on signal 11 (Segmentation fault).
The reason I'm baffled is that the print statements after the function are still displayed, but if I comment out the function, no errors. So, where the heap am I getting a Segmentation fault?? I'm getting the error with mpiexec -n 2 ./a.out and an 'n' size bigger than 9.
If you actually wanted the entire runnable code, let me know. Really I was hoping not so much for the precise answer but more how to use the gdb/valgrind tools to debug this problem and others like it (and how to read their output).
(And yes, I realize the 'sort' function isn't sorting yet).
The problem here is simple, yet difficult to see unless you use a debugger or print out exhaustive debugging information:
Look at the code where MPI_Recv is called. The recv_buf variable should be supplied as an argument instead of &recv_buf.
MPI_Recv( recv_buf , your_size, MPI_LONG, my_rank-1, 0, MPI_COMM_WORLD, &Stat);
The rest seems ok.

Resources