MPI Segmentationi fault MPI_Scatter using C - c

I am new in this area and use OpenMPI and C. I try to find out why my code leads to an Segmentatioin fault. I red already a lot about MPI but I did not find any help. It took me already hours. So I decided to ask here for help.
I get the expected result of my code. But I also get every time an error message.
Is it right how I use MPI_Scatter in my case?
Here is my simple code:
#include <stdio.h>
#include "mpi.h"
#include <stdlib.h>
const int MASTER_RANK = 0;
#define DIM 3
int main(int argc, char* argv[])
{
int numProc, rank;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numProc);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
int n = 9;
double *m;
double *m_client;
m_client = (double *)malloc(3);
if(rank == MASTER_RANK)
{
m = (double* )malloc(n);
for(int i=0; i<n; i++)
{
m[i] = (double)i+1.0;
}
}
MPI_Scatter(m, 3, MPI_DOUBLE, m_client, 3, MPI_DOUBLE, MASTER_RANK, MPI_COMM_WORLD);
printf("Process %d:\n", rank);
for(int i=0; i < 3; i++)
{
printf(" (%lf", m_client[i]);
m_client[i] += 1000*rank;
printf(" -> %lf)", m_client[i]);
printf("\n");
}
printf( "\n" );
MPI_Gather(m_client, 3, MPI_DOUBLE, m, 3, MPI_DOUBLE, MASTER_RANK, MPI_COMM_WORLD);
if(rank == MASTER_RANK)
{
printf("Master: Received= \n");
for(int i=0; i<numProc; i++)
{
for(int j=0; j < 3; j++)
{
int idx = i*3 + j;
printf("%lf ", m[idx]);
}
printf("from Process %d\n", i);
}
}
free(m);
free(m_client);
MPI_Finalize();
exit(0);
}
I build my MPI file by using mpicc mpifile.c -o mpifile and run it with mpirun -np 3 ./mpifile. I use 3 processes.
The error I get is:
[Samuel-Z97-HD3:14361] *** Process received signal ***
[Samuel-Z97-HD3:14361] Signal: Segmentation fault (11)
[Samuel-Z97-HD3:14361] Signal code: (128)
[Samuel-Z97-HD3:14361] Failing at address: (nil)
I am using Ubuntu and vim / Geany.

Your code has two problems.
Both your malloc() calls are having wrong sizes. You should allocate number of bytes instead of number of doubles. E.g. instead of calling malloc(3), call malloc(3*sizeof(double)).
Another problem is that your variable m should be initialized to NULL. Alternatively, you could surround free(m) with if(rank == MASTER_RANK). As is, a non-master process calls free(m) where m is uninitialized and could contain arbitrary value.

Related

MPI_Scatter is receiving wrong values

My goal is to take an array of 6 integers and split them among 3 processes. However, the numbers in receiveBuffer are not correct. I don't know why the three processes don't contain integers from the original array.
#include<stdio.h>
#include <math.h>
#include <stdlib.h>
#include <time.h>
#include <assert.h>
#include "mpi.h"
#define ARRAY_SIZE 6
// simple print array method
void printArray(int arr[], int size)
{
int i;
for (i=0; i < size; i++)
printf("%d ", arr[i]);
printf("\n");
}
int main (int argc, char *argv[])
{
srand(time(NULL));
int array[ARRAY_SIZE];
int rank, numNodes;
// fill array with random numbers and print
for(int i = 0; i < ARRAY_SIZE; i++)
array[i] = rand();
printArray(array, ARRAY_SIZE);
MPI_Init( &argc, &argv );
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size (MPI_COMM_WORLD,&numNodes);
int receiveBuffer[ARRAY_SIZE/numNodes];
if(rank == 0)
{
MPI_Scatter(array, ARRAY_SIZE/numNodes, MPI_INT, &receiveBuffer, 0, MPI_INT, rank, MPI_COMM_WORLD);
}
printf("ID: %d with %d items.\n", rank, ARRAY_SIZE/numNodes);
printArray(receiveBuffer, ARRAY_SIZE/numNodes);
MPI_Finalize();
return 0;
}
Additionally, why is the original array printing for each process? Doesn't parallelization begin after INIT?
There are two issues with your usage of MPI_Scatter()
MPI_Scatter() is a collective operation, and has hence to be invoked by all the ranks of the communicator (e.g. not only rank zero)
since you use MPI_INT for both send and receive datatype, the send and receive counts should be equal (e.g. use ARRAY_SIZE/numNodes instead of 0)
The MPI standard does not specify what happens before MPI_Init(), and it is very common that mpirun spawns all the tasks, so they all execute the code before MPI_Init(). That's why MPI_Init() is generally invoked at the very beginning of a MPI program.

Open MPI Waitall() Segmentation Fault

I'm new with MPI and I'm trying to develop a non-blocking programm (with Isend and Irecv). The functionality is very basic (it's educational):
There is one process (rank 0) who is the master and receives messages from the slaves (rank 1-P). The master only receives results.
The slaves generates an array of N random numbers between 0 and R and then they do some operations with those numbers (again, it's just for educational purpose, the operations don't make any sense)
This whole process (operations + send data) is done M times (this is just for comparing different implementations; blocking and non-blocking)
I get a Segmentation Fault in the Master process when I'm calling the MPI_waitall() funcion
#include <stdio.h>
#include <stdlib.h>
#include "mpi.h"
#include <math.h>
#include <time.h>
#define M 1000 //Number of times
#define N 2000 //Quantity of random numbers
#define R 1000 //Max value of random numbers
double SumaDeRaices (double*);
int main(int argc, char* argv[]) {
int yo; /* rank of process */
int p; /* number of processes */
int dest; /* rank of receiver */
/* Start up MPI */
MPI_Init(&argc, &argv);
/* Find out process rank */
MPI_Comm_rank(MPI_COMM_WORLD, &yo);
/* Find out number of processes */
MPI_Comm_size(MPI_COMM_WORLD, &p);
MPI_Request reqs[p-1];
MPI_Status stats[p-1];
if (yo == 0) {
int i,j;
double result;
clock_t inicio,fin;
inicio = clock();
for(i = 0; i<M; i++){ //M times
for(j = 1; j<p; j++){ //for every slave
MPI_Irecv(&result, sizeof(double), MPI_DOUBLE, j, i, MPI_COMM_WORLD, &reqs[j-1]);
}
MPI_Waitall(p-1,reqs,stats); //wait all slaves (SEG_FAULT)
}
fin = clock()-inicio;
printf("Tiempo total de ejecucion %f segundos \n", ((double)fin)/CLOCKS_PER_SEC);
}
else {
double* numAleatorios = (double*) malloc( sizeof(double) * ((double) N) ); //array with numbers
int i,j;
double resultado;
dest=0;
for(i=0; i<M; i++){ //again, M times
for(j=0; j<N; j++){
numAleatorios[j] = rand() % R ;
}
resultado = SumaDeRaices(numAleatorios);
MPI_Isend(&resultado,sizeof(double), MPI_DOUBLE, dest, i, MPI_COMM_WORLD,&reqs[p-1]); //send result to master
}
}
/* Shut down MPI */
MPI_Finalize();
exit(0);
} /* main */
double SumaDeRaices (double* valores){
int i;
double sumaTotal = 0.0;
//Raices cuadradas de los valores y suma de estos
for(i=0; i<N; i++){
sumaTotal = sqrt(valores[i]) + sumaTotal;
}
return sumaTotal;
}
There are several issues with your code. First and foremost in your Isend you pass &resultado several times without waiting until previous non-blocking operation finishes. You are not allowed to reuse the buffer you pass to Isend before you make sure the operation finishes.
Instead I recommend you using normal Send, because in contrast to synchronous send (SSend) normal blocking send returns as soon as you can reuse the buffer.
Second, there is no need to use message tags. I recommend you to just set tag to 0. In terms of performance it is simply faster.
Third, the result shouldn't be a simple variable, but an array of size at least (p-1)
Fourth, I do not recommend you to allocate arrays on stack, like MPI_Request and MPI_Status if the size is not a known small number. In this case the size of array is (p-1), so you better use malloc for this data structure.
Fifth, if you do not check status, use MPI_STATUSES_IGNORE.
Also instead of sizeof(double) you should specify number of items (1).
But of course the absolutely best version is just to use MPI_Gather.
Moreover, generally there is no reason not to run computations on the root node.
Here is slightly rewritten example:
#include <stdio.h>
#include <stdlib.h>
#include "mpi.h"
#include <math.h>
#include <time.h>
#define M 1000 //Number of times
#define N 2000 //Quantity of random numbers
#define R 1000 //Max value of random numbers
double SumaDeRaices (double* valores)
{
int i;
double sumaTotal = 0.0;
//Raices cuadradas de los valores y suma de estos
for(i=0; i<N; i++) {
sumaTotal = sqrt(valores[i]) + sumaTotal;
}
return sumaTotal;
}
int main(int argc, char* argv[]) {
int yo; /* rank of process */
int p; /* number of processes */
/* Start up MPI */
MPI_Init(&argc, &argv);
/* Find out process rank */
MPI_Comm_rank(MPI_COMM_WORLD, &yo);
/* Find out number of processes */
MPI_Comm_size(MPI_COMM_WORLD, &p);
double *result;
clock_t inicio, fin;
double *numAleatorios;
if (yo == 0) {
inicio = clock();
}
numAleatorios = (double*) malloc( sizeof(double) * ((double) N) ); //array with numbers
result = (double *) malloc(sizeof(double) * p);
for(int i = 0; i<M; i++){ //M times
for(int j=0; j<N; j++) {
numAleatorios[j] = rand() % R ;
}
double local_result = SumaDeRaices(numAleatorios);
MPI_Gather(&local_result, 1, MPI_DOUBLE, result, 1, MPI_DOUBLE, 0, MPI_COMM_WORLD); //send result to master
}
if (yo == 0) {
fin = clock()-inicio;
printf("Tiempo total de ejecucion %f segundos \n", ((double)fin)/CLOCKS_PER_SEC);
}
free(numAleatorios);
/* Shut down MPI */
MPI_Finalize();
} /* main */

Efficient usage of MPI_Allreduce

I have on every processor a list range with numbers. I want to determine the maximum number of every row of these lists range.
The first four lists range for every processor P0-P3. The red list contains the maximum values of each row which every processor after MPI_Allreduce gets.
Here is a working version of my code:
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <mpi.h>
//#define KEY_MAX 100
typedef struct{
int myrank;
int numprocs;
int *range;
} SubDomainKeyTree;
void compRange(SubDomainKeyTree *s, int myrank, int numprocs){
s->myrank = myrank;
s->numprocs = numprocs;
// Allocate memory for (numprocs+1) ranges
s->range = malloc((numprocs+1) * sizeof(int));
// Compute range values
for(int p=0; p<=numprocs; p++){
s->range[p] = rand()%100;
}
for(int p=0; p<s->numprocs; p++){
if(s->myrank == p){
for(int k=0; k<=s->numprocs; k++){
printf("Processor %d: %d random number is %d\n", p, k, s->range[k]);
}
printf("\n");
}
}
}
void compDynRange(SubD *s){
int rangeForAll[s->numprocs+1];
//////////////////////////////////
// This is not really efficient //
//////////////////////////////////
for(int r=0; r<=s->numprocs; r++){
MPI_Allreduce(&s->range[r], &rangeForAll[r], 1, MPI_INT, MPI_MAX, MPI_COMM_WORLD);
}
for(int p=0; p<s->numprocs; p++){
if(s->myrank == p){
for(int k=0; k<=s->numprocs; k++){
s->range[k] = rangeForAll[k];
printf("Processor %d: %d random number after MPI_Allreduce is %d\n", p, k, s->range[k]);
}
printf("\n");
}
}
}
int main(int argc, char **argv){
int nameLen;
char processorName[MPI_MAX_PROCESSOR_NAME];
int myrank; // Rank of processor
int numprocs; // Number of processes
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Get_processor_name(processorName,&nameLen);
MPI_Status status;
time_t t;
srand((unsigned)time(NULL)+myrank*numprocs+nameLen);
SubD s;
compRange(&s, myrank, numprocs);
compDynRange(&s);
MPI_Finalize();
return 0;
}
I use a for-loop which seems highly inefficient to me. Here I compute the maximum value of every row of all lists one after the other.
But can I use MPI_Allreduce without that for-loop?
I already tried that instead of the for-loop which does not work.
MPI_Allreduce(&s->range, &rangeForAll, s->numprocs+1, MPI_INT, MPI_MAX, MPI_COMM_WORLD);
Can someone give me a hint how I can do that?
As already hinted in comment, the error you had in you code was that instead of passing the arrays containing your send and receive buffers, you were passing some pointers to them. I imagine that this error was simply coming fro the change from a single element used initially (like &s->range[r]) which was perfectly correct, to the full array by just removing the indexed access (ie &s->range) which was wrong.
So as explained, using:
MPI_Allreduce(s->range, rangeForAll, s->numprocs+1, MPI_INT, MPI_MAX, MPI_COMM_WORLD)
just does the trick. However, since you want to get the results into the s->range arrays rather than the temporary rangeFarAll ones, you'd better off not defining the later at all, and use the MPI_IN_PLACE keyword as sending parameter and s->range as receiving one. The call becomes:
MPI_Allreduce(MPI_IN_PLACE, s->range, s->numprocs+1, MPI_INT, MPI_MAX, MPI_COMM_WORLD)
and s->range acts both as sending and receiving buffer. Therefore, the final results will all be in the s->range buffers after the call, sparing you the need of doing the copy explicitly.

Scattering a contiguous 2D Array using MPI

I don't seem to get this right, i want to scatter a 2D block and just print the scattered parts, this is for testing purpose
Here's what I'm trying to do ...
#include <stdio.h>
#include <stdlib.h>
#include "mpi/mpi.h"
void print2D(int**, int, int);
int main(int argc, char* argv[])
{
int rank, pSize, r1, c1, rowsPerProcess, i, j;
int** matA = NULL, **partMat = NULL;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &pSize);
if(rank == 0)
{
scanf("%d%d", &r1, &c1);
rowsPerProcess = r1/pSize; // number of rows assigned to each process (Assume this produces no remainder)
//Contiguous allocation
int* mat1D = (int*)malloc(r1 * c1 * sizeof(int));
matA = (int**)malloc(r1 * sizeof(int*));
for(i=0; i<r1; i++)
matA[i] = &mat1D[i * c1];
for(i=0; i<r1; i++) //Fill the 2D array
for(j=0; j<c1; j++)
matA[i][j] = i + j;
printf("Original Matrix:\n");
print2D(matA, r1, c1);
}
MPI_Bcast(&c1, 1, MPI_INT, 0, MPI_COMM_WORLD); //Columns
MPI_Bcast(&rowsPerProcess, 1, MPI_INT, 0, MPI_COMM_WORLD); //Rows in every process
//Again, contiguous allocation for the partial 2D array
int* partMat1D = (int*)malloc(rowsPerProcess * c1 * sizeof(int));
partMat = (int**)malloc(rowsPerProcess * sizeof(int*));
for(i=0; i<rowsPerProcess; i++)
partMat[i] = &partMat1D[i * c1];
MPI_Scatter(&(matA[0][0]), rowsPerProcess * c1, MPI_INT, &(partMat[0][0]),
rowsPerProcess * c1, MPI_INT, 0, MPI_COMM_WORLD);
printf("FROM PROCESS %d:\n", rank);
print2D(partMat, rowsPerProcess, c1);
MPI_Finalize();
return 0;
}
void print2D(int** mat, int row, int col)
{
if(mat != NULL)
{
int i, j;
for(i=0; i<row; i++)
{
for(j=0; j<col; j++)
printf("%d ", mat[i][j]);
printf("\n");
}
}
}
and I receive a segmentation fault error on running this with 4 processes
e.g.
8 3
Original Matrix:
0 1 2
1 2 3
2 3 4
3 4 5
4 5 6
5 6 7
6 7 8
7 8 9
FROM PROCESS 0:
0 1 2
1 2 3
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 2637 RUNNING AT minix-VirtualBox
= EXIT CODE: 139
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
Only process 0 print its part correctly.
Am i using MPI_Scatter wrong? how do i fix this?
how do i scatter this 2D block right?
On the non-master processes, you initialize matA to NULL, and then proceed to evaluate the expression &matA[0][0]. This dereferences matA, crashing your process even before it enters MPI_Scatter().
Since you are programming in C, you can avoid those double pointers: C has quite good multidimensional array support built in since C99, and you can use it to make your life much easier. Just declare matA as a pointer to a line array
int (*matA)[c1];
then allocate an array of r1 rows for it
matA = malloc(r1*sizeof(*matA));
After that you can fill your array with the same code as you used in your code above:
for(i=0; i<r1; i++) //Fill the 2D array
for(j=0; j<c1; j++)
matA[i][j] = i + j;
Also, since your 2D array needs only one malloc() call, you can also dispose it off with a single free() call:
free(matA);
Note: You cannot do this in C++, it does not support variable length arrays the way C does, and probably never will. This is one of the few points where C is much more powerful than C++.

MPI_Scatter of 2D array and malloc

I'm trying to write a program in C with the MPI library, in which the master process creates a 2D-array and distributes its rows of to the other processes. The matrix has dimensions p*p, in which p is the number of processes.
Here's the code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <mpi.h>
int **createMatrix (int nrows, int ncols) {
int **matrix;
int h, i, j;
if (( matrix = malloc(nrows*sizeof(int*))) == NULL) {
printf("Malloc error");
exit(1);
}
for (h=0; h<nrows; h++) {
if (( matrix[h] = malloc( ncols * sizeof(int))) == NULL) {
printf("Malloc error 2");
exit(1);
}
}
for (i=0; i<ncols; i++) {
for (j=0; j<nrows; j++) {
matrix[i][j] = ((i*nrows) + j);
}
}
return matrix;
}
void printArray (int *row, int nElements) {
int i;
for (i=0; i<nElements; i++) {
printf("%d ", row[i]);
}
printf("\n");
}
void printMatrix (int **matrix, int nrows, int ncols) {
int i;
for (i=0; i<nrows; i++) {
printArray(matrix[i], ncols);
}
}
int main (int argc, char **argv) {
if (MPI_Init(&argc, &argv) != MPI_SUCCESS) {
perror("Error initializing MPI");
exit(1);
}
int p, id;
MPI_Comm_size(MPI_COMM_WORLD, &p); // Get number of processes
MPI_Comm_rank(MPI_COMM_WORLD, &id); // Get own ID
int **matrix;
if (id == 0) {
matrix = createMatrix(p, p); // Master process creates matrix
printf("Initial matrix:\n");
printMatrix(matrix, p, p);
}
int *procRow = malloc(sizeof(int) * p); // received row will contain p integers
if (procRow == NULL) {
perror("Error in malloc 3");
exit(1);
}
if (MPI_Scatter(*matrix, p, MPI_INT, // send one row, which contains p integers
procRow, p, MPI_INT, // receive one row, which contains p integers
0, MPI_COMM_WORLD) != MPI_SUCCESS) {
perror("Scatter error");
exit(1);
}
printf("Process %d received elements: ", id);
printArray(procRow, p);
MPI_Finalize();
return 0;
}
The output I receive when running this code is
$ mpirun -np 4 test
Initial matrix:
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
Process 0 received elements: 0 1 2 3
Process 1 received elements: 1 50 32 97
Process 2 received elements: -1217693696 1 -1217684120 156314784
Process 3 received elements: 1 7172196 0 0
Process 0 apparently receives the correct input, but the other processes show numbers I can't make sense of. Also note that the numbers of process 1 and 3 are consistent over multiple runs of the program, whereas the numbers of process 2 change in each run.
It seems to me that there's something wrong in my memory allocating or pointer usage, but I'm quite new to programming in C. Could anyone explain to me how and why this output is produced? Secondary, obviously, I'm also interested in how to solve my issue :) thanks in advance!
I think you have fundamentally misunderstood what the scatter operation does and how MPI expects memory to be allocated and used.
MPI_Scatter takes the source array and splits it into pieces, sending a unique piece to each member of the MPI communicator. In your example, you would need your matrix to be an allocation of contiguous p*p elements in linear memory, which would send p values to each process. Your source "matrix" is an array of pointers. There is no guarantee that the rows are sequentially arranged in memory, and MPI_Scatter doesn't know how to traverse the array of pointers you have passed it. As a result, the call is simply reading beyond the end of the first row you passed by indirection of the matrix pointer, treating whatever follows that in memory as data. That is why you get garbage values in the processes which receive the data after the first row.
All MPI data copy routines expect that source and destination memory are "flat" linear arrays. Multidimensional C arrays should be stored in row major order not in arrays of pointers as you have done here. A cheap and nasty hack of your example to illustrate a scatter call working correctly would be like this:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <mpi.h>
int *createMatrix (int nrows, int ncols) {
int *matrix;
int h, i, j;
if (( matrix = malloc(nrows*ncols*sizeof(int))) == NULL) {
printf("Malloc error");
exit(1);
}
for (h=0; h<nrows*ncols; h++) {
matrix[h] = h+1;
}
return matrix;
}
void printArray (int *row, int nElements) {
int i;
for (i=0; i<nElements; i++) {
printf("%d ", row[i]);
}
printf("\n");
}
int main (int argc, char **argv) {
if (MPI_Init(&argc, &argv) != MPI_SUCCESS) {
perror("Error initializing MPI");
exit(1);
}
int p, id;
MPI_Comm_size(MPI_COMM_WORLD, &p); // Get number of processes
MPI_Comm_rank(MPI_COMM_WORLD, &id); // Get own ID
int *matrix;
if (id == 0) {
matrix = createMatrix(p, p); // Master process creates matrix
printf("Initial matrix:\n");
printArray(matrix, p*p);
}
int *procRow = malloc(sizeof(int) * p); // received row will contain p integers
if (procRow == NULL) {
perror("Error in malloc 3");
exit(1);
}
if (MPI_Scatter(matrix, p, MPI_INT, // send one row, which contains p integers
procRow, p, MPI_INT, // receive one row, which contains p integers
0, MPI_COMM_WORLD) != MPI_SUCCESS) {
perror("Scatter error");
exit(1);
}
printf("Process %d received elements: ", id);
printArray(procRow, p);
MPI_Finalize();
return 0;
}
which does this:
$ mpicc -o scatter scatter.c
$ mpiexec -np 4 scatter
Initial matrix:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Process 0 received elements: 1 2 3 4
Process 1 received elements: 5 6 7 8
Process 2 received elements: 9 10 11 12
Process 3 received elements: 13 14 15 16
ie. when you pass the data stored in linear memory, it works. The equivalent row major array would be statically allocated like this:
int matrix[4][4] = { { 1, 2, 3, 4 },
{ 5, 6, 7, 8 },
{ 9, 10, 11, 12 },
{ 13, 14, 15, 16 } };
Note the difference between a statically allocated two dimensional array, and the array of pointers which your code allocated dynamically. They are not at all the same thing, even though they look superficially similar.

Resources