Reading multiple files using MPI-IO - c

Question

I'm trying to read in multiple files using MPI-IO in C. I'm following this example : http://users.abo.fi/Mats.Aspnas/PP2010/examples/MPI/readfile1.c
However I'm reading in a matrix a doubles instead of a string of chars. Here is that implementation:
/*
Simple MPI-IO program that demonstrate parallel reading from a file.
Compile the program with 'mpicc -O2 readfile1.c -o readfile1'
*/
#include <stdlib.h>
#include <stdio.h>
#include "mpi.h"
#define FILENAME "filename.dat"
double** ArrayAllocation() {
int i;
double** array2D;
array2D= (double**) malloc(num_procs*sizeof(double*));
for(i = 0; i < num_procs; i++) {
twoDarray[i] = (double*) malloc(column_size*sizeof(double));
}
return array2D;
}
int main(int argc, char* argv[]) {
int i, np, myid;
int bufsize, nrchar;
double *buf; /* Buffer for reading */
double **matrix = ArrayAllocation();
MPI_Offset filesize;
MPI_File myfile; /* Shared file */
MPI_Status status; /* Status returned from read */
/* Initialize MPI */
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &myid);
MPI_Comm_size(MPI_COMM_WORLD, &np);
/* Open the files */
MPI_File_open (MPI_COMM_WORLD, FILENAME, MPI_MODE_RDONLY,
MPI_INFO_NULL, &myfile);
/* Get the size of the file */
MPI_File_get_size(myfile, &filesize);
/* Calculate how many elements that is */
filesize = filesize/sizeof(double);
/* Calculate how many elements each processor gets */
bufsize = filesize/np;
/* Allocate the buffer to read to, one extra for terminating null char */
buf = (double *) malloc((bufsize)*sizeof(double));
/* Set the file view */
MPI_File_set_view(myfile, myid*bufsize*sizeof(double), MPI_DOUBLE,
MPI_DOUBLE,"native", MPI_INFO_NULL);
/* Read from the file */
MPI_File_read(myfile, buf, bufsize, MPI_DOUBLE, &status);
/* Find out how many elemyidnts were read */
MPI_Get_count(&status, MPI_DOUBLE, &nrchar);
/* Set terminating null char in the string */
//buf[nrchar] = (double)0;
printf("Process %2d read %d characters: ", myid, nrchar);
int j;
for (j = 0; j <bufsize;j++){
matrix[myid][j] = buf[j];
}
/* Close the file */
MPI_File_close(&myfile);
if (myid==0) {
printf("Done\n");
}
MPI_Finalize();
exit(0);
}
However when I try to call MPI_File_open after I close the first file, I get an error. Do I need multiple communicators to perform this? Any tips will be appreciated.

Answer 1

The code in ArrayAllocation above does not quite match the logic of the main program. The matrix is allocated as an array of pointers to vectors of doubles before MPI is initialized, therefore it is impossible to set the number of rows to the number of MPI processes.
The column_size is also not known until the file size is determined.
It is a general convention in the C language to store matrices by rows. Violating this convention might confuse you or the reader of your code.
All in all in order to get this program working you need to declare
int num_procs, column_size;
as global variables before the definition of ArrayAllocation and move the call to this function down below the line where bufsize is calculated:
...
/* Calculate how many elements each processor gets */
bufsize = filesize/np;
num_procs = np;
column_size = bufsize;
double **matrix = ArrayAllocation();
...
With the above modifications this example should work on any MPI implementation that supports MPI-IO. I have tested it with OpenMPI 1.2.8.
In order to generate a test file you could use for instance the following code:
FILE* f = fopen(FILENAME,"w");
double x = 0;
for(i=0;i<100;i++){
fwrite(&x, 1,sizeof(double), f);
x +=0.1;
}
fclose(f);