Sum of the number 1 to 10000 using MPI_Scatter - c

Question

I'm currently developing a piece of code that has 10 processes. Process 0 reads a total of 10000 (from lab7.csv). Afterwards, it distributes the array to all of the processes. In order to do this, I created an array named "intArray[10000]" that is shared by all processes. 49893236 is the correct summation.
The following code makes use of 10 processors to compute the sum of numbers ranging from 1 to 1000. This aggregate is calculated by each of the processors, and the results are shown on the screen.
As a result, a follower error is shown.
I didn't figure out what the problem is. Please assist me in this matter.
#include <mpi.h>
#include <stdio.h>
#include <string.h>
int main()
{
int rank, nodes;
MPI_Init(NULL, NULL);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &nodes);
MPI_Status status;
int intArray[10000];
int subIntArray[1000];
if(rank == 0) {
// Substitute the full file path for the string file_path
FILE *fp = fopen("./lab7.csv", "r");
int i = 0;
int num;
if (!fp) {
printf("Can't open file\n");
} else {
while (fscanf(fp, "%d", &num) > 0)
{
intArray[i] = num;
i++;
}
// Close the file
fclose(fp);
}
}
MPI_Scatter(intArray, 10000, MPI_INT, subIntArray, 1000, MPI_INT, 0, MPI_COMM_WORLD);
int ans = 0;
int total = 0;
int start = rank * 1000;
int end = start + 999;
for(int i = start; i <= end; i++) {
ans = ans + subIntArray[i];
}
if(rank != 0) {
MPI_Ssend(&ans, 1, MPI_INT, 0, 0, MPI_COMM_WORLD);
} else {
total = ans;
for(int j = 1; j < 10; j++) {
MPI_Recv(&ans, 1, MPI_INT, j, 0, MPI_COMM_WORLD, &status);
total += ans;
}
printf("Total is %d\n", total);
}
MPI_Finalize();
return 0;
}
The PBS Job file as followers,
#PBS -l nodes=2
#PBS -l walltime=00:02:00
#PBS -l select=5
cat $PBS_NODEFILE
NPROC=10
cd $PBS_O_WORKDIR
MPISIZE=$NPROC
MPIPROG=`basename $PBS_JOBNAME .pbs`
echo 'Running MPI program' $MPIPROG 'on' $MPISIZE 'processes'
echo 'Started at' `date`
echo '--------------------------------------------------------------------------------'
(time mpirun -n $MPISIZE ./$MPIPROG) 2>&1
echo '--------------------------------------------------------------------------------'
echo 'Finished at' `date`
This is the error message that is shown on the terminal.

Answer 1

The orientation of MPI_Scatter is a bit different than the one you have.
You say: I have NTOT data elements and I want to send them to NODECOUNT nodes so I want each node to process NTOT / NODECOUNT of data. This doesn't work if NTOT is not an exact multiple of NODECOUNT
But, MPI_Scatter is oriented the other way: I have NPER number of elements that each node should process, and NODECOUNT nodes, so the total number of elements is NTOT = NPER * NODECOUNT This is how the manpage example shows it.
You want to give a count of NPER to MPI_Scatter and not NTOT. And, you want the send and receive counts to match.
Also, because MPI_Scatter does the split for you, the slave nodes should not use start/end as you calculated, but always do:
start = 0;
end = NPER - 1;
Also, in your code ...
You were indexing into subIntArray as if you could access 0-9999 instead of 0-999, so you were going beyond the end of the array and had UB (undefined behavior)
It's a bit shaky to hardwire 10, 100, 1000 everywhere. Better to use the some #define and the actual node code nodes
And, you assume you have 10000 valid input data elements rather than calculating this based on your i index variable when you do fscanf
Here is the corrected code, with some extra debug code I used.
I used preprocessor conditionals to denote old vs new code (e.g.):
#if 0
// old code
#else
// new code
#endif
#if 1
// new code
#endif
Also, I didn't have your data files, so I had to synthesize the input data.
Anyway, here it is:
#include <mpi.h>
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
#include <unistd.h>
int rank;
int nodes;
FILE *xfdbg;
#define dbgprtattr(_lvl) \
__attribute__((__format__(__printf__,_lvl,_lvl + 1)))
#if DEBUG || _USE_ZPRT_
#define dbgprt(_fmt...) \
_dbgprt(_fmt)
#else
#define dbgprt(_fmt...) \
do { } while (0)
#endif
void dbgprtattr(1)
_dbgprt(const char *fmt,...)
{
va_list ap;
char buf[10000];
char *bp = buf;
bp += sprintf(bp,"[%d] ",rank);
va_start(ap,fmt);
bp += vsprintf(bp,fmt,ap);
va_end(ap);
fputs(buf,xfdbg);
fflush(xfdbg);
}
//#define NTOT 10000
//#define NPER (NTOT / nodes)
int
main()
{
MPI_Init(NULL, NULL);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &nodes);
MPI_Status status;
char logf[100];
sprintf(logf,"log_%2.2d",rank);
xfdbg = fopen(logf,"w");
#if 0
int BIGSIZE = nodes * 1000;
int NPER = NTOT / nodes;
#else
//int NPER = 1000;
int NPER = 50;
//int NTOT = NPER * nodes;
#endif
int val;
int truetotal = 0;
#if 0
int intArray[NTOT];
#else
int intArray[nodes][NPER];
#endif
#if 1
int subIntArray[NPER];
#else
int subIntArray[NTOT];
#endif
if (rank == 0) {
#if 0
// Substitute the full file path for the string file_path
FILE *fp = fopen("./lab7.csv", "r");
int i = 0;
int num;
if (!fp) {
printf("Can't open file\n");
}
else {
while (fscanf(fp, "%d", &num) > 0) {
intArray[i] = num;
i++;
}
// Close the file
fclose(fp);
}
#endif
for (int nd = 0; nd < nodes; ++nd) {
for (int i = 0; i < NPER; ++i) {
val = (nd << 16) | i;
intArray[nd][i] = val;
truetotal += val;
}
}
}
dbgprt("main: hello\n");
#if 0
MPI_Scatter(intArray, NTOT, MPI_INT,
subIntArray, NPER, MPI_INT,
0, MPI_COMM_WORLD);
dbgprt("main: post\n");
#endif
#if 1
MPI_Scatter(intArray, NPER, MPI_INT,
subIntArray, NPER, MPI_INT,
0, MPI_COMM_WORLD);
dbgprt("main: post\n");
#endif
#if 0
MPI_Scatter(intArray, NTOT, MPI_INT,
subIntArray, NTOT, MPI_INT,
0, MPI_COMM_WORLD);
dbgprt("main: post\n");
#endif
//sleep(10);
int ans = 0;
int total = 0;
#if 0
int start = rank * NPER;
int end = start + NPER - 1;
#else
int start = 0;
int end = NPER - 1;
#endif
dbgprt("main: START start=%d end=%d\n",start,end);
for (int i = start; i <= end; i++) {
dbgprt("main: DATA i=%d sub=%8.8X\n",i,subIntArray[i]);
ans = ans + subIntArray[i];
}
dbgprt("main: loopdone ans=%d\n",ans);
if (rank != 0) {
MPI_Ssend(&ans, 1, MPI_INT, 0, 0, MPI_COMM_WORLD);
}
else {
total = ans;
for (int j = 1; j < nodes; j++) {
MPI_Recv(&ans, 1, MPI_INT, j, 0, MPI_COMM_WORLD, &status);
total += ans;
}
printf("Total is %d\n", total);
printf("Total is %d (TRUE)\n", truetotal);
}
fclose(xfdbg);
MPI_Finalize();
return 0;
}