I am trying to build a beowulf cluster with Raspberry Pi. I downloaded the following packages of OpenMPI: openmpi-bin, openmpi-common, libopenmpi1.3, libopenmpi-dbg, libopenmpi-dev. I used static IP on each of the Raspberrys and tested the connection between each of them and it was working. I also enabled the ssh and I used it to login from one Raspberry to all the other Raspberrys.
This is the following code I used:
#include <mpi.h>
int main(int argc, char *argv[])
{
int tid,nthreads;
char *cpu_name;
/* add in MPI startup routines */
/* 1st: launch the MPI processes on each node */
MPI_Init(&argc,&argv);
/* 2nd: request a thread id, sometimes called a "rank" from
the MPI master process, which has rank or tid == 0
*/
MPI_Comm_rank(MPI_COMM_WORLD, &tid);
/* 3rd: this is often useful, get the number of threads
or processes launched by MPI, this should be NCPUs-1
*/
MPI_Comm_size(MPI_COMM_WORLD, &nthreads);
/* on EVERY process, allocate space for the machine name */
cpu_name = (char *)calloc(80,sizeof(char));
/* get the machine name of this particular host ... well
at least the first 80 characters of it ... */
gethostname(cpu_name,80);
printf("hello MPI user: from process = %i on machine=%s, of NCPU=%i processes\n",
tid, cpu_name, nthreads);
MPI_Finalize();
return(0);
}
I tested the code first with only 2 boards and it worked fine no problems no errors and it printed the printf statement in the code.
I created a host file which includes the following:
Pi0
Pi1
Pi2
Pi3
I have 4 Raspberry Pis. I copied the code on each of them and compiled the code also using the following statements:
mpicc -g -0O -c hello-mpi.c -o hello-mpi.o
mpicc -g hello-mpi.o -o hello-mpi
I executed the code with the following statement:
mpirun -np 4 -hostfile hostfile hello-mpi
when I run the program nothing happens it just gives me an new empty line. No errors are given also. Any suggestion what I can I do to make it work
Related
I am trying to debug code written for Raspberry Pi on my Ubuntu laptop. I am using:
the Raspberry Pi Toolchain and (partial ?) root fs obtained with git clone git://github.com/raspberrypi/tools.git executed in ~/rpi
qemu virtualizer installed from Ubuntu repositiories - qemu-arm reports
version 2.0.0
the following test case:
#include <pthread.h>
#include <stdio.h>
static void *thread(void *arg)
{
static char szReturn[] = "Me too";
puts((char *)arg);
puts("Hello, I'm Thread");
return szReturn;
}
int main(int argc, char *argv[])
{
pthread_t tid;
char szGreeting[] = "Hello, I'm Main", *reply;
if (pthread_create(&tid, NULL, &thread, szGreeting) != 0)
{
perror("pthread_create says: ");
return 1;
}
puts("Nice to meet you");
if (pthread_join(tid, (void **)&reply) == 0)
puts(reply);
return 0;
}
After adding ~/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin to the PATH (I am using 64 bit Ubuntu), I do (from the directory in which the test code resides):
arm-linux-gnueabihf-gcc -g -O0 test.c -o test -lpthread
qemu-arm -L ~/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/arm-linux-gnueabihf/libc ./test
I get the expected output, so the thread creation and execution works.
However, if I do:
qemu-arm -g 1234 -L ~/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/arm-linux-gnueabihf/libc ./test in one terminal, to launch qemu and have it waiting for the debugger to connect to it
arm-linux-gnueabihf-gdb -ex="target remote localhost:1234" -x="set breakpoint pending on" -ex="break main" ./test in another terminal, to start debugging the program with gdb
I get in gdb, after 'continue' I reach main, I set breakpoints to thread, 21 and 24, I next OK to the ptread_create call, so debugging the main thread works, but:
when it executes pthread_create, instead of hitting one of the 3 breakpoints (the thread, the create failure or the create success cases), gdb reports "inferior terminated' and the session ends
back in the qemu terminal though, the program has generated the normal output, so the thread was created and executed, and the whole program worked as expected
So gdb can not debug multithreaded programs in this configuration/setup?! What am I missing?
Could the fact that I am running Ubuntu in a Parallels virtual machine on an OSX machine have something to do with it? I don't have a dedicated Ubuntu machine to test, but I doubt it.
Update: On a native Fedora system the behavior is even worse: the breakpoints are also not taken, but it also crashes in puts and pthread_join.
Thanks in advance
I need to use MUMPS in my project. I wanted to test the installation with the simple example of solving equations (below). The problem is, that this code works correctly in my personal PC, but it gives me this error every time I try to run it in my work computer:
*** The MPI_Comm_f2c() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[espreso-ws:3263] Local abort before MPI_INIT completed successfully; not able to aggregate error messages, and not able to guarantee that all other processes were killed!
I've tried to look for possible solutions, but I've found only these questions:
error: The MPI_Send() function was called before MPI_INIT was invoked
http://www.open-mpi.org/community/lists/users/2012/05/19262.php
None of them provides any solution.
So now, I'm incredibly confused about this and I don't really know, what should I try to do. There IS MPI_Init() invoked before MPI_Comm_f2c() of course.
Do you know, how to solve this problem?
mumps_solve.c
/*
* file c_example.c
* This file is part of MUMPS 4.10.0
* To run: aprun -n 2 ./dsimpletest < input_simpletest_real
*/
/* Example program using the C interface to the
* double real arithmetic version of MUMPS, dmumps_c.
* We solve the system A x = RHS
*/
#include <stdio.h>
#include <string.h>
#include <mpi.h>
#include <dmumps_c.h>
#define JOB_INIT -1
#define JOB_END -2
#define USE_COMM_WORLD -987654
int main(int argc, char ** argv)
{
DMUMPS_STRUC_C id;
int n = 2;
int nz = 4;
int irn[] = {1,1,2,2};
int jcn[] = {1,2,1,2};
double a[4];
double rhs[2];
int myid, ierr;
ierr = MPI_Init(&argc, &argv);
ierr = MPI_Comm_rank(MPI_COMM_WORLD, &myid);
/* Define A and rhs */
rhs[0]=5.0;rhs[1]=2.0;
a[0]=3.0;a[1]=2.0;a[2]=1.0;a[3]=4.0;
/* Initialize a MUMPS instance. Use MPI_COMM_WORLD */
id.job=JOB_INIT;id.par=1; id.sym=0;id.comm_fortran=USE_COMM_WORLD;
dmumps_c(&id); // here the program crashes
/* Define the problem on the host */
if (myid == 0) {
id.n = n; id.nz =nz; id.irn=irn; id.jcn=jcn;
id.a = a; id.rhs = rhs;
}
#define ICNTL(I) icntl[(I)-1] /* macro s.t. indices match documentation */
/* No outputs */
id.ICNTL(1)=-1; id.ICNTL(2)=-1; id.ICNTL(3)=-1; id.ICNTL(4)=0;
/* Call the MUMPS package. */
id.job=6;
dmumps_c(&id);
id.job=JOB_END; dmumps_c(&id); /* Terminate instance */
if (myid == 0) {
printf("Solution is : (%8.2f %8.2f)\n", rhs[0],rhs[1]);
}
ierr = MPI_Finalize();
return 0;
}
Makefile
.DEFAULT_GOAL=all
BINARIES=mumps_solve
.PHONY=all
all: mumps_solve
mumps_solve: mumps_solve.o
mpicc mumps_solve.o -o mumps_solve -ldmumps
mumps_solve.o: mumps_solve.c
mpicc -c mumps_solve.c -o mumps_solve.o
.PHONY=clean
clean:
rm -f ${BINARIES} *.o
The problem could be hidden in the linking. I see that the actual call which raises the error is in a library against which you are linking.
In short: you maybe linked against multiple copies of MPI. One is initialized, one is not. The latter is called by your library and raises an error.
Solution in short: compile both the external library and your own code again, verifying that all compilation is done by the exact same physical copy of mpicc.
The library could be compiled and linked against a different copy of your MPI distribution than the copy against which you link your main library. If that is the case, there are multiple copies of the global variable which keeps track of the initialization state of MPI. Your call to MPI_Init() goes to the MPI copy which you linked against by invoking mpicc in your Makefile (located at which mpicc). If you call MPI_Initialized(int*) after MPI_Init(), you should find that it returns true.
If you had the chance to modify the source of the external library (mumps), and call MPI_Initialized(int*) just before the line that crashes, you should find that it returns false, even though you called MPI_Init(). This is a hint that the library and your binary are linked against different copies of your MPI distribution.
In my case (after two days of bumping my head against the wall), it was even a matter of different dynamically linked versions of MPI. I had the luck to be in control of the compilation of the library, and could hence modify the version of MPI it links against.
I was writing a project using MPI for a parallel programming course, and decided to name one of my functions connect(). But whenever I tried to mpirun the program (using recent versions of Open MPI on Linux and OS X), I would receive output from the connect() function, even if I had not called connect() from main(); also, some of the output from main() would not appear.
This is a simplified program with the issue:
#include <stdlib.h>
#include <stdio.h>
#include <mpi.h>
void connect(); //function name breaks mpi
int main(void) {
int comm_sz, my_rank;
MPI_Init(NULL, NULL);
MPI_Comm_size(MPI_COMM_WORLD, &comm_sz);
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
printf("my_rank is %d\n", my_rank);
fflush(stdout);
MPI_Finalize();
return EXIT_SUCCESS;
}
void connect() {
printf("\nNot main! \n");
return;
}
and the output:
[me#host ~]$ mpicc bad.c -Wall
[me#host ~]$ mpirun -n 1 a.out
Not main!
--------------------------------------------------------------------------
orterun noticed that process rank 0 with PID 17245 on node host exited on signal 13 (Broken pipe).
--------------------------------------------------------------------------
I was about to ask on Stack Overflow what was wrong in the first place, until I discovered that renaming the function fixes it. So what I'm curious about now is why naming the function connect() prevents the program from running correctly. Could it also be an issue with mpirun/Open RTE?
Possible leads:
There's a connect() function in <sys/socket.h>, but I haven't yet found it mentioned in the MPI header files.
There's also a Connect() function (with an uppercase C) in "ompi/mpi/cxx/intracomm.h" which is indirectly included by <mpi.h>, but I thought case mattered in C/C++, and it looks like a method of a C++ class.
If I try executing the program like a normal one, it works when run on OS X, but not on Linux:
mac:~ me$ ./a.out
my_rank is 0
vs
[me#linux ~]$ ./a.out
Not main!
I would guess that one of the MPI functions you call is in turn calling the connect() system call. But since ELF executables have a flat namespace for symbols, your connect() is being called instead.
The problem doesn't happen on Mac OS because Mach-O libraries have a two-level namespace, so symbols in different libraries don't conflict with each other.
If you make your function static, that would probably avoid the problem as well.
I have a small piece of code that should run on multilple processes, which is :
#include <stdio.h>
#include "mpi.h"
main(int argc, char **argv)
{
int ierr, num_procs, my_id;
ierr = MPI_Init(&argc, &argv);
/* find out MY process ID, and how many processes were started. */
ierr = MPI_Comm_rank(MPI_COMM_WORLD, &my_id);
ierr = MPI_Comm_size(MPI_COMM_WORLD, &num_procs);
printf("Hello world! I'm process %i out of %i processes\n",
my_id, num_procs);
ierr = MPI_Finalize();
}
the output is :
Hello world! I'm process 0 out of 1 processes. although it should run on more than one process
We edited the run configurations arguments to "-np 2" so it would run on 2 processes but it always gives us 1 process no matter what the value is.
The used environment is:
Eclipse Juno on Ubuntu 12.04
Source of code: http://condor.cc.ku.edu/~grobe/docs/intro-MPI-C.shtml[^]
It seems that you are trying to launch your MPI application directly, i.e. starting the compiled executable with -np 2, like this:
$ ./my_app -np 2
That's not the right way of launching MPI programs. Instead, you should call your MPI implementation's launcher (usually named mpirun or mpiexec), and pass the name of your executable and -np 2 to that. So if your executable is called my_app, then instead of the above command, you should run:
$ mpirun -np 2 ./my_app
Consult your MPI implementation's documentation for the specifics.
Some small points about mpirun or mpiexec commands:
If you are trying to run you app on multiple nodes, make sure that your app is copied to all the nodes.In addition, make sure that all needed binary files (executable & libraries) could be found directly from command line. I personally, prefer to run my MPI programs within a shell script like this:
#!/bin/sh
PATH=/path/to/all/executable/files \
LD_LIBRARY_PATH=/path/to/all/libraries \
mpirun -np 4 ./my_app arg1 arg2 arg3
I have a program on C that uses both MPI and OpenMP. In order to compile such program on Windows system I have downloaded and installed a gcc compiler provided by MinGW. Using this compiler I can compile and execute C programs with OpenMP using the key -fopenmp for gcc. Such programs run without problems. In order to compile and execute C programs with MPI I have downloaded and installed MPICH2. Now I can compile and run such programs without problems, specifying additional parameters for gcc, provided by MinGW. But when I want to compile and run a program that uses both OpenMP and MPI I have a problem. I specified both keys -fopenmp and keys for MPI program for gcc compiler. Compilator didn't give me any error. I tried to launch my program by mpiexec, provided by MPICH2. My program didn't want to work (It was a HelloWorld program and it didn't print anything to output). Please help me to compile and launch such programs correctly.
Here is my HelloWorld program, that doesn't produce any output.
#include <stdio.h>
#include <mpi.h>
int main(int argc, char ** argv)
{
int thnum, thtotal;
int pid, np;
MPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD,&pid);
MPI_Comm_size(MPI_COMM_WORLD,&np);
printf("Sequental %d out of %d!\n",pid,np);
MPI_Barrier(MPI_COMM_WORLD);
#pragma omp parallel private(thnum,thtotal)
{
thnum = omp_get_thread_num();
thtotal = omp_get_num_threads();
printf("parallel: %d out of %d from proc %d out of %d\n",thnum,thtotal,pid,np);
}
MPI_Barrier(MPI_COMM_WORLD);
MPI_Finalize();
return 0;
}
You can use the mpicc compiler with the -openmp option. For example,
mpicc -openmp hello.c -o hello
This might not be the root cause of your problem, but the MPI standard mandates that threaded programs use MPI_Init_thread() instead of MPI_Init(). In your case there are no MPI calls from within the parallel region so threading level of MPI_THREAD_FUNNELED should suffice. You should replace the call to MPI_Init() with:
int provided;
MPI_Init_thread(&argc, &argv, MPI_THREAD_FUNNELED, &provided);
if (provided < MPI_THREAD_FUNNELED)
{
MPI_Abort(MPI_COMM_WORLD, 1);
return 1; // Usually not reached
}
Although some MPI libraries might not advertise threading support (provided as returned is MPI_THREAD_SINGLE) they still work fine with hybrid OpenMP/MPI codes if one does not make MPI calls from within parallel regions.
The OpenMP portion of your program might require #include <omp.h> :
parallel: 0 out of 2 from proc 0 out of 0
parallel: 1 out of 2 from proc 0 out of 0