How to make/configure an MPI project to run on multiple processes? - c

I have a small piece of code that should run on multilple processes, which is :
#include <stdio.h>
#include "mpi.h"
main(int argc, char **argv)
{
int ierr, num_procs, my_id;
ierr = MPI_Init(&argc, &argv);
/* find out MY process ID, and how many processes were started. */
ierr = MPI_Comm_rank(MPI_COMM_WORLD, &my_id);
ierr = MPI_Comm_size(MPI_COMM_WORLD, &num_procs);
printf("Hello world! I'm process %i out of %i processes\n",
my_id, num_procs);
ierr = MPI_Finalize();
}
the output is :
Hello world! I'm process 0 out of 1 processes. although it should run on more than one process
We edited the run configurations arguments to "-np 2" so it would run on 2 processes but it always gives us 1 process no matter what the value is.
The used environment is:
Eclipse Juno on Ubuntu 12.04
Source of code: http://condor.cc.ku.edu/~grobe/docs/intro-MPI-C.shtml[^]

It seems that you are trying to launch your MPI application directly, i.e. starting the compiled executable with -np 2, like this:
$ ./my_app -np 2
That's not the right way of launching MPI programs. Instead, you should call your MPI implementation's launcher (usually named mpirun or mpiexec), and pass the name of your executable and -np 2 to that. So if your executable is called my_app, then instead of the above command, you should run:
$ mpirun -np 2 ./my_app
Consult your MPI implementation's documentation for the specifics.

Some small points about mpirun or mpiexec commands:
If you are trying to run you app on multiple nodes, make sure that your app is copied to all the nodes.In addition, make sure that all needed binary files (executable & libraries) could be found directly from command line. I personally, prefer to run my MPI programs within a shell script like this:
#!/bin/sh
PATH=/path/to/all/executable/files \
LD_LIBRARY_PATH=/path/to/all/libraries \
mpirun -np 4 ./my_app arg1 arg2 arg3

Related

mpirun was unable to launch the specified application as it could not access or execute an executable with C

I created a clustering program using MPI using 4 nodes. But when I run the MPI program, I can't execute the program even though I've been given full access to the folder I created.
The program I made to count the number of processors running from each node/client
#include <mpi.h>
#include <stdio.h>
int main(int argc, char** argv){
MPI_Init(NULL,NULL);
int world_size;
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
int world_rank;
MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
char processor_name[MPI_MAX_PROCESSOR_NAME];
int name_len;
MPI_Get_processor_name(processor_name, &name_len);
printf("hello world from processor %s, rank %d out of %d processors\n",processor_name,world_rank,world_size);
MPI_Finalize();
return 0;
}
but when I run the program with the command : mpirun -np 8 --hostfile /mnt/sharedfolder/hostfile /mnt/sharedfolder/hello_world.out
a message like this appears
mpirun was unable to launch the specified application as it could not access
or execute an executable:
Executable: /mnt/sharedfolder/hello_world.out
Node: node-02
while attempting to start process rank 0.
--------------------------------------------------------------------------
8 total processes failed to start
previously I have given full access to the firewall on the google cloud platform on each node.please help me, I'm still learning

Open MPI - mpirun exits with error on simple program

I have recently installed OpenMPI on my computer and when I try to run a simple Hello World program, it exits with the next error:
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
This is the program's source code:
#include <mpi.h>
#include <stdio.h>
int main(int argc, char *argv[])
{
int size, rank;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
printf("Hello\n");
MPI_Finalize();
return 0;
}
This is how I compile the program:
mpicc -o hello hello.c
and I execute it with
mpirun -np 2 hello
It throws no errors on compilation, and if I run ./hello, it runs ok.
Excuse my english, any correction will be welcome.
you have to ./ the executable name
try this, mpirun -np 2 ./hello
Try:
mpirun -x LD_PRELOAD=libmpi.so -np 2 hello
If it works, you probably have an issue with your OpenMPI installation. A simple workaround would be to define an alias. If ou use bash, add in ~/.bashrc:
alias mpirun='mpirun -x LD_PRELOAD=libmpi.so'

OpenMPI Execution Problems on Raspberry Pi

I am trying to build a beowulf cluster with Raspberry Pi. I downloaded the following packages of OpenMPI: openmpi-bin, openmpi-common, libopenmpi1.3, libopenmpi-dbg, libopenmpi-dev. I used static IP on each of the Raspberrys and tested the connection between each of them and it was working. I also enabled the ssh and I used it to login from one Raspberry to all the other Raspberrys.
This is the following code I used:
#include <mpi.h>
int main(int argc, char *argv[])
{
int tid,nthreads;
char *cpu_name;
/* add in MPI startup routines */
/* 1st: launch the MPI processes on each node */
MPI_Init(&argc,&argv);
/* 2nd: request a thread id, sometimes called a "rank" from
the MPI master process, which has rank or tid == 0
*/
MPI_Comm_rank(MPI_COMM_WORLD, &tid);
/* 3rd: this is often useful, get the number of threads
or processes launched by MPI, this should be NCPUs-1
*/
MPI_Comm_size(MPI_COMM_WORLD, &nthreads);
/* on EVERY process, allocate space for the machine name */
cpu_name = (char *)calloc(80,sizeof(char));
/* get the machine name of this particular host ... well
at least the first 80 characters of it ... */
gethostname(cpu_name,80);
printf("hello MPI user: from process = %i on machine=%s, of NCPU=%i processes\n",
tid, cpu_name, nthreads);
MPI_Finalize();
return(0);
}
I tested the code first with only 2 boards and it worked fine no problems no errors and it printed the printf statement in the code.
I created a host file which includes the following:
Pi0
Pi1
Pi2
Pi3
I have 4 Raspberry Pis. I copied the code on each of them and compiled the code also using the following statements:
mpicc -g -0O -c hello-mpi.c -o hello-mpi.o
mpicc -g hello-mpi.o -o hello-mpi
I executed the code with the following statement:
mpirun -np 4 -hostfile hostfile hello-mpi
when I run the program nothing happens it just gives me an new empty line. No errors are given also. Any suggestion what I can I do to make it work

pass mpi rank to bash script through system() C/C+ ; bizzare error

I have an C, with MPI, code that should takes should execute a bash script in different directories depending on the mpi-rank. For example: If I run this with mpirun -np 10 mycode.o It should each process should execute the call system(s) where s is a string created from snprintf(s, sizeof(s), "/home/myaccount/myscript.sh %d", rank);
The full code is below:
#include <stdio.h>
#include <mpi.h> /* MPI header file */
#include <stdlib.h>
void main(int argc, char *argv[])
{
int rank, size;
/* init into MPI */
MPI_Init(&argc, &argv);
/* my rank - my id */
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
/* how many processes in the virtual machine */
MPI_Comm_size(MPI_COMM_WORLD, &size);
/*create script input (convert rank to string)*/
char s[256];
snprintf(s, sizeof(s), "/home/myaccount/myscript.sh %d", rank);
/* run script */
system(s);
/* out of the virtual machine */
MPI_Finalize();
}
And a test script is:
#!/bin/bash
for((k=16;k<22;k+=2));
do echo cd /otherdirectory/test/"$k"/"$1"/;
done;
exit;
which when I run with mpirun -np 10 should give:
cd /otherdirectory/test/16/0/
cd /otherdirectory/test/16/9/
cd /otherdirectory/test/16/8/
And so on. But instead returns
cd /otherdirectory/test/16/is_csh/
cd /otherdirectory/test/18/is_csh/
cd /otherdirectory/test/20/is_csh/
And I can't figure-out where the "is_csh" is coming from. If I use printf("%s\n") instead of system(s), I do get what I expect above.
Any ideas?
You output $k after /test/. $k runs from 16 to 20. So this is what you should expect.
I would assume that you have a .bashrc in your home directory, or maybe a /etc/bashrc, which trashes $1, so when your script is finally reached, the argument was overwritten by something else.
Or maybe your system shell is csh or tcsh or something similar, and you switch to bash after logging in. When you start your script from the command line, bash executes it directly. But the system() call will get your shell from /etc/passwd and run <shell> -c <args to system>, so a .login or .cshrc or similar file that gets executed by csh on startup might be the culprit as well.

MPI on PBS cluster Hello World

I am using mpiexec to run a couple of hello world executables. They each run, but the number of processes is always 1 where it looks like there should be 4 processes. Does someone understand why? Also I'm not sure why stty is giving me an invalid argument. Thanks!
Here is the output:
/bin/stty: standard input: invalid argument
Hello world from process 0 of 1
Hello world from process 0 of 1
Hello world from process 0 of 1
Hello world from process 0 of 1
Here is the c file:
#include <stdio.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
int rank, size;
MPI_Init (&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
printf("Hello world from process %d of %d\n", rank, size);
fflush(stdout);
MPI_Finalize();
return 0;
}
Here is the submission script:
#!/bin/bash
#PBS -N helloWorld
#PBS -l select=4:ncpus=2
#PBS -j oe
#PBS -o output
#PBS -l walltime=3:00
cd $PBS_O_WORKDIR
mpiexec ./helloWorld
Steven:
The above should work; it looks like something along the line (PBS <-> MPI library <-> mpiexec) is misconfigured.
The first, most obvious guess -- is the mpiexec the same mpi launching program that corresponds to the library you compiled with? If you do a which mpiexec in your script, do you get something that corresponds to the which mpicc when you compile the program? Do you have to do anything like a "module load [mpi package]" before you compile?
Similarly, is your mpiexec PBS-aware? If not, you might have to specify a hostfile (${PBS_NODEFILE}) somehow, and the number of processors.
What mpi are you using, and what system are you running on -- is it a publically available system with documentation we can look at?

Resources