Can "#pragma omp parallel for" be used inside a loop in the following form:
For (i=0;i<...;.i+=1)
{ #pragma omp parallel for
for(j=0;j<...;j+=1)
{ Some code.....}
Will this just parralelize the loop on 'j' ?
Thanks on advence !
Yes, it can be used like that. But compiler directives have to be on a line of their own
for( ... )
{
#pragma omp parallel for
for( ...
//..
Also, this will indeed only execute the inner loop in parallel. If you need both loops to execute in parallel you need a second #pragma omp parallel for above the outer loop.
It can be used like you said but this is not something good to have if I were you and want to avoid fork/join with each iteration I would do it like this:
#pragma omp parallel for
for(j loop)
{
for(i loop)
{
//Modify your code if necessary.
}
}
in the above snippet if your maximum thread count is 10 and your j length is 100 then there will be 10 threads each will run 10 iterations of j.
Related
I can write some code that looks like this:
if (make_parallel)
{
#pragma omp parallel for
for (int i=0; i<n; i++)
{
//For loop stuff
}
}
else
{
//Identical for loop, minus the parallelisation
for (int i=0; i<n; i++)
{
//For loop stuff
}
}
Is there a tidier way of doing this, such that the for loop doesn't have to be duplicated?
[Edit] - So this solution works at the preprocessing level
#define USE_OPENMP
//...
#ifdef USE_OPENMP
#pragma omp parallel for
#endif
for(int i=0;i<n;i++)
But that isn't ideal. But the more I think about it, since defining the parallel region is also preprocessing it probably can't be made conditional in a tidy way?
You can simply use a if clause like this:
int enabled = 1;
#pragma omp parallel if(enabled)
#pragma omp for nowait
for(int i=0;i<n;i++)
Note that this condition could be evaluated by the runtime and not the compiler (GCC does that for example). As a result, the overhead can be significantly higher than your if-else solution for very small loops or the ones in which each iteration is very cheap. The reason is that compilers often do not propagate constants to the OpenMP callback computing function and so the code can make unnecessary operations. This solution has the benefit of being cleaner, helping compilers to produce a smaller binaries and also help to reduce the compilation time (since the loop is compiled once).
I would like to set the number of threads in OpenMP. When I use
omp_set_num_threads(2);
printf("nthread = %d\n", omp_get_num_threads());
#pragma omp parallel for
...
I see nthreads=1. Explained here, the number of reported threads belongs to serial section which is always 1. However, when I move it to the line after #pragma, I get compilation error that after #pragma, for is expected. So, how can I fix that?
Well, yeah, omp parallel for expects a loop in the next line. You can call omp_get_num_threads inside that loop. Outside a parallel section, you can call omp_get_max_threads for the maximum number of threads to spawn. This is what you are looking for.
int max_threads = omp_get_max_threads();
#pragma omp parallel for
for(...) {
int current_threads = omp_get_num_threads();
assert(current_threads == max_threads);
}
#pragma omp parallel
{
int current_threads = omp_get_num_threads();
# pragma omp for
for(...) {
...
}
}
My OpenMP program of matrix multiplication which consists of nesting of for loops is taking more execution time than the non-nested version of the parallel program. This is the block where I have used nested parallelisation.
pragma omp parallel
omp_set_nested(1);
#pragma omp parallel for
for(i=0;i<N;i++) {
#pragma omp parallel for
for(j=0;j<N;j++) {
C[i][j]=0.; // set initial value of resulting matrix C = 0
#pragma omp parallel for
for(m=0;m<N;m++) {
C[i][j]=A[i][m]*B[m][j]+C[i][j];
}
printf("C:i=%d j=%d %f \n",i,j,C[i][j]);
}
}
I am currently working on a matrix computation with OpenMP. I have several loops in my code, and instead on calling for each loop #pragma omp parallel for[...] (which create all the threads and destroy them right after) I would like to create all of them at the beginning, and delete them at the end of the program in order to avoid overhead.
I want something like :
#pragma omp parallel
{
#pragma omp for[...]
for(...)
#pragma omp for[...]
for(...)
}
The problem is that I have some parts those have to be execute by only one thread, but in a loop, which contains loops those have to be execute in parallel... This is how it looks:
//have to be execute by only one thread
int a=0,b=0,c=0;
for(a ; a<5 ; a++)
{
//some stuff
//loops which have to be parallelize
#pragma omp parallel for private(b,c) schedule(static) collapse(2)
for (b=0 ; b<8 ; b++);
for(c=0 ; c<10 ; c++)
{
//some other stuff
}
//end of the parallel zone
//stuff to be execute by only one thread
}
(The loop boundaries are quite small in my example. In my program the number of iterations can goes until 20.000...)
One of my first idea was to do something like this:
//have to be execute by only one thread
#pragma omp parallel //creating all the threads at the beginning
{
#pragma omp master //or single
{
int a=0,b=0,c=0;
for(a ; a<5 ; a++)
{
//some stuff
//loops which have to be parallelize
#pragma omp for private(b,c) schedule(static) collapse(2)
for (b=0 ; b<8 ; b++);
for(c=0 ; c<10 ; c++)
{
//some other stuff
}
//end of the parallel zone
//stuff to be execute by only one thread
}
}
} //deleting all the threads
It doesn't compile, I get this error from gcc: "work-sharing region may not be closely nested inside of work-sharing, critical, ordered, master or explicit task region".
I know it surely comes from the "wrong" nesting, but I can't understand why it doesn't work. Do I need to add a barrier before the parallel zone ? I am a bit lost and don't know how to solve it.
Thank you in advance for your help.
Cheers.
Most OpenMP runtimes don't "create all the threads and destroy them right after". The threads are created at the beginning of the first OpenMP section and destroyed when the program terminates (at least that's how Intel's OpenMP implementation does it). There's no performance advantage from using one big parallel region instead of several smaller ones.
Intel's runtimes (which is open source and can be found here) has options to control what threads do when they run out of work. By default they'll spin for a while (in case the program immediately starts a new parallel section), then they'll put themselves to sleep. If the do sleep, it will take a bit longer to start them up for the next parallel section, but this depends on the time between regions, not the syntax.
In the last of your code outlines you declare a parallel region, inside that use a master directive to ensure that only the master thread executes a block, and inside the master block attempt to parallelise a loop across all threads. You claim to know that the compiler errors arise from incorrect nesting but wonder why it doesn't work.
It doesn't work because distributing work to multiple threads within a region of code which only one thread will execute doesn't make any sense.
Your first pseudo-code is better, but you probably want to extend it like this:
#pragma omp parallel
{
#pragma omp for[...]
for(...)
#pragma omp single
{ ... }
#pragma omp for[...]
for(...)
}
The single directive ensures that the block of code it encloses is only executed by one thread. Unlike the master directive single also implies a barrier at exit; you can change this behaviour with the nowait clause.
I have this parallel for loop
struct p
{
int n;
double *l;
}
#pragma omp parallel for default(none) private(i) shared(p)
for (i = 0; i < p.n; ++i)
{
DoSomething(p, i);
}
Now, it is possible that inside DoSomething(), p.n is increased because new elements are added to p.l. I'd like to process these elements in a parallel fashion. OpenMP manual states that parallel for can't be used with lists, so DoSomething() adds these p.l's new elements to another list which is processed sequentially and then it is joined back with p.l. I don't like this workaround. Anyone knows a cleaner way to do this?
A construct to support dynamic execution was added to OpenMP 3.0 and it is the task construct. Tasks are added to a queue and then executed as concurrently as possible. A sample code would look like this:
#pragma omp parallel private(i)
{
#pragma omp single
for (i = 0; i < p.n; ++i)
{
#pragma omp task
DoSomething(p, i);
}
}
This will spawn a new parallel region. One of the threads will execute the for loop and create a new OpenMP task for each value of i. Each different DoSomething() call will be converted to a task and will later execute inside an idle thread. There is a problem though: if one of the tasks add new values to p.l, it might happen after the creator thread has already exited the for loop. This could be fixed using task synchronisation constructs and an outer loop like this:
#pragma omp single
{
i = 0;
while (i < p.n)
{
for (; i < p.n; ++i)
{
#pragma omp task
DoSomething(p, i);
}
#pragma omp taskwait
#pragma omp flush
}
}
The taskwait construct makes for the thread to wait until all queued tasks are executed. If new elements were added to the list, the condition of the while would become true again and a new round of tasks creation will happen. The flush construct is supposed to synchronise the memory view between threads and e.g. update optimised register variables with the value from the shared storage.
OpenMP 3.0 is supported by all modern C compilers except MSVC, which is stuck at OpenMP 2.0.