Private and Shared Variables

In threaded code, a variable can be shared or private within a parallel region.

A shared variable has the same address in the execution context of every thread.

A private variable has a different address in the execution context of every thread.

A thread cannot access the private variables of another thread. All threads can access shared variables.

There are default criteria for which variables are shared and which are private that are to some extent language-dependent.

The default for C/C++ (and Python where relevant) is all variables declared static, and all in whole-file scope are shared. Within a parallel region, anything not private is also shared.

In Fortran, by default all variables in COMMON or declared SAVE are shared. All module (before the CONTAINS) are shared.

The default for private variables is the first loop variable encountered (all languages). Variables on the stack in functions (C/C++) or subroutines (Fortran) that are invoked within a parallel region are also private by default.

Stack is a segment of memory used to store temporary variables in subprograms.Stack variables will go out of scope or may be automatically deallocated when the subprogram exits.

In C/C++, variables declared within a parallel region are private by default. In Fortran variables must be declared within the nonexecutable preamble to executable code, unless a BLOCK/END BLOCK section is declared. Variables can be declared within the block and these are private by default.

The defaults can be overridden with the private, firstprivate, lastprivate, shared, default, reduction, and copyin clauses to the directive. The clauses can be applied to a limited set of directives

Private Clauses

Private

A typical multidimensional loop is

for (i = 0; i < N; i++) {
    for (j = 0; j <N; j++) {
        a[i][j] = myMIN(a[i][j])+myMEAN(a[i][j];
    }
}

Either loop could be executed in parallel, but we prefer to make outer loops parallel, to reduce the number of forks/joins.

We then must give each thread its own private copy of variable j. The private clause directs the compiler to make one or more variables private. The syntax is the same for all languages.

private(<variable list>)

The private variable is set up within each thread and has no specific value until initialized within the parallel region.

C/C++

#pragma omp parallel for private(j)
    for (i = 0; i <N; i++)
        for (j = 0; j < N; j++)
            a[i][j] = myMIN(a[i][j)+myMEAN(a[i][j]);

Note that according to our rule for defaults, we could also make both loop indices as follows, without the need for the clause:

#pragma omp parallel for 
    for (int i = 0; i <N; i++)
        for (int j = 0; j < N; j++)
            a[i][j] = myMIN(a[i][j)+myMEAN(a[i][j]);

Fortran

Note the loop order for cache efficiency.

!$omp parallel do private(i)
do j=1,N
    do i=1,N
        a(i,j)=min(a(i,j),a(i,j)+tmp)
    enddo
enddo
!$omp end parallel do

Python Wrapping function omitted.

with omp("parallel for private(j)"):
    for i in range(N):
        for j in range(N):
            a[i,j]=min(a[i,j],a[i,j]+tmp)

Firstprivate/Lastprivate

The firstprivate clause is like `private but additionally initializes the private variable to its last value outside the parallel region.

The lastprivateclause sets the corresponding variable outside the parallel region to the final private value from the thread that executed the last iteration (in the case of for/do).

Example

These examples illustrate private, firstprivate, and lastprivate. Note that the Fortran version uses both the firstprivate and private clauses; this is permitted and fairly common, especially in Fortran.

The lastprivate clause is not yet implemented in OMP4Py at this time.

C++

#include <stdio.h>
#include <omp.h>

int main() {

    int x, y, z;
    int tid;

    int nthreads=4;

    x=10;

#pragma omp parallel for private(x) num_threads(nthreads)
    for (int i=0; i<nthreads; i++) {
        tid=omp_get_thread_num();
        x=1000*(tid+1);
        printf("Thread %d gets x= %d\n",tid, x);
    }
    
    printf("Outside parallel region x= %d\n", x);

    y=20;
#pragma omp parallel for firstprivate(y) num_threads(nthreads)
    for (int i=0; i<nthreads; i++) {
        tid=omp_get_thread_num();
        int w=y*100*(tid+1);
        printf("Thread %d gets w= %d\n",tid, w);
    }
    
    printf("Outside parallel region y= %d\n", y);

    z=30;
#pragma omp parallel for lastprivate(z) num_threads(nthreads)
    for (int i=0; i<nthreads; i++) {
        tid=omp_get_thread_num();
        z=3000*(tid+1);
        printf("Thread %d gets z= %d\n",tid, z);
    }
    
    printf("Outside parallel region z= %d\n", z);
}

Fortran

program privatevar
use omp_lib

   integer ::  tid
   integer ::  nthreads=4
   integer ::  w,x,y,z

   x=10

   !$omp parallel do private(x) num_threads(nthreads)
   do i=1,nthreads
      tid=omp_get_thread_num()
      x=1000*(tid+1)
      write(*,'(a,i3,a,i6)') "Thread ",tid," gets x=",x
   enddo
   !$omp end parallel do
    
   write(*,'(a,i6)') "Outside parallel region x= ",x

   y=20;
   !$omp parallel do firstprivate(y) private(w) num_threads(nthreads)
   do i=1,nthreads
      tid=omp_get_thread_num()
      w=y*100*(tid+1)
      write(*,'(a,i3,a,i6)') "Thread ",tid," gets w=",w
   enddo
   !$omp end parallel do
    
   write(*,'(a,i6)') "Outside parallel region y= ",y

   z=30
   !$omp parallel do lastprivate(z) num_threads(nthreads)
   do i=1,nthreads
      tid=omp_get_thread_num()
      z=3000*(tid+1);
      write(*,'(a,i3,a,i6)') "Thread ",tid," gets z=",z
   enddo
   !$omp end parallel do
    
   write(*,'(a,i6)') "Outside parallel region z= ",z

end program

Python

from omp4py import *

@omp
def private(nthreads):
    x=10
    with omp("parallel for private(x) num_threads(nthreads)"):
        for i in range(nthreads):
            tid=omp_get_thread_num()
            x=1000*(tid+1)
            print(f"Thread {tid} gets x={x} ")
    
    print(f"Outside parallel region x={x}")

@omp
def firstpriv(nthreads):
    y=20
    with omp("parallel for firstprivate(y) num_threads(nthreads)"):
        for i in range(nthreads):
            tid=omp_get_thread_num();
            w=y*100*(tid+1);
            print(f"Thread {tid} gets w={w} ")
    
    print(f"Outside parallel region y={y}")

"""
@omp
def lastpriv(nthreads):
    z=30
    with omp("parallel for lastprivate(y) num_threads(nthreads)"):
        for i in range(nthreads):
            tid=omp_get_thread_num();
            z=3000*(tid+1);
            print(f"Thread {tid} gets z={z}")
    
    print(f"Outside parallel region z= {z}")
"""


nthreads=4
private(nthreads)
firstpriv(nthreads)
#lastprivate not yet implemented, try later
#lastpriv(nthreads)