Building and Running C/C++/Fortran Codes on HPC Systems | RC Learning Portal

Building an Executable

Tue, 14 Apr 2026 16:42:59 +0000

Users who write their own programs in C, C++, or Fortran, or who use community codes written in these languages, must always convert their source code into a runnable executable before they can use the software. The process of creating an executable is called building, and consists of compiling, or converting source into machine-language files, and linking, which links these files along with any external libraries into an executable.

Compilers

A compiler is a program that converts human-written source code directly into a standalone program called an executable (or binary). This is in contrast to an interpreter, which is a program that executes source code, often called a script in this case, line by line.

Compilers go through a multi-stage process to convert source code to an executable.

When running an interpreter, the executable is the interpreter itself. Your script cannot be run directly. Some scripts can invoke their own interpreters and run standalone, but they are not themselves binaries.

Linkers

The compiler first produces an object file for each source file. In Unix these end in .o

Object files are binary (machine language) but cannot be executed. They must be linked into an executable.

Libraries are special archives of compiled code that can be invoked through their application programming interface, or API. Like the object files, they must be linked into an executable in order to be utilized.

The program that generates the executable from object files and any external libraries is the linker (also called a loader). Linkders are nearly always invoked through the compiler, not separately. The linker joins all object files as specified, and if run through the appropriate compiler it also links the compiler’s runtime libraries for the source language. These are libraries used to carry out procedures intrinsic to the language. This happens automatically if the compiler matches the language of the main program; it is not necessary to add the runtime libraries explicitly. However, in mixed-language programming it may be necessary to add them to the linking instructions.

Building

The process of compiling and linking is usually called buildint the executable. The result is a file in machine language which cannot be read by (most) humans. Binaries/executables are specific to a platform, a combination of machine architecture and operating system. You cannot run a Windows binary on a Linux system, and vice versa.

Compiler Suites

Tue, 14 Apr 2026 16:42:59 +0000

Compilers are usually run through either an Integrated Development Environment, such as VSCOde, or through the command line in a shell. They are usually shipped by their vendor in suites which consist of multiple programming languages sharing a common backend.

Lmod

Many HPC sites, including UVA’s, provide multiple compiler suites and compiler versions, which we manage through lmod modules. These should not be confused with modules in programming languages such as Fortran or Python; in this context the module is a script that sets up the environment for a specified software package.

For compilers, loading a module sets some necessary variables so that the correct compiler(s), linkers, and runtime libraries are in scope in the user’s environment.

Modules are loaded with the load command.

module load gcc

sets up the default version of the GCC suite.

Modules set important environment variables such as LOAD_LIBRARY_PATH variable. Modules will set this variable appropriately, but since it must have the correct value, the modules used to build must also be loaded for running.

Gnu Compiler Collection (gcc)

The GNU Compiler Collection includes front ends for C, C++, and Fortran, as well as libraries for these languages (libc, libstdc++, libgfortran).

gcc is the C compiler
g++ is the C++ compiler
gfortran is the Fortran compiler

To see a list of available versions, type

module spider gcc

This will result in output similar to (versions will vary over time):

---------------------------------------------------------------------------
  gcc:
----------------------------------------------------------------------------
    Description:
      The GNU Compiler Collection includes front ends for C, C++,
      Objective-C, Fortran, Java, and Ada, as well as libraries for these
      languages (libstdc++, libgcj,...).

     Versions:
        gcc/system
        gcc/7.1.0
        gcc/9.2.0
     Other possible modules matches:
        gcccuda

Intel

Intel compilers frequently produce the fastest binaries for Intel architectures and are usually recommended for users who want the best performance.

icx is the C compiler
icpx is the C++ compiler
ifx is the Fortran compiler

module spider intel

NVIDIA HPC SDK

We also offer the NVIDIA HPC SDK (software development kit) compilers. This suite is particularly strong at programming for general-purpose GPUs, mainly of NVIDIA architecture. They provide tools such as OpenACC and OpenMP for programming for GPGPUs, but also support interfaces to CUDA through the higher-level languages, in particular C++ and Fortran.

nvcc
nvc++
nvfortran

module spider nvhpc

Compiling a Single-File Program

Tue, 14 Apr 2026 16:42:59 +0000

Suppose your program is short and can be contained within a single file. If not told otherwise a compiler will attempt to compile and link the source file(s) it is instructed to compile in one step. With only one file, no separate invocation of the linker is required.

The default name for the executable is a.out. The option -o is used to name the binary something else.

Examples for each compiler:

Build a C program
- gcc -o myprog mycode.c
- icx -o myprog mycode.c
- nvcc -o myprog mycode.c
Build a C++ program
- g++ -o myprog mycode.cxx
- icpx –o myprog mycode.cxx
- nvc++ -o myprog mycode.cxx
Build a Fortran program
- gfortran –o myprog mycode.f90
- ifx –o myprog mycode.f90
- nvfortran –o myprog mycode.f90

Exercise

Load a compiler module (your choice). Copy one of the following files from /share/resources/tutorials/compilers to your home directory, or to a directory you create for the examples.

mycode.c
mycode.cxx
mycode.f90

Use the appropriate compiler to build your executable. Choose any name you wish for the executable.

File Permissions

On a Unix system, an executable must have the right permission. Unless something goes very wrong, the compiler/linker will set the “execute bit” for your executable; you should not need to do that yourself.

In your directory that contains at least one executable run

ls -l

Make sure you are in the same directory as the executable you want to run. Type

./myprog

Use the name you assigned your executable. ./ means current directory and is necessary because that is not in your default path.

Compiling and Linking Multiple Files

Tue, 14 Apr 2026 16:42:59 +0000

Most programs consist of multiple files. For Unix compilers the -c option suppresses linking. The compiler must then be run again to build the executable from the object files.

g++ -c mymain.cxx
g++ -c mysub1.cxx
g++ -c mysub2.cxx 
g++ -o myprog mymain.o mysub1.o mysub2.o

These compiler flags are similar for all three compiler suites.

ifx -c mymain.f90 
ifx -c mysub1.f90 
ifx -c mysub2.f90
ifx -o myprog mymain.o mysub1.o mysub2.o

Linkers and Libraries

When the executable is created any external libraries must also be linked. The compiler will search a standard path for libraries. On Unix this is typically /usr/lib, /usr/lib64, /usr/local/lib, /lib

There are two kinds of library, static and dynamic. If the library name ends in .a it is a static library; in this case, the machine code is bundled into the executable at compile time. If the name ends in .so it is a shared library; the machine code is loaded at runtime. For shared libraries, the compiler must know the path to the library at compile time, and the executable must be able to find the library at runtime. For Unix/Linux systems, shared librries are the default.

If you must use paths other than the defaults, you must specify those paths to the compiler. -L followed by a path is the option for Unix compilers; then the library must be named libfoo.a or libfoo.so. Another option provides the shortened name of the library; it is referenced as -lfoo in either case.

Example

g++ -o mycode -L/usr/lib64/foolib mymain.o mysub1.o mysub2.o -lfoo

As a general rule, libraries must be compiled by the same compiler that you use for your program. However, Intel and NVHPC compilers can usually link C (not necessarily C++) system libraries.

For most Unix compilers, order matters! The object file containing the code being called must be linked after the object file that uses it. Libraries come last, and must also be in the correct order.

Library and Header Paths on HPC

Tue, 14 Apr 2026 16:42:59 +0000

Many codes need headers (C/C++) or modules (Fortran) for procedure prototypes, etc. This is especially common if it is using an external library. As was true for libraries, there is a system default search path for these “include files.”

The compiler flag is -I/path/to/includes

The path to Fortran module files is also specified by the -I flag.

The lmod modules system we use is hierarchical. You must first load your choice of compiler. From there you can type

module avail

The modules at the top will be for libraries built with that compiler. You will generally need to provide paths for these libraries by -L and -I flags. You can find the paths with

printenv | grep ROOT

Most recent modules have an environment variable NAME_ROOT, e.g.

HDF5_ROOT

Example

A number of scientific and engineering codes need the input/output libraries HDF5 or NetCDF. We will use NetCDF for this example.

module load gcc
module load netcdf

printenv | grep ROOT

The following should be typed on a single line:

gfortran -o myexec -L${NETCDF_ROOT}/lib -I${NETCDF_ROOT}/include mycode.f90 -lnetcdff -lnetcdf

Exercises

Exercise 1. Multiple files (no C example here)

Copy from /share/resources/tutorials/compilers your choice of files

Copy main.cxx or main.f90
Copy sub1.cxx or sub1.f90
Copy sub2.cxx or sub2.f90
C++ copy sub1.h and sub2.h as well

Use these three files to create an executable.

Exercise 2. External Libraries

Load the netcdf module for your choice of compiler suite as indicated above.
Copy simple_xy_wr.c or simple_xy_wr.cpp or simple_xy_wr.f90 (note the C++ suffix is cpp here).
Compile and link the examples. You will need to add -I${NETCDF_ROOT}/include and -L${NETCDF_ROOT}/lib to your link command. For C use -lnetcdf for the library. For Fortran use -lnetcdff -lnetcdf in that order. For C++ use -lnetcdf_c++ -lnetcdf in that order.

Build Tools: Make

Tue, 14 Apr 2026 16:42:59 +0000

Building a program with more than one or two files can become very difficult to manage. Various build tools have been developed.

Make

The oldest and still most common is make. Even many newer tools like cmake generate a makefile, at least on Unix.

Make is a tool to manage builds. It has a rigid and peculiar syntax. It will look for a file named makefile first, followed by Makefile (on case-sensitive systems). The makefile defines one or more targets. The target is the product of one or more rules.

The target is defined with a colon following its name. If there are dependencies those follow the colon. Dependencies are other files that are required to create the current target.

Targets and Rules

Example

myexec: main.o module.o 
<tab>gfortran -o myexec main.o module.o

The tab is required in the rule. Don’t ask why. The angle brackets are to indicate the character and are not typed.

Macros (automatic targets) for rules:

$@ represents the file name of the current target
$< represents the name of the first prerequisite

Variables, Comments, and Continuations

We can define variables in makefiles

F90=gfortran
CC=gcc
CXX=icpc

We then refer to them as $(F90), $(CC), etc.

Common variables: F90, CC, CXX, FFLAGS, F90FLAGS, CFLAGS, CXXFLAGS, CPPFLAGS (for the preprocessor), LDFLAGS.

Comments may be inserted into a Makefile. Anything from a # onward is ignored, unless it is a backslash \.

The backslash is the line-continuation marker. Be sure it is the last character on the line. If it appears as the last character in a comment line, it allows the comment to be extended over multiple lines.

Suffix Rules

If all files with a given suffix (.c, .cxx, .f90, etc.) are to be compiled the same way, we can write a suffix rule to handle them. This uses a phony target called .SUFFIXES. The rule must begin with a tab as usual.

.SUFFIXES: .f90 .o
<tab>$(F90) $(F90FLAGS) -c $< 

.SUFFIXES: .cxx .cpp .o 
<tab>$(CXX) -c $(CXXFLAGS) -c <$

Pattern Rules in Make

Tue, 14 Apr 2026 16:42:59 +0000

This is an extension by Gnu make (gmake), but nearly every make, especially on Linux, is gmake now. They are similar to suffix rules.

Gmake contains built-in pattern rules so it can handle common cases if you do not write your own rules. For example, to compile a C code it will by default use

%.o : %.c
        $(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@

A pattern rule that is particularly useful for Fortran 90+:

%.mod: %.o

This can be helpful because make’s default pattern rule for .mod is for a programming language called Modula, but Fortran uses .mod for compiled module interface files. Adding this pattern rule overrides the built-in rule for that suffix.

Make Options

Files with names other than makefile or Makefile can be used with the -f option

make -f Make.linux

Make creates the first target in the file unless a specific target is specified.(Some Makefiles are written to require a target.). To generate a different target, provide its name on the command line.

make pw.x

makemake

Tue, 14 Apr 2026 16:42:59 +0000

The makemake command is a script that will create a skeleton Makefile. Users will have to edit it to provide names, paths as needed, etc. but it is far easier than starting from scratch.

Create a directory for the source files you wish to build and type

makemake

The makemake script cannot distinguish files to be included in the build versus other files you may have in the directory. It will pick up all files ending in .c, .cxx, .cpp, .f, or .f90. First edit the Makefile to remove any irrelevant file names.

At minimum you must name your program through the PROG variable. You should also specify the name of your compiler explicitly; e.g. gcc not cc.

The “phony” target clean is common in Makefiles. It makes it easy to remove all compiled files and start over. At minimum, this is necessary when compiler options are changed.

Makemake Skeleton:

PROG = 

SRCS =  code.cxx file1.cxx file2.cxx 

OBJS =  code.o file1.o file2.o

LIBS = 

CC = cc

CXX = c++

CFLAGS = -O 

CXXFLAGS = -O

FC = f77 

FFLAGS = -O 

F90 = f90 

F90FLAGS = -O

LDFLAGS = 

all: $(PROG) 

$(PROG): $(OBJS)
        $(CXX) $(LDFLAGS) -o $@ $(OBJS) $(LIBS) 

.PHONY: clean

clean: 
        rm -f $(PROG) $(OBJS) *.mod 

.SUFFIXES: $(SUFFIXES) .f .f90 .F90 .f95

.SUFFIXES: $(SUFFIXES) .c .cpp .cxx 

.f90.o .f95.o .F90.o: 
        $(F90) $(F90FLAGS) -c $< 

.f.o: 
        $(FC) $(FFLAGS) -c $< 

.c.o: 
        $(CC) $(CFLAGS) -c $< 

.cpp.o .cxx.o:
        $(CXX) $(CXXFLAGS) -c $< 

main.o: code.cxx file1.h file2.h

Exercise

Copy or move your multi-file program into its own directory. Run makemake in the directory.

Edit the resulting Makefile appropriately. You may remove any lines pertaining to languages you aren’t using.

Run make.

Check that your executable works.

Configure

Tue, 14 Apr 2026 16:42:59 +0000

The configure is a way to generate a Makefile automatically.

We will not discuss creating configure scripts, since they can be quite complex. But much software is distributed with them. Configure usually takes many options and they vary by package. To see them, run from its directory

./configure --help

Most configure scripts assume the software will be installed into /usr or /usr/local. You do not have permission to do this on an HPC system, so you will nearly always need to use the -prefix option.

./configure -prefix=/home/mst3k/udunits

Configure should produce a Makefile. When it has completed you can run make as usual. This is usually followed by

make install

Refer to the documentation for your package for more details.

Exercise

Copy udunits.tar.gz from /share/resources/tutorials/compilers

Untar it (tar xf udunits.tar.gz). Cd into its directory. Run

configure --help

After examining the options, run it with at least the prefix option. You may wish to create a new directory for installing the package.

Build the library.

Install the library into the directory prepared for it.

CMake

Tue, 14 Apr 2026 16:42:59 +0000

CMake is a platform-independent build system. Unlike configure, it can be used on Windows natively. It is popular especially for software that must be built for multiple platforms.

CMake usually requires the creation of a separate build directory below the top-level directory of the project. CMake uses -D flags as the equivalents of many options to configure. CMake caches your configuration into a file named CMakeCache.txt – if you make changes you must remove this file or your changes will be ignored. It will be in the build directory if you created one.

Useful CMake Flags

You will often need to add

-DCMAKE_INSTALL_PREFIX=/path/to/install/location

This is the equivalent of the -prefix option to configure.

CMake does not pay (much) attention to your paths. It has its own means of finding compilers and libraries. You may need to set some environment variables if you use a compiler loaded from an lmod module.

-DCMAKE_C_COMPILER=gcc
-DCMAKE_CXX_COMPILER=g++
-DCMAKE_FC_COMPILER=gfortran

There are many versions of CMake and some CMake scripts provided by developers require a particular version or range of versions. The system cmake may be old or not suitable. Newer versions are available through modules.

module spider cmake

The default module version should be sufficiently recent to build most cmake projects.

module load cmake

Example

We will use an example prepared by Thom Troy (ttroy50 on GitHub) Copy cmake-example.tar.gz from /share/resources/tutorials/compilers

Extract the tar-file and cd to the directory it creates.

Make an installation directory.

Make a build directory. Cd into it.

Load a compiler if you haven’t already, for example gcc.

Load the cmake module.

Run (type cmake command all on one line)

cmake .. -DCMAKE_INSTALL_PREFIX=/home/yourid/yourdir -DCMAKE_CXX_COMPILER=g++

make

This simple example doesn’t create an installation target, so move the binary to the intended location.

Shared Memory Parallel Codes

Tue, 14 Apr 2026 16:42:59 +0000

So far we have assumed that the code to be built is serial, i.e it runs on a single core. However, many programs that are intended to run on HPC systems make use of some form of parallelism.

Shared-memory parallelism can be used on individual workstations, or on a single node of an HPC system, as long as multiple cores are present on the device. Since nearly all modern computing devices are multicore, shared-memory parallelism is widely used in many domains. Subprocesses called threads are started by a root process. Each thread (including the root) is run on its own core.

On a multinode HPC system, programs using shared-memory parallelism must be restricted to run on a single node. This is usually handled by a resource manager such as Slurm. The directive for Slurm is

#SBATCH --cpus-per-task=N

#SBATCH -c N

where N represents the number of cores requested. This directive ensures that all cores assigned to the job are on the same node.

OpenMP

OpenMP is one of the most popular libraries for this type of application. OpenMP is built in to the compiler, so whether it is supported and which version is implemented is determined by the compiler vendor. It is invoked via the compiler and linker flags.

gcc -fopenmp
Intel -qopenmp
NVHPC -mp

Exercise

Copy omphello.c or omphello.f90 from /share/resources/compilers

Using your choice of compiler, build an executable

gcc –fopenmp omphello.c
icc –qopenmp omphello.c
pgcc –mp omphello.c

gfortran –fopenmp omphello.f90
ifort –qopenmp omphello.f90
pgfortran –mp omphello.c

Add a -o <name> flag to your build step if you want a name other than a.out.

Generally the default for OpenMP is to detect the cores and create one thread per core in parallel regions. This is not always desirable for several reasons. We can specify the number of threads for parallel regions with an environment variable. For OpenMP we use the OMP_NUM_THREADS environment variable.

Run your executable with 4 threads:

export OMP_NUM_THREADS=4
./a.out

On shared, resource-managed systems like Rivanna, The thread number must equal the number of cores requested. We can automate this with a Slurm built-in environment variable. In your SLURM script set

export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}

Building and Running C/C++ and Fortran Codes on the HPC System

Fri, 28 Feb 2020 00:00:00 -0500

Distributed Memory Parallel Programs

Tue, 14 Apr 2026 16:42:59 +0000

Programs using distributed-memory parallelism can run on multiple nodes. They consist of independent processes that communicate through a library, usually the Message Passing Interface MPI.

Building and Running MPI Programs

MPI is an external library. It must be built for the compiler with which it will be used. We provide compiled versions of MPI for gcc and NVHPC. For Intel we recommend using the vendor’s IntelMPI.

OpenMPI with gcc and NVHPC

module load gcc

module load nvhpc

Follow this with

module load openmpi

Compile and link your code with the wrappers

mpicc for C
mpicxx for C++
mpif90 for Fortran

The wrappers insert the appropriate header/module paths and library paths, and also the correct libraries to link. You do not add them explicitly.

IntelMPI with Intel

module load intel

Use the wrappers

mpiicx for C
mpiicpx for C++
mpiifx for Fortran

Running MPI Programs

MPI programs must be run under the control of an executor program. Usually this is called mpirun or mpiexec, but when submitting to SLURM we must use mpirun for OpenMPI and srun for IntelMPI. The executor must be told how many processes to start, and if on more than one host the list of host IDs must be provided. On a local system this is controlled by command-line options to mpirun, e.g.

mpirun -np 8 ./mympicode

However, on Slurm the provided MPI executors obtain the number of processes and the hostlist directly from the job assignments. Do not specify them on the command line.

#Intel
srun ./mympicode 
#OpenMPI
mpirun ./mympicode

To request resources through Slurm for an MPI program, use the following directives

#SBATCH -N NN
#SBATCH --ntasks-per-node=NT

where NN is the number of nodes and NT is the number of tasks (processes) to run on each node. The total number of processes will be NN*NT.

Exercise

Copy the file mpihello.c or mpihello.f90 from /share/resources/tutorials/compilers. C++ programmers please note: the C++ bindings are deprecated in MPI, we just use the C routines.

Load the appropriate module for the compiler you are using.

Build the code with the appropriate wrapper.

You can run a quick test on the frontend with 4 cores. You do not need a hostfile if all processes are on one node.

Write a SLURM script using srun to run the program. Submit to the parallel partition for testing, using at least two nodes.

Editors and Integrated Development Environments

Tue, 14 Apr 2026 16:42:59 +0000

Integrated Development Environments (IDEs) are a graphical user interface for code development. A currently popular IDE is VSCode. We recommend that VSCode be run through our Open OnDemand VSCode interactive application.

Programmers who want to use a graphical user interface to edit source files, but not necessarily a full IDE, can use them on the FastX graphical frontend.