Building containers

Table of contents

  1. Building with sudo rights
  2. Building without sudo rights
  3. Building with small temporary directories
  4. Building a generic container
  5. Building an optimised container

Rebuilding containers is necessary if you want to change or add software, or optimise for a specific machine.

The recipes offer customisation options through environment variables near the top. Environment variables that a user may want to modify are summarised in the table below. Other variables should not requrie modification unless there are specific reasons for doing so.

Variable Values Purpose
HAS_CUDA true/false Install Nvidia CUDA and link IDG to it if set to true.
HAS_MKL true/false Install the Intel Math Kernel Library en link IDG to it if set to true.
MARCH GCC compatible architecture Passed to GCC’s -march option to generate code optimised for the specified CPU architecture.
MTUNE GCC compatible architecture Passed to GCC’s -mtune option to optimise code for the specified CPU architecture.
NOAVX512 true/false Disables the use of AVX512 instructions if set to true. Recommended when building for older or generic hardware on a modern machine, or when building AMD-specific containers on an Intel machine.
CPPSTD C++ standard Passed to GCC’s -std option when building C++ code, specifying which C++ standard to use.

In 2021 Singularity was renamed to Apptainer. There is little if any change in functionality, but if apptainer-related commands or environment variables mentioned below do not work for you, replace instances of apptainer with singularity and instances of APPTAINER_ with SINGULARITY_. See the official announcement for more information.

Building with sudo rights

If have root permissions on the machine you are building on, a container can be built with

sudo apptainer build <container name> <recipe file>

Building without sudo rights

If you do not have root permissions on the machine you are building on, the Singularity/Apptainer installation has to have the fakeroot feature enabled. Once that is available, a container can be built by running

apptainer build --fakeroot <container name> <recipe file>

There is a potential issue surrounding getopt and the Fedora base container on which FLoCs is built. If you encounter the following error during build:

INFO:    User not listed in /etc/subuid, trying root-mapped namespace
INFO:    The %post section will be run under fakeroot
INFO:    Starting build...
Copying blob 633a2a041c85 skipped: already exists  
Copying config 15e89348c0 done   | 
Writing manifest to image destination
2024/07/05 11:30:56  info unpack layer: sha256:633a2a041c85d22a814d80c8fd43a885066522afeea82db646cff81983f3dbaf
INFO:    Running post scriptlet
/.singularity.d/libs/fakeroot: line 46: getopt: command not found
/.singularity.d/libs/fakeroot: line 50: getopt: command not found
fakeroot, create a fake root environment.
   usage: fakeroot [-l|--lib fakerootlib] [-f|--faked fakedbin]
                   [-i file] [-s file] [-u|--unknown-is-real]
		   [-b|--fd-base fd] [-h|--help] [-v|--version]
                   [--] [command]
FATAL:   While performing build: while running engine: exit status 1

and getopt is available on your host machine, try including the following at the top of the recipe file as a stop-gap:

%files
    /usr/bin/getopt /usr/bin/getopt

Building with small temporary directories

If the machine you are running at has little space available on e.g. /var or /tmp you may run into trouble during the build. To avoid this, the build’s temporary and cache directories can be set as follows:

sudo APPTAINER_TMPDIR=<custom tempdir> APPTAINER_CACHEDIR=<custom cachedir> apptainer build <container name> <recipe file>

Building a generic container

A generic container is intended to be compatible with a wide variety of systems. This entails:

  1. Setting MARCH=x86-64 and MTUNE=generic to disable CPU architecture optmisation by setting.
  2. Setting NOAVX512=true to disable AVX512 instructions.
  3. Setting HAS_MKL=false to not use Intel MKL.

This should produce a container capable of running on a reasonably wide range of CPUs launched in the past decade or so.

Building an optimised container

Enabling compiler optimisations can result is a substantial performance increase over a generic container. This entails:

  1. Setting MARCH and MTUNE to values suitable for your CPU.
  2. Setting NOAVX512=false if your CPU supports AVX512 instructions.
  3. Setting HAS_MKL=true if you have an Intel CPU or if you have an AMD CPU using the AMD-specific recipe that uses the suite of AMD Optimised CPU Libraries (AOCL).

You can find the recommended march, mtune and AVX512 settings by running obtain_march_mtune.sh. It is important to use a recent GCC such that it can recognise your CPU properly. The container will currently use GCC 11.