Page tree
Skip to end of metadata
Go to start of metadata

1.0 MODULES

To be able to deal with numerous software/libraries/compilers and their versions, Ukko2 and Kale use a module system. With the module system you can easily customize your working environment to be exactly as you want it.

Modules system modifies the environment variables of the user's shell so that the correct versions of executables are in the path and linker can find the correct version of needed libraries. For example, the command mpicc points to different compilers depending of the loaded module. Loaded modules do propagate to the compute nodes like any other environment variables when you submit a slurm job.

It is recommended to do a 'module purge' followed by 'module load <intended module list here>' in the batch script immediately after #SBATH -lines. This guarantees that your login host module environment is purged and replaced with intended execution environment.

Note that not all modules are compatible with each other. Most of the modules have dependencies set, and they check for compatibility, but there may be rare exceptions.

1.1 Module system, programs and modules

To list all programs that have been installed:

rpm -qa

To list all available modules that are compatible with the modules that you have currently loaded:

module avail

To load a module:

module load <modulename>

To switch or swap between two modules:

module swap <module1> <module2>

To list all loaded modules:

module list

To get info about a module:

module help <module-name>

To list all available modules:

module spider

If you add the name or part of the name as an argument, you will get more information about matching modules. If a version number is provided, you also see that module's requirements.
module spider int

To unload a module:

module unload <module-name>

To unload all modules:

module purge

2.0 Virtual Environments

To create a virtual environment, in this example Python 3.5.2:

Purging modules

If not sure if correct modules are loaded, or not sure about dependencies, it is useful to purge and load correct set.

module purge
module load Python/3.5.2-foss-2016b
mkdir python352 # any directory name, for your virtual environment
virtualenv python352
cd python352
source bin/activate pip install biopython # etc, any python packages you need

You may also install additional packages with pip inside the virtualenv:

./configure --prefix=$HOME/python352
make
make install

For example Tensorflow does need virtual environment to be set up.

If a module or a program you need is missing, contact the it support it4science@helsinki.fi


3.0 Additional Resources

NERSC has a very good resources about modules, including examples if you wish to create your own.

If you wish to build your own modules.