Modules: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
{{Portal|User}}
== Software and Modules ==
== Software and Modules ==


Typically with many scientific software packages, the Grid5000 users will compile and install them into their home directories or managed them in Kadeploy customized environment. To make building and maintaining the user software environment easier we have installed a number of software packages with [https://spack.readthedocs.io/en/latest/ Spack] package management system. These software packages are managed by the modules environment management system and available directly on Grid5000 SSH frontends or cluster's nodes.
Grid'5000 provides a set of software (mainly scientific-related) using the ''module'' command line tool. They are available from Grid5000 frontends or cluster's nodes (only on standard, big, and nfs environment if deployment is used).


Please take a look and see what is available for your use. If you need other software installed that will be of use to the Grid5000 community in general, feel free to contact us at [mailto:users@lists.grid5000.fr Grid5000 users mailing-list] and we can look into it.
If you need additional software to be installed, feel free to contact [mailto:support-starff@lists.grid5000.fr Grid5000 support team] and we can look into it.


==== Modules ====
==== Modules ====


The module system is designed to manage your environment to make available software packages in your PATH. To get started, it is great to look a a listing of what is available.
The module system is designed to load software and make it available by modifying environment (such as your PATH variable).
 
To get started, list available software using:


  $ module av
  $ module av
  ------------------ /grid5000/spack/share/spack/modules/linux-debian9-x86_64 ------------------
  --------------------------------------------- /grid5000/spack/share/spack/modules/linux-debian9-x86_64 ---------------------------------------------
  cuda/9.2.88_gcc-6.4.0         jdk/11.0.2_9_gcc-6.4.0         openmpi/2.0.4_gcc-6.4.0
autoconf/2.69_gcc-6.4.0                        gcc/7.4.0_gcc-6.4.0                            llvm/7.0.1_gcc-6.4.0
  cudnn/5.1_gcc-6.4.0           libfabric/1.7.0_gcc-6.4.0     openmpi/3.0.0_gcc-6.4.0
automake/1.16.1_gcc-6.4.0                      gcc/8.3.0_gcc-6.4.0                            memkind/1.7.0_gcc-6.4.0
  cudnn/6.0_gcc-6.4.0           likwid/4.2.1_gcc-6.4.0         openmpi/3.1.3_gcc-6.4.0
boost/1.69.0_gcc-6.4.0                        gmp/6.1.2_gcc-6.4.0                            miniconda2/4.5.11_gcc-6.4.0
cmake/3.13.4_gcc-6.4.0                        hwloc/1.11.11_gcc-6.4.0                        miniconda3/4.5.11_gcc-6.4.0
cuda/10.0.130_gcc-6.4.0                        hwloc/2.0.2_gcc-6.4.0                          mpfr/4.0.1_gcc-6.4.0
cuda/7.5.18_gcc-6.4.0                          intel-mkl/2017.4.239_gcc-6.4.0                netlib-lapack/3.8.0_gcc-6.4.0
cuda/8.0.61_gcc-6.4.0                          intel-mkl/2018.1.163_gcc-6.4.0                netlib-xblas/1.0.248_gcc-6.4.0
  cuda/9.0.176_gcc-6.4.0                        intel-mkl/2019.1.144_gcc-6.4.0                numactl/2.0.12_gcc-6.4.0
cuda/9.1.85_gcc-6.4.0                         intel-mpi/2019.1.144_gcc-6.4.0                 openblas/0.3.5_gcc-6.4.0
cuda/9.2.88_gcc-6.4.0                          intel-parallel-studio/cluster.2019.2_gcc-6.4.0 openmpi/3.1.3_gcc-6.4.0
  cudnn/5.1_gcc-6.4.0                           intel-tbb/2019.2_gcc-6.4.0                    papi/5.6.0_gcc-6.4.0
cudnn/6.0_gcc-6.4.0                           isl/0.19_gcc-6.4.0                            swig/3.0.12_gcc-6.4.0
  cudnn/7.3_gcc-6.4.0                            jdk/11.0.2_9_gcc-6.4.0                        tar/1.31_gcc-6.4.0
gcc/6.4.0_gcc-6.4.0                           libfabric/1.7.1_gcc-6.4.0
gcc/6.5.0_gcc-6.4.0                           likwid/4.3.2_gcc-6.4.0


To load something into your environment use the load command:
To load something into your environment use the ''load'' command:


  $ module load openmpi
  $ module load gcc
$ gcc --version
gcc (GCC) 8.3.0


If you don't want to use the latest version, you can also specify it:
By default, module loads the latest version available of a software (sorted by alphabetical order). You can also specify the version you need:


  $ module load openmpi/3.0.0_gcc-6.4.0
  $ module load gcc/7.4.0_gcc-6.4.0
  $ mpirun --version
  $ gcc --version
  mpirun (Open MPI) 3.0.0
  gcc (GCC) 7.4.0


You can also find out more information about the package using the show or whatis commands
You can also find out more information about the package using ''whatis'' or ''show'' commands:


  $ module show openmpi
  $ module whatis gcc
  $ module whatis openmpi
  $ module show gcc


If you want to remove one or all the currently loaded modules, you can use
If you want to unload one or all the currently loaded modules, you can use:


  $ module unload openmpi
  $ module unload gcc
  $ module purge
  $ module purge
The full documentation of module command is available at: http://modules.sourceforge.net/
==== Packages that require connection to license server ====
Some packages available with module are licensed and require a connection to a license server. If your institution provide a license server for this software, you have to forward the connection to the license server from the node where the software will be used.
For instance, to use Intel software from an Inria institute, you have to forward ports 29030 and 34430 between your node and jetons.inria.fr server, and configure Intel stack where to find the connection:
# Assuming that your connecting from a network where jetons.inria.fr is available
ssh -R 29030:jetons.inria.fr:29030 -R 34430:jetons.inria.fr:34430 <your_node>.g5k
module load intel-parallel-studio
export INTEL_LICENSE_FILE=29030@127.0.0.1
icc -v

Revision as of 15:34, 6 May 2019

Software and Modules

Grid'5000 provides a set of software (mainly scientific-related) using the module command line tool. They are available from Grid5000 frontends or cluster's nodes (only on standard, big, and nfs environment if deployment is used).

If you need additional software to be installed, feel free to contact Grid5000 support team and we can look into it.

Modules

The module system is designed to load software and make it available by modifying environment (such as your PATH variable).

To get started, list available software using:

$ module av
--------------------------------------------- /grid5000/spack/share/spack/modules/linux-debian9-x86_64 ---------------------------------------------
autoconf/2.69_gcc-6.4.0                        gcc/7.4.0_gcc-6.4.0                            llvm/7.0.1_gcc-6.4.0
automake/1.16.1_gcc-6.4.0                      gcc/8.3.0_gcc-6.4.0                            memkind/1.7.0_gcc-6.4.0
boost/1.69.0_gcc-6.4.0                         gmp/6.1.2_gcc-6.4.0                            miniconda2/4.5.11_gcc-6.4.0
cmake/3.13.4_gcc-6.4.0                         hwloc/1.11.11_gcc-6.4.0                        miniconda3/4.5.11_gcc-6.4.0
cuda/10.0.130_gcc-6.4.0                        hwloc/2.0.2_gcc-6.4.0                          mpfr/4.0.1_gcc-6.4.0
cuda/7.5.18_gcc-6.4.0                          intel-mkl/2017.4.239_gcc-6.4.0                 netlib-lapack/3.8.0_gcc-6.4.0
cuda/8.0.61_gcc-6.4.0                          intel-mkl/2018.1.163_gcc-6.4.0                 netlib-xblas/1.0.248_gcc-6.4.0
cuda/9.0.176_gcc-6.4.0                         intel-mkl/2019.1.144_gcc-6.4.0                 numactl/2.0.12_gcc-6.4.0
cuda/9.1.85_gcc-6.4.0                          intel-mpi/2019.1.144_gcc-6.4.0                 openblas/0.3.5_gcc-6.4.0
cuda/9.2.88_gcc-6.4.0                          intel-parallel-studio/cluster.2019.2_gcc-6.4.0 openmpi/3.1.3_gcc-6.4.0
cudnn/5.1_gcc-6.4.0                            intel-tbb/2019.2_gcc-6.4.0                     papi/5.6.0_gcc-6.4.0
cudnn/6.0_gcc-6.4.0                            isl/0.19_gcc-6.4.0                             swig/3.0.12_gcc-6.4.0
cudnn/7.3_gcc-6.4.0                            jdk/11.0.2_9_gcc-6.4.0                         tar/1.31_gcc-6.4.0
gcc/6.4.0_gcc-6.4.0                            libfabric/1.7.1_gcc-6.4.0
gcc/6.5.0_gcc-6.4.0                            likwid/4.3.2_gcc-6.4.0

To load something into your environment use the load command:

$ module load gcc
$ gcc --version
gcc (GCC) 8.3.0

By default, module loads the latest version available of a software (sorted by alphabetical order). You can also specify the version you need:

$ module load gcc/7.4.0_gcc-6.4.0
$ gcc --version
gcc (GCC) 7.4.0

You can also find out more information about the package using whatis or show commands:

$ module whatis gcc
$ module show gcc

If you want to unload one or all the currently loaded modules, you can use:

$ module unload gcc
$ module purge

The full documentation of module command is available at: http://modules.sourceforge.net/

Packages that require connection to license server

Some packages available with module are licensed and require a connection to a license server. If your institution provide a license server for this software, you have to forward the connection to the license server from the node where the software will be used.

For instance, to use Intel software from an Inria institute, you have to forward ports 29030 and 34430 between your node and jetons.inria.fr server, and configure Intel stack where to find the connection:

# Assuming that your connecting from a network where jetons.inria.fr is available 
ssh -R 29030:jetons.inria.fr:29030 -R 34430:jetons.inria.fr:34430 <your_node>.g5k
module load intel-parallel-studio
export INTEL_LICENSE_FILE=29030@127.0.0.1
icc -v