Debian 13 Trixie environments release notes: Difference between revisions

From Grid5000
Jump to navigation Jump to search
(Created page with "{{Portal|User}} __TOC__ There is some significant evolutions since previous debian11 environments. <span id="common-componants-changes-debian"></span> = Common componants changes (Debian) = {| class="wikitable" |- ! Componant ! style="text-align: center;"| Debian 11 Bullseye ! style="text-align: center;"| Debian 13 Trixie |- | Linux kernel | style="text-align: center;"| 5.10 | style="text-align: center;"| 6.12 |- | GCC | style="text-align: center;"| 10.2 | style="tex...")
 
 
(29 intermediate revisions by 2 users not shown)
Line 5: Line 5:
There is some significant evolutions since previous debian11 environments.
There is some significant evolutions since previous debian11 environments.


<span id="common-componants-changes-debian"></span>
= Common componants changes (Debian) =
= Common componants changes (Debian) =


Line 46: Line 45:
| style="text-align: center;"| 10.0
| style="text-align: center;"| 10.0
|-
|-
| Open MPI
| RDMA
| style="text-align: center;"| 4.1
| style="text-align: center;"| 33.2
| style="text-align: center;"| 5.0
| style="text-align: center;"| 56.1
|-
| Singularity
| style="text-align: center;"| 4.1.2
| style="text-align: center;"| N/A
|-
| Apptainer (Singularity replacement)
| style="text-align: center;"| N/A
| style="text-align: center;"| 1.4.5
|}
|}


For more informations, see [https://www.debian.org/releases/trixie/release-notes/whats-new.en.html#desktops-and-well-known-packages debian.org Release Notes]
For more informations, see [https://www.debian.org/releases/trixie/release-notes/whats-new.en.html#desktops-and-well-known-packages debian.org Release Notes].


<span id="environments-specific-changes"></span>
= Environments specific changes =
= Environments specific changes =


Available in debian13-big and debian13-std (default) environments
Available in debian13-big and debian13-std (default) environments :


{| class="wikitable"
{| class="wikitable"
Line 70: Line 76:
| Nvidia CUDA
| Nvidia CUDA
| style="text-align: center;"| 11.2.x
| style="text-align: center;"| 11.2.x
| style="text-align: center;"| via modules
| style="text-align: center;"| 13.0 via modules
|-
|-
| AMD drivers (amdgpu)
| AMD drivers (amdgpu)
Line 78: Line 84:
| AMD ROCm
| AMD ROCm
| style="text-align: center;"| 6.3.x
| style="text-align: center;"| 6.3.x
| style="text-align: center;"| via modules
| style="text-align: center;"| 7.1 via modules
|-
|-
| Singularity
| Open MPI
| style="text-align: center;"| 4.1.2
| style="text-align: center;"| 4.1
| style="text-align: center;"| N/A
| style="text-align: center;"| 4.1 via modules
|-
| Apptainer (Singularity replacement)
| style="text-align: center;"| N/A
| style="text-align: center;"| 1.4.5
|}
|}


Line 101: Line 103:
| style="text-align: left;"| Nvidia CUDA
| style="text-align: left;"| Nvidia CUDA
| style="text-align: center;"| N/A
| style="text-align: center;"| N/A
| style="text-align: center;"| N/A
| style="text-align: center;"| via modules (planned)
|}
|}


⚠ Note some “old” Nvidia GPU clusters use a new environment named debian'''nvlegacy'''13-big and debian'''nvlegacy'''13-std (default). The only change in this environment flavor is that the proprietary nvidia module kernel is used, instead of the more recent opensource one, as it is mandatory for Nvidia GPU Compute Capability &lt; 7.5 . For more informations, see [https://developer.nvidia.com/blog/nvidia-transitions-fully-towards-open-source-gpu-kernel-modules/ this Nvidia explanation].
Note some “old” Nvidia GPU clusters use a new dedicated environment named debian'''nvlegacy'''13 for big and std (default) variants. The only change in this environment flavor is that the proprietary nvidia module kernel is used, instead of the more recent open source one, as it is mandatory for Nvidia GPU Compute Capability < 7.5 .


Some other useful informations: - [https://gitlab.inria.fr/grid5000/environments-recipes/-/blob/master/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp Kameleon environment recipe - software versions] - [https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/user-kernel-space-compat-matrix.html User and AMD GPU Driver (amdgpu) support matrix]
For more informations, see [https://developer.nvidia.com/blog/nvidia-transitions-fully-towards-open-source-gpu-kernel-modules/ this Nvidia explanation].
 
Some other useful informations:
 
* [https://gitlab.inria.fr/grid5000/environments-recipes/-/blob/master/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp Kameleon environment recipe - software versions]
* [https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/user-kernel-space-compat-matrix.html User and AMD GPU Driver (amdgpu) support matrix]


<span id="modules-evolution"></span>
== Modules evolution ==
== Modules evolution ==


The default provided modules are no more build with [https://spack.readthedocs.io Spack], but with [https://codeberg.org/guix-science/guix-modules GUIX]
The default provided modules are no more build with [https://spack.readthedocs.io Spack], but with [https://codeberg.org/guix-science/guix-modules GUIX].


For more informations, see - [https://www.grid5000.fr/w/News#Upcoming_changes_to_modules ''Upcoming changes to modules'' news] - [https://www.grid5000.fr/w/Modules ''Modules'' wiki page].
For more informations, see


'''TODO''' Note ARM64 CPU architecture modules are not available for now. We are [https://gitlab.inria.fr/moyens-de-calcul/environnement-logiciel/-/work_items/9 working on it].
* The [https://grid5000.gitlabpages.inria.fr/resources-explorer/software/ list of modules available online].
* The [https://www.grid5000.fr/w/News#Upcoming_changes_to_modules ''Upcoming changes to modules'' news].
* The [https://www.grid5000.fr/w/Modules ''Modules'' wiki page].


<span id="nvidia-cuda-and-amd-rocm"></span>
{{Note|type=info|text=ARM64 CPU architecture modules are not available for now. We are [https://gitlab.inria.fr/moyens-de-calcul/environnement-logiciel/-/work_items/9 working on it].}}
=== Nvidia CUDA and AMD ROCm ===


Only Nvidia and amdgpu drivers (kernel modules) are provided in big and default environments. Tools like CUDA and ROCm have to be loaded with module commands.
{{Warning|text=Libraries available on the system and in the modules are not always compatible. To avoid any issue always use *only* the software in the modules, or *only* those from the system, don't mix and match.}}


<span id="cuda"></span>
=== CUDA and ROCm ===
=== CUDA ===


An example if environment debian13 : <code>module load cuda-toolkit nvidia-driver-libs</code> (default is currently ''cuda-toolkit/13'')
Only nvidia and amdgpu drivers (kernel modules) are provided in big and default environments. Tools like CUDA and ROCm have to be loaded with module commands.


An example if environment debiannvlegacy13 : <code>module load cuda-toolkit/12 nvidia-driver-libs</code>
==== Nvidia CUDA modules ====
 
If you want to use cuda from the modules and run the compiled code, you need to load two modules: ''cuda-toolkit'', which provide the cuda-related software, and ''nvidia-driver-libs'', which allows the packaged cuda to see the system's drivers.
You likely also want to use a toolchain to compile non-cuda code, and therefore will need to load ''gcc-toolchain''.
 
 
If the default environment is debian13 the module load looks like this : <code>module load gcc-toolchain cuda-toolkit nvidia-driver-libs</code> (default is currently ''cuda-toolkit/13'').
 
And if the default environment is debiannvlegacy13 : <code>module load gcc-toolchain cuda-toolkit/12 nvidia-driver-libs</code>


Compatibility matrix:
Compatibility matrix:
Line 141: Line 155:
| style="text-align: left;"| debiannvlegacy13
| style="text-align: left;"| debiannvlegacy13
| style="text-align: center;"| proprietary (legacy)
| style="text-align: center;"| proprietary (legacy)
| style="text-align: center;"| &lt; 7.5
| style="text-align: center;"| < 7.5
| style="text-align: center;"| OK
| style="text-align: center;"| OK
| style="text-align: center;"| KO
| style="text-align: center;"| KO
Line 154: Line 168:
For more informations on CUDA GPU Compute Capability, see [https://developer.nvidia.com/cuda/gpus/legacy Nvidia documentation].
For more informations on CUDA GPU Compute Capability, see [https://developer.nvidia.com/cuda/gpus/legacy Nvidia documentation].


<span id="rocm"></span>
==== AMD ROCm (Radeon Open Compute) modules ====
=== ROCm ===


'''TODO'''
There are a lot of ROCm modules available, you can load them individually, or you can use the meta-module ''rocm'' to make it available:


ROCm module has some others modules dependancies. <code>module load rocm-hip-runtime hipblas hipblas-common rocblas rocm-toolchain gcc-toolchain</code> To make it easier to use, a ''rocm-abaca'' meta module to load has been defined : <code>module load rocm-abaca</code>
<code>module load cmake gcc-toolchain rocm</code>


<span id="tools-changes"></span>
=== Open MPI ===
= Tools changes =


<span id="dhcp-client-and-ipv6"></span>
OpenMPI is no more installed in the environment and only available through the modules. You have to load it with <code>module load openmpi</code>.
== DHCP client and IPv6 ==


ISC DHCP tools (client and server) are deprecated. See [https://www.isc.org/dhcp/ ISC Note]. The client has been replaced by '''dhcpcd'''. So the commonly used <code>dhclient &lt;interface_name&gt;</code> is no more available, and is be replaced by <code>dhcpcd &lt;interface_name&gt;</code>.
Loading ''openmpi'' doesn't load a toolchain, you '''must''' load a toolchain and make sure to use it, otherwise you will end up using the system's toolchain and you will run into issues; you can use <code>module load gcc-toolchain</code> to make a compatible gcc available.
 
If your code depends on other librairies, please make sure they come from the modules and not the system.
 
== Tools changes ==
 
=== DHCP client and IPv6 ===
 
ISC DHCP tools (client and server) are deprecated. See [https://www.isc.org/dhcp/ ISC Note].
 
The client has been replaced by '''dhcpcd'''. So the commonly used <code>dhclient &lt;interface_name&gt;</code> is no more available, and is replaced by <code>dhcpcd &lt;interface_name&gt;</code>.


Node network interface will have '''both IPv4 and IPv6 addresses''' by default.
Node network interface will have '''both IPv4 and IPv6 addresses''' by default.


<span id="network-time-protocol-ntp-client"></span>
=== Network Time Protocol (NTP) client ===
== Network Time Protocol (NTP) client ==


For system time synchronizaton, '''ntpsec and ntpsec-ntpdate''' are used instead of no more available ntp et ntpdate tools packages.
For system time synchronizaton, '''ntpsec and ntpsec-ntpdate''' are used instead of no more available ntp et ntpdate tools packages.


<span id="cpu-frequency-scaling"></span>
=== CPU frequency scaling ===
== CPU frequency scaling ==
 
cpufrequtils has been replaced by '''linux-cpupower''' (introduced with kernel 3.1).


cpufrequtils has been replaced by '''linux-cpupower''' (introduced with kernel 3.1). See the [https://manpages.debian.org/trixie/linux-cpupower/cpupower.1.en.html cpupower manpage] for more informations.
See the [https://manpages.debian.org/trixie/linux-cpupower/cpupower.1.en.html cpupower manpage] for more informations.


<span id="open-mpi"></span>
=== Singularity to Apptainer ===
= Open MPI =


<blockquote>Debian11|12 disable many providers by default. We restore UCX and Fabric while keeping openib disabled to avoid useless warnings
[https://apptainer.org/ Apptainer] is available in replacement of Singularity.
</blockquote>
'''TODO''' [https://intranet.grid5000.fr/bug/17590 Bug 17590]: debian 13 - Choix de configuration Open MPI


<span id="pmem"></span>
=== PMem (Persistent Memory) ===
= PMem =


Debian ipmctl package, a utility for configuring and managing Intel® Optane™ Persistent Memory modules is not compatible with python 3.13 provided by Debian 13 repository.
Debian ipmctl package, a utility for configuring and managing Intel® Optane™ Persistent Memory modules is not compatible with python 3.13 provided by Debian 13 repository.


So the [https://www.grid5000.fr/w/Grenoble:Hardware#troll Troll cluster in Grenoble] featuring PMEM remains on debian11 environnement by default.
So the [https://www.grid5000.fr/w/Grenoble:Hardware#troll Troll cluster in Grenoble] featuring PMem remains on debian11 environnement by default.


<span id="singularity-to-apptainer"></span>
== Custom Linux kernel ==
= Singularity to Apptainer =


[https://apptainer.org/ Apptainer] is available in replacement of Singularity.
For big and std environments variants, we had to use a custom Linux kernel build to enable and continue using control groups v1 (cgroupv1) instead of newer cgroupv2 to keep compatibility with currently used OAR 2.5.10.

Latest revision as of 21:39, 23 August 2026


There is some significant evolutions since previous debian11 environments.

Common componants changes (Debian)

Componant Debian 11 Bullseye Debian 13 Trixie
Linux kernel 5.10 6.12
GCC 10.2 14.2
LLVM/Clang toolchain 11 19
Perl 5.32 5.40
Python 3.9 3.13
Ruby 2.7 3.3
OpenJDK 17 25
Qemu 5.2 10.0
RDMA 33.2 56.1
Singularity 4.1.2 N/A
Apptainer (Singularity replacement) N/A 1.4.5

For more informations, see debian.org Release Notes.

Environments specific changes

Available in debian13-big and debian13-std (default) environments :

x86_64 (AMD64) componants debian11 envs debian13 envs
Nvidia drivers 535.x 580.x
Nvidia CUDA 11.2.x 13.0 via modules
AMD drivers (amdgpu) 6.3.x 30.x
AMD ROCm 6.3.x 7.1 via modules
Open MPI 4.1 4.1 via modules
aarch64 (ARM64) componants debian11 env debian13 env
Nvidia drivers N/A 580.x
Nvidia CUDA N/A via modules (planned)

Note some “old” Nvidia GPU clusters use a new dedicated environment named debiannvlegacy13 for big and std (default) variants. The only change in this environment flavor is that the proprietary nvidia module kernel is used, instead of the more recent open source one, as it is mandatory for Nvidia GPU Compute Capability < 7.5 .

For more informations, see this Nvidia explanation.

Some other useful informations:

Modules evolution

The default provided modules are no more build with Spack, but with GUIX.

For more informations, see

Note.png Note

ARM64 CPU architecture modules are not available for now. We are working on it.

Warning.png Warning

Libraries available on the system and in the modules are not always compatible. To avoid any issue always use *only* the software in the modules, or *only* those from the system, don't mix and match.

CUDA and ROCm

Only nvidia and amdgpu drivers (kernel modules) are provided in big and default environments. Tools like CUDA and ROCm have to be loaded with module commands.

Nvidia CUDA modules

If you want to use cuda from the modules and run the compiled code, you need to load two modules: cuda-toolkit, which provide the cuda-related software, and nvidia-driver-libs, which allows the packaged cuda to see the system's drivers. You likely also want to use a toolchain to compile non-cuda code, and therefore will need to load gcc-toolchain.


If the default environment is debian13 the module load looks like this : module load gcc-toolchain cuda-toolkit nvidia-driver-libs (default is currently cuda-toolkit/13).

And if the default environment is debiannvlegacy13 : module load gcc-toolchain cuda-toolkit/12 nvidia-driver-libs

Compatibility matrix:

environment Nvidia kernel module Nvidia GPU compute capability CUDA 12.9 CUDA 13
debiannvlegacy13 proprietary (legacy) < 7.5 OK KO
debian13 open ≥ 7.5 OK OK

For more informations on CUDA GPU Compute Capability, see Nvidia documentation.

AMD ROCm (Radeon Open Compute) modules

There are a lot of ROCm modules available, you can load them individually, or you can use the meta-module rocm to make it available:

module load cmake gcc-toolchain rocm

Open MPI

OpenMPI is no more installed in the environment and only available through the modules. You have to load it with module load openmpi.

Loading openmpi doesn't load a toolchain, you must load a toolchain and make sure to use it, otherwise you will end up using the system's toolchain and you will run into issues; you can use module load gcc-toolchain to make a compatible gcc available.

If your code depends on other librairies, please make sure they come from the modules and not the system.

Tools changes

DHCP client and IPv6

ISC DHCP tools (client and server) are deprecated. See ISC Note.

The client has been replaced by dhcpcd. So the commonly used dhclient <interface_name> is no more available, and is replaced by dhcpcd <interface_name>.

Node network interface will have both IPv4 and IPv6 addresses by default.

Network Time Protocol (NTP) client

For system time synchronizaton, ntpsec and ntpsec-ntpdate are used instead of no more available ntp et ntpdate tools packages.

CPU frequency scaling

cpufrequtils has been replaced by linux-cpupower (introduced with kernel 3.1).

See the cpupower manpage for more informations.

Singularity to Apptainer

Apptainer is available in replacement of Singularity.

PMem (Persistent Memory)

Debian ipmctl package, a utility for configuring and managing Intel® Optane™ Persistent Memory modules is not compatible with python 3.13 provided by Debian 13 repository.

So the Troll cluster in Grenoble featuring PMem remains on debian11 environnement by default.

Custom Linux kernel

For big and std environments variants, we had to use a custom Linux kernel build to enable and continue using control groups v1 (cgroupv1) instead of newer cgroupv2 to keep compatibility with currently used OAR 2.5.10.

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Debian_13_Trixie_environments_release_notes&oldid=109121"