Guix: Difference between revisions

From Grid5000
Jump to navigation Jump to search
(17 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Portal|User}}
{{Portal|Tutorial}}
{{Pages|HPC}}
{{TutorialHeader}}
The Guix package manager is available on Grid'5000 to provide an alternative way of installing software on nodes.
The Guix package manager is available on Grid'5000 to provide an alternative way of installing software on nodes.


Line 7: Line 12:
* The [https://hpc.guix.info/ Guix HPC blog], with information about Guix usage in a HPC and reproducible context
* The [https://hpc.guix.info/ Guix HPC blog], with information about Guix usage in a HPC and reproducible context


= How to use Guix on Grid'5000 =
In order to use guix on Grid'5000 you must ensure that some environment variables are correctly set:
ddelabroye@node:~$ env | grep GUIX
GUIX_DAEMON_SOCKET=guix://guix.sophia.grid5000.fr
GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
GUIX_PROFILE=/home/ddelabroye/.guix-profile


If those variables are not set you will have to manually source <code>/etc/profile.d/guix.sh</code>:
'''Note''': For now, Guix is only available on Grid'5000 nodes with x86_64 architecture (not on POWER or ARM nodes).
node$ . /etc/profile.d/guix.sh
 


In order to initialize your <code>.guix-profile</code> you might want to do a <code>guix pull</code> first (be patient it will take a while the first time):
= Quick start =
node$ guix pull


Guix is available on the standard environment directly through the <code>guix</code> command.
Guix is available on frontends and nodes (using standard or deployed ''-nfs'' and ''-big'' environments) directly through the <code>guix</code> command.


For example:
For example:


  node$ guix install hello
  $ guix install hello


Each Grid'5000 site hosts a Guix server communicating with each other.
To make the ''hello'' executable available, programs installed by Guix must be added to your PATH. You can do this by simply running:
So if you install a package on Nancy and then decide to install the same package in Grenoble the guix server from Grenoble will fetch it from Nancy.
$ export GUIX_PROFILE=~/.guix-profile
$ . "$GUIX_PROFILE/etc/profile"


'''Note''': To work properly, a <code>.guix</code> directory is created in your home directory (it's a Grid'5000-specific directory). If you remove it, Guix will stop working since your profile is stored there.
(you can add these two commands to your ''.bashrc'' to make it persistent)


{{Warning|text=If you have a guix profile because you have used guix before you should move it somewhere else or it won't work with the guix we provide}}
You can then use what you just installed:
$ hello
Hello, world!


= Compilation =
To update available packages from Guix ''channels'',  use:
$ guix pull


When you try to install or compile a package, the following steps happen:
(be patient it will take a while the first time)
* Guix first tries to fetch the compiled package from ci.guix.gnu.org
* If not found, it will try to find it on all Grid'5000 Guix servers (on other sites)
* If it still not found, it will be built on a dedicated machine on Grid'5000, which is not accessible to the users


= Supported architecture =
= Advanced usage examples =
For now, Guix is only available on nodes with x86_64 architecture (not on POWER or ARM nodes).
== Generating image to use with singularity ==


= Usage example =
You can use Guix to create container image for ''singularity'' with [https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html guix pack]. For instance, to build an image containing emacs and bash, use:
== Generating image to use with singularity ==


You can use Guix to create container for singularity with [https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html guix pack]:
  $ guix pack -f squashfs bash emacs gcc
  $ guix pack -f squashfs bash emacs gcc
  ....
  ....
  /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs
  /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs
 
  $ singularity run /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs
The built image can be run using singularity. For instance, from a node:
 
  node$ singularity run /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
Line 57: Line 56:
  No runscript found in container, executing /bin/sh
  No runscript found in container, executing /bin/sh
  sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
  sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
 
  $ gcc -v
  Singularity> gcc -v
  Using built-in specs.
  Using built-in specs.
  COLLECT_GCC=gcc
  COLLECT_GCC=gcc
Line 69: Line 68:
  gcc version 10.2.0 (GCC)
  gcc version 10.2.0 (GCC)


'''Note''': If you forget the bash (or similar) package, singularity run and singularity exec will fail with an unhelpful “no such file or directory” message.


'''Note''': If you forget the bash (or similar) package, singularity run and singularity exec will fail with an unhelpful “no such file or directory” message.
For more information of singularity usage under Grid'5000, see [[Singularity]] page.


== Container ==
== Container ==
You can easily isolate the execution of your program in a container:
 
  $ guix environment --ad-hoc --container coreutils gcc
Guix [https://guix.gnu.org/manual/en/html_node/Invoking-guix-container.html#Invoking-guix-container container] command can be used to easily isolate the execution of your program. For instance, this will instanciate an isolated container containing ''coreutils'', ''gcc'' and ''procps''
  ddelabroye@chetemi-8 ~ [env]$ gcc -v
 
  node$ guix environment --ad-hoc --container coreutils gcc procps
 
  node ~ [env]$ gcc -v
  Using built-in specs.
  Using built-in specs.
  COLLECT_GCC=gcc
  COLLECT_GCC=gcc
Line 85: Line 88:
  Supported LTO compression algorithms: zlib
  Supported LTO compression algorithms: zlib
  gcc version 10.2.0 (GCC)
  gcc version 10.2.0 (GCC)
'''Note''': This feature is only available from nodes, not frontends
= Configuration details =
Here are some details of Guix deployment and configuration under Grid'5000 that may help in case of problem or for your customized deployment.
== Guix profile ==
'''Note''': To work properly, a <code>.guix</code> directory is created in your home directory (it's a Grid'5000-specific directory). If you remove it, Guix will stop working since your profile is stored there.
{{Warning|text=If you have a guix profile because you have used guix before you should move it somewhere else or it won't work with the guix we provide}}
== Environment variables ==
In order to use Guix on Grid'5000, some environment variables are correctly set:
$ env | grep GUIX
GUIX_DAEMON_SOCKET=guix://guix.sophia.grid5000.fr
GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
GUIX_PROFILE=/home/ddelabroye/.guix-profile
If those variables are not set you will have to manually source <code>/etc/profile.d/guix.sh</code>:
node$ . /etc/profile.d/guix.sh
== Package installation steps ==
When you try to install or compile a package, the following steps happen:
* Guix first tries to fetch the compiled package from ci.guix.gnu.org
* If not found, it will try to find it on all Grid'5000 Guix servers. (Each Grid'5000 site hosts a Guix server communicating with each other. If you install a package on Nancy and then decide to install the same package in Grenoble, it will be fetched from Nancy).
* If it still not found, it will be compiled on a dedicated machine on Grid'5000, which is not accessible to the users

Revision as of 19:38, 23 March 2021

Note.png Note

This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

The Guix package manager is available on Grid'5000 to provide an alternative way of installing software on nodes.

This page describes the use of Guix on Grid'5000. For additional information about Guix, see:


Note: For now, Guix is only available on Grid'5000 nodes with x86_64 architecture (not on POWER or ARM nodes).


Quick start

Guix is available on frontends and nodes (using standard or deployed -nfs and -big environments) directly through the guix command.

For example:

$ guix install hello

To make the hello executable available, programs installed by Guix must be added to your PATH. You can do this by simply running:

$ export GUIX_PROFILE=~/.guix-profile
$ . "$GUIX_PROFILE/etc/profile"

(you can add these two commands to your .bashrc to make it persistent)

You can then use what you just installed:

$ hello
Hello, world!

To update available packages from Guix channels, use:

$ guix pull

(be patient it will take a while the first time)

Advanced usage examples

Generating image to use with singularity

You can use Guix to create container image for singularity with guix pack. For instance, to build an image containing emacs and bash, use:

$ guix pack -f squashfs bash emacs gcc
....
/gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs

The built image can be run using singularity. For instance, from a node:

node$ singularity run /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs
WARNING: passwd file doesn't exist in container, not updating
WARNING: group file doesn't exist in container, not updating
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
No runscript found in container, executing /bin/sh
sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Singularity> gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/gnu/store/xpdy9vvqym9xv7praqkwsm3gdzn2kv1p-gcc- 
10.2.0/libexec/gcc/x86_64-unknown-linux-gnu/10.2.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with:
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.0 (GCC)

Note: If you forget the bash (or similar) package, singularity run and singularity exec will fail with an unhelpful “no such file or directory” message.

For more information of singularity usage under Grid'5000, see Singularity page.

Container

Guix container command can be used to easily isolate the execution of your program. For instance, this will instanciate an isolated container containing coreutils, gcc and procps

node$ guix environment --ad-hoc --container coreutils gcc procps
node ~ [env]$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/gnu/store/xpdy9vvqym9xv7praqkwsm3gdzn2kv1p-gcc- 
10.2.0/libexec/gcc/x86_64-unknown-linux-gnu/10.2.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with:
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.0 (GCC)

Note: This feature is only available from nodes, not frontends

Configuration details

Here are some details of Guix deployment and configuration under Grid'5000 that may help in case of problem or for your customized deployment.

Guix profile

Note: To work properly, a .guix directory is created in your home directory (it's a Grid'5000-specific directory). If you remove it, Guix will stop working since your profile is stored there.

Warning.png Warning

If you have a guix profile because you have used guix before you should move it somewhere else or it won't work with the guix we provide

Environment variables

In order to use Guix on Grid'5000, some environment variables are correctly set:

$ env | grep GUIX
GUIX_DAEMON_SOCKET=guix://guix.sophia.grid5000.fr
GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
GUIX_PROFILE=/home/ddelabroye/.guix-profile

If those variables are not set you will have to manually source /etc/profile.d/guix.sh:

node$ . /etc/profile.d/guix.sh

Package installation steps

When you try to install or compile a package, the following steps happen:

  • Guix first tries to fetch the compiled package from ci.guix.gnu.org
  • If not found, it will try to find it on all Grid'5000 Guix servers. (Each Grid'5000 site hosts a Guix server communicating with each other. If you install a package on Nancy and then decide to install the same package in Grenoble, it will be fetched from Nancy).
  • If it still not found, it will be compiled on a dedicated machine on Grid'5000, which is not accessible to the users