Difference between revisions of "Docker"

From Grid5000
Jump to: navigation, search
(Managing Docker images)
(Nvidia-docker)
 
(39 intermediate revisions by 11 users not shown)
Line 1: Line 1:
<!--
 
{{Maintainer|Florent Didier}}
 
 
{{Portal|User}}
 
{{Portal|User}}
{{Status|In production}}
+
{{Portal|Tutorial}}
-->
+
{{TutorialHeader}}
 
 
 
'''Docker''' is a software technology that provides operating-system-level virtualization. In this page, we present some tools that are specifically designed to use Docker on Grid'5000.
 
'''Docker''' is a software technology that provides operating-system-level virtualization. In this page, we present some tools that are specifically designed to use Docker on Grid'5000.
  
= Installing Docker from a node =
+
= Installing Docker from a node in the standard environment =
 
You can install and use Docker from a node with the standard environment by following these instructions:
 
You can install and use Docker from a node with the standard environment by following these instructions:
  
 
; Reserve a node
 
; Reserve a node
 
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code>-I</code>}}
 
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code>-I</code>}}
(you can also use option <code>-t deploy</code> and Kadeploy if you prefer)
 
  
; Install Docker
+
See [[Getting_Started]] for more reservation options.
The script g5k-setup-docker, available from the standard environment, installs Docker locally. Option -c makes docker command available without needing to log out and reconnect to the node
+
 
{{Term|location=node|cmd=<code class="command">g5k-setup-docker</code> <code>-c</code>}}
+
; Install Docker from the standard environment
 +
The script g5k-setup-docker, available from the standard environment, installs Docker locally and makes Docker commands available without user needing to log out (by giving read write permissions to the file /var/run/docker.sock)
 +
{{Term|location=node|cmd=<code class="command">g5k-setup-docker</code> <code>-t</code>}}
 +
 
 +
The <code>-t</code> option allows to use <code>/tmp/</code> to store docker images, which is useful because <code>/tmp/</code> has more disk space available. Technically speaking, it does a bind mount of <code>/tmp/docker</code> on <code>/var/lib/docker</code>.
  
 
; Run Docker
 
; Run Docker
 
You can now check that Docker is up and running
 
You can now check that Docker is up and running
 
{{Term|location=node|cmd=<code class="command">docker run</code> <code>hello-world</code>}}
 
{{Term|location=node|cmd=<code class="command">docker run</code> <code>hello-world</code>}}
 +
 +
= Installing Docker from a deployed node =
 +
 +
In addition to the standard environment, g5k-setup-docker works with the environments '''debian11-x64-nfs''' and '''debian11-x64-big''' (as well as with '''debian10''' variants and the other architectures '''ppc64''' and '''arm64''').
 +
 +
Reserve a node, [[Getting_Started#Deploying_your_nodes_to_get_root_access_and_create_your_own_experimental_environment|deploy an environment]], connect to the node and run the script as root with its full path:
 +
 +
{{Term|location=root@node| cmd=<code class="command">/grid5000/code/bin/g5k-setup-docker</code>}}
 +
 +
You can also use the <code>-t</code> option to store docker images in <code>/tmp/</code> to have more space, but be aware that everything will be deleted if you reboot the node.
 +
 +
= Nvidia-docker =
 +
On nodes with one or more GPUs, you may want to use Nvidia's Docker container images (e.g. see [https://www.nvidia.com/en-us/gpu-cloud/containers/]). To do so, you have to install Nvidia's [https://github.com/nvidia/nvidia-docker nvidia-docker].
 +
 +
To ease this task, Grid'5000 provides a script to automate the installation. Just run :
 +
 +
{{Term|location=node| cmd=<code class="command">g5k-setup-nvidia-docker</code> <code>-t</code>}}
 +
 +
Options are the same as with the <code>g5k-setup-docker</code> script presented above. Please note that you can use <code>g5k-setup-docker</code> or <code>g5k-setup-nvidia-docker</code>, but not both all together.
 +
 +
{{Warning|text=An official Nvidia's Docker image for Debian 11 is not available for now, so the Debian 10 container image is still used. But cgroup v2 needs also to be disabled on the host where nvidia-docker is run.
 +
While cgroups v2 are already disabled in the Grid'5000 '''debian11 std environment''', it is not the case of other Grid'5000 debian 10 environments.
 +
You may however to adapt those environments to change the kernel boot parameters in order to set <code>kernel_params: "g5k_kernel_params: "systemd.unified_cgroup_hierarchy=false systemd.legacy_systemd_cgroup_controller=true"</code> which disables cgroup v2 (see [[Advanced_Kadeploy#Create_a_new_environment_from_a_customized_environment]] for more information on the modification of Grid'5000 environments).}}
  
 
= Using docker-machine and docker-g5k =
 
= Using docker-machine and docker-g5k =
The Spirals Team in Lille developped two tools to ease Docker usage on Grid'5000:
+
The Spirals Team in Lille developed two tools to ease Docker usage on Grid'5000:
 
* a docker-machine driver
 
* a docker-machine driver
 
* the docker-g5k program
 
* the docker-g5k program
Line 36: Line 59:
  
 
= Managing Docker images =
 
= Managing Docker images =
Users sometimes want to load Docker images quickly or store private Docker images in a registry. Achieving these goals is not easy, because Grid'5000 network connection to the outside has poor performance (<1Gbps). However, we propose the following solutions to manage your images.
+
One difficult aspect of using Docker is efficient images management, as (1) it is often required to load such images on many nodes simultaneously; (2) Grid'5000 connection to the internet is limited to 1 Gbps.
 +
The following strategies are recommended to manage Docker images:
  
 
== Use docker load/save ==
 
== Use docker load/save ==
To load and save images quickly, we encourage you to use <code>docker load</code> and <code>docker save</code> commands, with .tar docker images stored in your Grid'5000 home. User's home are NFS mounted in all nodes deployed with the standard environment, with good network performance.
+
To load and save images quickly, we encourage you to use <code>docker load</code> and <code>docker save</code> commands, with .tar docker images stored in your Grid'5000 home. Users home are NFS mounted, with good network performance.
  
Example:
+
; Example:
 
{{Term|location=node1|
 
{{Term|location=node1|
cmd=<code class="command">docker pull</code> <code>centos:7</code><br>
+
cmd=<code class="command">docker pull</code> <code>alpine</code><br>
<code class="command">docker save</code> <code>-o ~/centos7.tar centos:7</code><br>
+
<code class="command">docker save</code> <code>-o ~/alpine.tar alpine</code><br>
 
}}
 
}}
 
{{Term|location=node2|
 
{{Term|location=node2|
cmd=<code class="command">docker load</code> <code>-i ~/centos7.tar</code><br>
+
cmd=<code class="command">docker load</code> <code>-i ~/alpine.tar</code><br>
<code class="command">docker run</code> <code>-it centos:7</code>
+
<code class="command">docker run</code> <code>-it alpine</code>
 
}}
 
}}
  
You can also use <code>docker load</code> command instead of retriving the image from Docker Hub, when deploying a Swarm with docker-g5k.
+
You can also use <code>docker load</code> command instead of retrieving the image from Docker Hub, when deploying a Swarm with docker-g5k.
  
 +
; Example with docker-g5k
 
Give read rights to your image
 
Give read rights to your image
{{Term|location=frontend|cmd=<code class="command">chmod 644</code> <code>~/centos7.tar</code>}}
+
{{Term|location=frontend|cmd=<code class="command">chmod 644</code> <code>~/alpine.tar</code>}}
  
 
Deploy three Docker nodes in Nancy (activate the Grid'5000 VPN before launching this command)
 
Deploy three Docker nodes in Nancy (activate the Grid'5000 VPN before launching this command)
 
{{Term|location=laptop|cmd=<code class="command">docker-g5k create-cluster</code> <code>--g5k-username "user" --g5k-password "********" --g5k-image "jessie-x64-std" --g5k-reserve-nodes "nancy:3"</code>}}
 
{{Term|location=laptop|cmd=<code class="command">docker-g5k create-cluster</code> <code>--g5k-username "user" --g5k-password "********" --g5k-image "jessie-x64-std" --g5k-reserve-nodes "nancy:3"</code>}}
  
Load Centos image on each node
+
Load Alpine image on each node
{{Term|location=laptop|cmd=<code>for i in {0..2}; do</code> <code class="command">docker-machine ssh</code> <code>nancy-${i} "docker load -i /home/user/centos7.tar"; done</code>}}
+
{{Term|location=laptop|cmd=<code>for i in {0..2}; do</code> <code class="command">docker-machine ssh</code> <code>nancy-${i} "docker load -i /home/user/alpine.tar"; done</code>}}
 +
 
 +
== Use Inria private registry ==
  
 +
The Inria registry has been closed by the end of 2019/beg. 2020.
  
== Use Inria private registry ==
+
As an alternative one can use gitlab.inria.fr to host docker images related to a git project, but this cannot be compared to the functionalities provided by a normal Docker registry.
Inria members can store private Docker images at [https://qlf-sesi-harbor.inria.fr/harbor/sign-in Inria private registry] (also look at the [https://partage.inria.fr/share/page/document-details?nodeRef=workspace://SpacesStore/b33e70e5-ca6e-4121-9051-782ca88603ba documentation]), but network throughput is low.
 
  
== Use a persistent VM ==
+
== Use a persistent virtual machine to host your registry ==
 
If you want to benefit from a private registry with good network performance, and are ready to manually install your own registry, you can request a [[Persistent_Virtual_Machine|Persistent Virtual Machine]]. You can look at [http://vmware.github.io/harbor/ Harbor] or [http://port.us.org/ Portus] open source projects, in order to install your own registry.
 
If you want to benefit from a private registry with good network performance, and are ready to manually install your own registry, you can request a [[Persistent_Virtual_Machine|Persistent Virtual Machine]]. You can look at [http://vmware.github.io/harbor/ Harbor] or [http://port.us.org/ Portus] open source projects, in order to install your own registry.
 +
 +
= Avoid network conflict =
 +
The default network used by docker is 172.16.0.1/16 which is use internally on Grid'5000 thus this can be a source of conflict.
 +
 +
If you want to avoid any conflict you can set an other network. On debian you will have to edit /etc/default/docker and specify the network you want to use like this:
 +
DOCKER_OPTS="--bip=192.168.42.1/24"
 +
 +
= Using docker-cache.grid5000.fr =
 +
 +
Due do the rate limiting of docker hub you might experience some troubles to fetch images. As a result you can use docker-cache.grid5000.fr which is a registry mirror.
 +
 +
To enable the use of the cache from a Grid'5000 node (as ''root'' or using ''sudo-g5k''), edit <code>/etc/docker/daemon.json</code> with:
 +
{
 +
  "registry-mirrors": [
 +
    "http://docker-cache.grid5000.fr"
 +
  ]
 +
}
 +
 +
then restart docker:
 +
 +
systemctl restart docker
 +
 +
The cache is automatically configured by <code>g5k-setup-docker</code>.

Latest revision as of 13:59, 9 November 2021

Note.png Note

This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

Docker is a software technology that provides operating-system-level virtualization. In this page, we present some tools that are specifically designed to use Docker on Grid'5000.

Installing Docker from a node in the standard environment

You can install and use Docker from a node with the standard environment by following these instructions:

Reserve a node
Terminal.png fnancy:
oarsub -I

See Getting_Started for more reservation options.

Install Docker from the standard environment

The script g5k-setup-docker, available from the standard environment, installs Docker locally and makes Docker commands available without user needing to log out (by giving read write permissions to the file /var/run/docker.sock)

Terminal.png node:
g5k-setup-docker -t

The -t option allows to use /tmp/ to store docker images, which is useful because /tmp/ has more disk space available. Technically speaking, it does a bind mount of /tmp/docker on /var/lib/docker.

Run Docker

You can now check that Docker is up and running

Terminal.png node:
docker run hello-world

Installing Docker from a deployed node

In addition to the standard environment, g5k-setup-docker works with the environments debian11-x64-nfs and debian11-x64-big (as well as with debian10 variants and the other architectures ppc64 and arm64).

Reserve a node, deploy an environment, connect to the node and run the script as root with its full path:

Terminal.png root@node:
/grid5000/code/bin/g5k-setup-docker

You can also use the -t option to store docker images in /tmp/ to have more space, but be aware that everything will be deleted if you reboot the node.

Nvidia-docker

On nodes with one or more GPUs, you may want to use Nvidia's Docker container images (e.g. see [1]). To do so, you have to install Nvidia's nvidia-docker.

To ease this task, Grid'5000 provides a script to automate the installation. Just run :

Terminal.png node:
g5k-setup-nvidia-docker -t

Options are the same as with the g5k-setup-docker script presented above. Please note that you can use g5k-setup-docker or g5k-setup-nvidia-docker, but not both all together.

Warning.png Warning

An official Nvidia's Docker image for Debian 11 is not available for now, so the Debian 10 container image is still used. But cgroup v2 needs also to be disabled on the host where nvidia-docker is run.

While cgroups v2 are already disabled in the Grid'5000 debian11 std environment, it is not the case of other Grid'5000 debian 10 environments.

You may however to adapt those environments to change the kernel boot parameters in order to set kernel_params: "g5k_kernel_params: "systemd.unified_cgroup_hierarchy=false systemd.legacy_systemd_cgroup_controller=true" which disables cgroup v2 (see Advanced_Kadeploy#Create_a_new_environment_from_a_customized_environment for more information on the modification of Grid'5000 environments).

Using docker-machine and docker-g5k

The Spirals Team in Lille developed two tools to ease Docker usage on Grid'5000:

  • a docker-machine driver
  • the docker-g5k program

Provision a node with docker-machine

The driver for docker-machine allows you to reserve a node on Grid'5000 and to install a Linux image and Docker Engine on it, from your computer and in one single command. The reserved node is then available from commands like docker-machine ssh (standard ssh commands don't work when nodes are reserved with docker-machine).

Create a Swarm with docker-g5k

The docker-g5k program allows you to reserve several nodes, possibly on different sites, and to install a Linux image and Docker Engine on each node, in one single command. This command configures a Swarm, i.e. a set of Docker containers distributed over several nodes that communicate with each other.

Managing Docker images

One difficult aspect of using Docker is efficient images management, as (1) it is often required to load such images on many nodes simultaneously; (2) Grid'5000 connection to the internet is limited to 1 Gbps. The following strategies are recommended to manage Docker images:

Use docker load/save

To load and save images quickly, we encourage you to use docker load and docker save commands, with .tar docker images stored in your Grid'5000 home. Users home are NFS mounted, with good network performance.

Example
Terminal.png node1:
docker pull alpine
docker save -o ~/alpine.tar alpine
Terminal.png node2:
docker load -i ~/alpine.tar
docker run -it alpine

You can also use docker load command instead of retrieving the image from Docker Hub, when deploying a Swarm with docker-g5k.

Example with docker-g5k

Give read rights to your image

Terminal.png frontend:
chmod 644 ~/alpine.tar

Deploy three Docker nodes in Nancy (activate the Grid'5000 VPN before launching this command)

Terminal.png laptop:
docker-g5k create-cluster --g5k-username "user" --g5k-password "********" --g5k-image "jessie-x64-std" --g5k-reserve-nodes "nancy:3"

Load Alpine image on each node

Terminal.png laptop:
for i in {0..2}; do docker-machine ssh nancy-${i} "docker load -i /home/user/alpine.tar"; done

Use Inria private registry

The Inria registry has been closed by the end of 2019/beg. 2020.

As an alternative one can use gitlab.inria.fr to host docker images related to a git project, but this cannot be compared to the functionalities provided by a normal Docker registry.

Use a persistent virtual machine to host your registry

If you want to benefit from a private registry with good network performance, and are ready to manually install your own registry, you can request a Persistent Virtual Machine. You can look at Harbor or Portus open source projects, in order to install your own registry.

Avoid network conflict

The default network used by docker is 172.16.0.1/16 which is use internally on Grid'5000 thus this can be a source of conflict.

If you want to avoid any conflict you can set an other network. On debian you will have to edit /etc/default/docker and specify the network you want to use like this:

DOCKER_OPTS="--bip=192.168.42.1/24"

Using docker-cache.grid5000.fr

Due do the rate limiting of docker hub you might experience some troubles to fetch images. As a result you can use docker-cache.grid5000.fr which is a registry mirror.

To enable the use of the cache from a Grid'5000 node (as root or using sudo-g5k), edit /etc/docker/daemon.json with:

{
  "registry-mirrors": [
    "http://docker-cache.grid5000.fr"
  ]
}

then restart docker:

systemctl restart docker

The cache is automatically configured by g5k-setup-docker.