Deep Learning Frameworks: Difference between revisions

From Grid5000
Jump to navigation Jump to search
(pytorch ppc64)
Line 96: Line 96:
For convenience, we provide a version of conda for ppc64 as a module:
For convenience, we provide a version of conda for ppc64 as a module:


<pre
<pre>
$ module load miniconda3
$ module load miniconda3
$ conda --help
$ conda --help
Line 131: Line 131:
Test that it works:
Test that it works:


<pre>$ python -c &quot;import torch; print(torch.cuda.is_available())&quot;</pre>
<pre>$ python -c &quot;import torch; print(torch.cuda.is_available())&quot;
True
</pre>

Revision as of 18:38, 27 January 2021

Note.png Note

This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

This page describes installation steps of common Deep Learning frameworks.

Deep learning on x86_64 nodes (common case)

pip will be used to install the frameworks (conda could be used much the same way). Installation is performed under your home directory.

Reserve some GPU nodes with OAR

  • Reserve a node with some GPUs (see the Hardware page for the list of sites and clusters with GPUs).

For instance, to reserve one GPU using OAR:

$ oarsub -I -l gpu=1

(remember to add '-q production' option if you want to reserve a GPU from Nancy "production" resources)


To reserve the full node:

$ oarsub -I -l host=1

To reserve a gpu or a full node on a specific cluster, add to the oarsub command:

-p cluster=<clustername>
  • Once connected to the node, check GPU presence and the available CUDA version:
$ nvidia-smi 
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
(...)

PyTorch

For instance (as of May 2020), selecting “Stable”, “Linux”, “Pip”, “Python”, “Cuda 10.1” gives this command to execute:

$ pip3 install torch==1.5.0+cu101 torchvision==0.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
  • Check if PyTorch is correctly installed to works with GPU:
$ python3 -c "import torch; print(torch.cuda.is_available())"

Tensorflow (with Keras)

  • Go on Tensorflow website to see the installation commands. As of May 2020 (tensorflow v2.2.0), it is:
$ pip3 install --upgrade pip
$ pip3 install tensorflow
  • To use GPUs, TensorFlow requires CudNN library. We provide it as a module to load:
$ module load cudnn
  • Now check if TensorFlow is correctly installed to works with GPU:
$ python3 -c "import tensorflow as tf; print('Num GPUs Available:', len(tf.config.experimental.list_physical_devices('GPU')))"

Note: This install TensorFlow v2. If you need TensorFlow v1, see https://www.tensorflow.org/guide/migrate

MXNet

  • Go on MXNet website to see the installation command that suits you.

For instance (as of May 2020), selecting “Linux”, “Python”, “GPU” and “Pip”, the command to execute (in order to use Cuda 10.1) is:

$ pip3 install mxnet-cu101
  • Check if PyTorch is correctly installed to works with GPU:
$ python3 -c "import mxnet; print('Num GPUs Available:', mxnet.context.num_gpus())"

Additional resources

  • An in-depth tutorial contributed by a Grid'5000 user, Ismael Bada
  • Many Docker images exist with ready-to-use Deep Learning software stack. They can be executed using Docker or Singularity tools (using appropriate options to enable GPU usage). See wiki pages to learn how to use these tools in Grid'5000.
  • If you want to use virtualenv to manage your Python packages, it is available in Grid'5000 standard environments. Create your environment with python3 -m venv <env_directory> and activate it using source <env_directory>/bin/activate before using pip and installed packages.
  • If you prefer to use conda to manage your Python packages, it is available in Grid'5000 as a module. Just execute "module load miniconda3" from a node or a frontend to make it available.


Deep learning on ppc64 nodes

We have an IBM cluster with many GPUs. But since it is running with a ppc64 architecture, many deep learning frameworks cannot be easily installed.

Reserve ppc64 GPU nodes with OAR

To reserve a full node:

$ oarsub -I -p cluster=drac -l host=1,walltime=1:00
  • Once connected to the node, check GPU presence and the available CUDA version:
$ nvidia-smi 
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
(...)
Note.png Note

ppc64 nodes come with a known-working Nvidia driver version in their default environment, but it only supports CUDA versions up to 10.1. If you install a more recent driver or deploy your own images, you may experience frequent system crashes with recent nvidia drivers on Debian or Ubuntu. CentOS seems unaffected by the crashes. See nvidia developer forum

IBM PowerAI conda channel

IBM provides a conda channel called PowerAI with several deep learning tools built for ppc64.

It is the easiest way to install tools, but the available versions are often not up-to-date. In addition, we are forced to use PowerAI 1.6.2 specifically because newer versions are not compatible with CUDA 10.1.

For convenience, we provide a version of conda for ppc64 as a module:

$ module load miniconda3
$ conda --help

See https://www.ibm.com/support/knowledgecenter/SS5SF7_1.7.0/navigation/wmlce_install.htm for instructions with PowerAI. We detail how to install a few tools below.

Note.png Note

Conda packages will be installed in your home directory at ~/.conda/. Deep learning tools can easily take several GB of space: you may need to clean up from time to time or request an increased disk quota.

PyTorch

Install with IBM PowerAI

Load conda and create an environment (PowerAI needs Python 3.7) :

$ module load miniconda3
$ eval "$(conda shell.bash hook)"
$ conda create --name pytorch-ppc64-py37 python=3.7
$ conda activate pytorch-ppc64-py37

Add PowerAI repository:

$ conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/

Install a pytorch version that is built against CUDA 10.1:

$ conda install pytorch powerai-release=1.6.2

Test that it works:

$ python -c "import torch; print(torch.cuda.is_available())"
True