PMEM-environment

Generating an environment with the PMEM tools

We will build a PMEM environment thanks to the kameleon tool.

Kameleon is one of the tools to build Grid'5000 environments, along with tgz-g5k. Contrary to tgz-g5k, it allows to script the environment build process, and is lighter (build in a VM).

Kameleon is provided as a ruby gem. One can install and use it without requiring root privileges.

Kameleon's documentation:

• http://kameleon.imag.fr
• https://www.grid5000.fr/w/Environments_creation_using_Kameleon_and_Puppet

Prepare the kameleon environment

Install kameleon

pneyron@fgrenoble:~$ gem install --user kameleon-builder
Fetching: progressbar-1.10.1.gem (100%)
Successfully installed progressbar-1.10.1
Fetching: kameleon-builder-2.10.4.gem (100%)
Successfully installed kameleon-builder-2.10.4
Parsing documentation for progressbar-1.10.1
Installing ri documentation for progressbar-1.10.1
Parsing documentation for kameleon-builder-2.10.4
Installing ri documentation for kameleon-builder-2.10.4
Done installing documentation for progressbar, kameleon-builder after 1 seconds
2 gems installed

Make sure to have the latest version of kameleon-builder: 2.10.4 as of writing this.

Make sure to have the kameleon command in your path

pneyron@fgrenoble:~$ which kameleon
/home/pneyron/.gem/ruby/2.5.0/bin/kameleon

Add the Grid'5000 environment recipes to your kameleon repositories

pneyron@fgrenoble:~$ kameleon repo add grid5000 https://gitlab.inria.fr/grid5000/environments-recipes
Cloning into '/home/pneyron/.kameleon.d/repos/grid5000'...
warning: redirecting to https://gitlab.inria.fr/grid5000/environments-recipes.git/
remote: Enumerating objects: 144, done.
remote: Counting objects: 100% (144/144), done.
remote: Compressing objects: 100% (96/96), done.
remote: Total 15154 (delta 67), reused 110 (delta 44), pack-reused 15010
Receiving objects: 100% (15154/15154), 1.90 MiB | 14.11 MiB/s, done.
Resolving deltas: 100% (9391/9391), done.

Create a customized environment recipe

We create a debiantesting-x64-pmem recipe, with will build on top of the debiantesting-x64-min recipe, which is maintained by the Grid'5000 tech team.

Instead of extending the debiantesting-x64-min recipe and thus building our environment from scratch (which means running the debian installer, etc), we choose to use the from_grid5000_environment/debian10-x64-custom recipe, which will let kameleon extract the already builded debiantesting-x64-min environment tarball and add our stuff on top of it. As a reminder, kameleon does all that in a VM.

Create the debiantesting-x64-pmem recipe, which extends from_grid5000_environment/debian10-x64-custom

pneyron@fgrenoble:~/debiantesting-x64-pmem$ mkdir debiantesting-x64-pmem
pneyron@fgrenoble:~/debiantesting-x64-pmem$ kameleon new debiantesting-x64-pmem grid5000/from_grid5000_environment/debian10-x64-custom
      create  grid5000/from_grid5000_environment/debian10-x64-custom.yaml
      create  grid5000/steps/backend/qemu.yaml
      create  grid5000/steps/backend/VM.yaml
      create  grid5000/steps/aliases/defaults.yaml
      create  grid5000/steps/checkpoints/qemu.yaml
      create  grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
      create  grid5000/steps/bootstrap/download_upstream_tarball.yaml
      create  grid5000/steps/bootstrap/create_appliance.yaml
      create  grid5000/steps/bootstrap/prepare_appliance.yaml
      create  grid5000/steps/bootstrap/start_qemu.yaml
      create  grid5000/steps/disable_checkpoint.yaml
      create  grid5000/steps/export/save_appliance_VM.yaml
      create  grid5000/steps/export/create_kadeploy_environment.yaml
      create  grid5000/steps/data/helpers/create_appliance.py
      create  grid5000/steps/data/qemu-sendkeys.rb
      create  grid5000/steps/data/helpers/export_appliance.py
      create  grid5000/steps/data/helpers/kaenv-customize.py
      create  grid5000/steps/env/bashrc
      create  grid5000/steps/env/functions.sh
      create  debiantesting-x64-pmem.yaml

Adapt the debiantesting-x64-pmem recipe and add our stuff

Edit the recipe file debiantesting-x64-pmem.yaml, to add the pmem tools and make the LDAP and home directories available.

#==============================================================================
# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
#==============================================================================
#
# DESCRIPTION: debiantesting-x64-pmem
#
#==============================================================================
# This recipe extends another. To look at the step involed, run:
#   kameleon dryrun debiantesting-x64-pmem.yaml
# To see the variables that you can override, use the following command:
#   kameleon info debiantesting-x64-pmem.yaml
---
extend: grid5000/from_grid5000_environment/debian10-x64-custom.yaml

global:
  ### Grid'5000 environment information
  ## (Uncomment and change any line if needed)

  ## Frontend to run kaenv3 on
  #grid5000_frontend: "frontend"

  ## Site used in the build
  #grid5000_site: "grenoble"

  ## Environment to build from
  grid5000_environment_import_name: debiantesting-x64-min
  #grid5000_environment_import_user: "deploy"
  #grid5000_environment_import_version: ""

  ## New environment description
  #grid5000_environment_export_name: "$${kameleon_recipe_name}"
  #grid5000_environment_export_format: "tar.gz"
  grid5000_environment_export_description: "debian testing with pmem tools"

  ## Set where to store the environment and the assiated kadeploy URL base
  #grid5000_environment_export_dir: "$HOME/public/"
  #grid5000_environment_export_baseurl: "local://$HOME/public/"

  ## Optionaly, the environment postinstall script can be changed, e.g. to
  ## enable NFS homes, LDAP accounts, if not enabled in the imported env.
  grid5000_environment_export_postinstall_script: "g5k-postinstall --net debian --fstab nfs --restrict-user current"

  ### You can add below any other global variable definition
  ## See the variables which can be overloaded, by running:
  ##   kameleon info debiantesting-x64-pmem.yaml
  ## Or define any new variable you would need. e.g.:
  #my_variable: my_value

bootstrap:
  ### The bootstrap section takes in charge the import of the Grid'5000
  ## environment to customize. No modification should be needed here.
  - "@base"

setup:
  ### The setup section is where to place your customization. Add all steps
  ## required by your customization.

  ## The debiantesting-x64-min environment does not provide LDAP and home directories
  ## This step fixes this.
  - prepare_user_access:
    - update-system:
      - exec_in: apt-get -y update && apt-get -y upgrade
    - install_nfs_service:
      - exec_in: apt-get install -y nfs-common autofs
    - install_ldap_service:
      - exec_in: apt-get install -y nslcd
    - setup_ldap:
      - exec_in: mkdir -p /etc/ldap/certificates/
      - local2in:
        - /etc/ldap/certificates/ca2019.grid5000.fr.cert
        - /etc/ldap/certificates/ca2019.grid5000.fr.cert
      - local2in:
        - /etc/ldap/ldap.conf 
        - /etc/ldap/ldap.conf 
      - local2in:
        - /etc/nsswitch.conf
        - /etc/nsswitch.conf
      - local2in:
        - /etc/nslcd.conf
        - /etc/nslcd.conf

  ## Install the PMEM tools, and some additional common tools
  - install_tools:
    - install_pmem_tools:
      - exec_in: apt-get install -y ipmctl ndctl daxctl
    - install_common_tools:
      - exec_in: apt-get install -y vim-nox hwloc-nox htop dstat

export:
  ### The export section takes in charge the export of your customized Grid'5000
  ## environment. No modification should be needed here.
  - "@base"

Build your environment

To build the environment, we use a Grid'5000 node

pneyron@fgrenoble:~/debiantesting-x64-pmem$ oarsub -I
[ADMISSION RULE] Set default walltime to 1 hour.
[ADMISSION RULE] Modify resource description with type constraints
[ADMISSION_RULE] Resources properties : \{'property' => 'type = \'default\'','resources' => [{'resource' => 'host','value' => '1'}]}
[ADMISSION RULE] Job properties : maintenance = 'NO'
Import job key from file: /home/pneyron/.ssh/id_rsa
OAR_JOB_ID=1931031
Interactive mode: waiting...
Starting...
Connect to OAR job 1931031 via the node dahu-20.grenoble.grid5000.fr
[OAR] OAR_JOB_ID=1931031
[OAR] Your nodes are:
      dahu-20.grenoble.grid5000.fr*32

[pneyron@dahu-20 debiantesting-x64-pmem](1931031-->59mn)$

In the job shell, we run the build

[pneyron@dahu-20 debiantesting-x64-pmem](1931031-->58mn)$ kameleon build debiantesting-x64-pmem.yaml
Creating kameleon build directory : /home/pneyron/debiantesting-x64-pmem/build/debiantesting-x64-pmem
Starting build recipe 'debiantesting-x64-pmem.yaml'
Step 1 : bootstrap/_init_bootstrap/_init_0_create_appliance
...
...
...
Step 29 : export/create_kadeploy_environment/create_kaenv_file
--> Running the step...
Step 30 : export/create_kadeploy_environment/copy_files
--> Running the step...
[local] Copying grid5000 environment files to /home/pneyron/public/
[local] '/home/pneyron/debiantesting-x64-pmem/build/debiantesting-x64-pmem/debiantesting-x64-pmem.tar.gz' -> '/home/pneyron/public/debiantesting-x64-pmem.tar.gz'
[local] 'debiantesting-x64-pmem.yaml' -> '/home/pneyron/public/debiantesting-x64-pmem.yaml'
Step create_kadeploy_environment took: 2 secs
Step 31 : export/_clean_export/_clean_1_start_vm
--> Running the step...
Step 32 : export/_clean_export/_clean_0_delete_initial_image_at_the_end
--> Running the step...
Step _clean_export took: 0 secs

Successfully built 'debiantesting-x64-pmem.yaml'
Total duration : 156 secs

Our environment is now ready to be deployed on a troll node !

We quit this job, we will create a new one on troll nodes and of type deploy.

Deploy the environment

Reserve a troll node for 2h in a deploy job

pneyron@fgrenoble:~/debiantesting-x64-pmem$ oarsub -p "cluster='troll'" -t deploy -l nodes=1,walltime=2 -I
[ADMISSION RULE] Modify resource description with type constraints
[ADMISSION_RULE] Resources properties : \{'property' => 'type = \'default\'','resources' => [{'resource' => 'host','value' => '1'}]}
[ADMISSION RULE] Job properties : ((cluster='dahu') AND deploy = 'YES') AND maintenance = 'NO'
Import job key from file: /home/pneyron/.ssh/id_rsa
OAR_JOB_ID=1931038
Interactive mode: waiting...
Starting...
Connect to OAR job 1931038 via the node frontend
[OAR] OAR_JOB_ID=1931038
[OAR] Your nodes are:
      troll-2.grenoble.grid5000.fr*32

[pneyron@fgrenoble debiantesting-x64-pmem](1931038-->119mn)$

Open the console of the node

Since the deployment is pretty long on the troll nodes, we open the console help our patience...

pneyron@fgrenoble:~$ kaconsole3 -m troll-2

Deploy the environment on the node

Run the deployment...

[pneyron@fgrenoble debiantesting-x64-pmem](1931038-->119mn)$ kadeploy3 -a ~/public/debiantesting-x64-pmem.yaml -f $OAR_NODE_FILE -k
Deployment #D-ea5e7cb0-da6a-4b46-8726-f9f41ee2c115 started
Grab the key file /home/pneyron/.ssh/authorized_keys
Grab the tarball file /home/pneyron/public/debiantesting-x64-pmem.tar.gz
Grab the postinstall file server:///grid5000/postinstalls/g5k-postinstall.tgz
Launching a deployment on troll-2.grenoble.grid5000.fr
...
...
...
End of step Deploy[BootNewEnvHardReboot] after 196s
End of deployment for troll-2.grenoble.grid5000.fr after 526s
End of deployment on cluster troll after 526s
Deployment #D-ea5e7cb0-da6a-4b46-8726-f9f41ee2c115 done

The deployment is successful on nodes
troll-2.grenoble.grid5000.fr
[pneyron@fgrenoble debiantesting-x64-pmem](1931038-->109mn)$

Well done, we can now ssh to the machine as root or with our user account !