PMEM-environment

From Grid5000
Jump to navigation Jump to search

Generating an environment with the PMEM tools

We will build a PMEM environment thanks to the kameleon tool.

Kameleon is one of the tools to build Grid'5000 environments, along with tgz-g5k. Contrary to tgz-g5k, it allows to script the environment build process, and is lighter (build in a VM).

Kameleon is provided as a ruby gem. One can install and use it without requiring root privileges.

Kameleon's documentation:

Prepare the kameleon environment

Install kameleon
pneyron@fgrenoble:~$ gem install --user kameleon-builder
Fetching: progressbar-1.10.1.gem (100%)
Successfully installed progressbar-1.10.1
Fetching: kameleon-builder-2.10.4.gem (100%)
Successfully installed kameleon-builder-2.10.4
Parsing documentation for progressbar-1.10.1
Installing ri documentation for progressbar-1.10.1
Parsing documentation for kameleon-builder-2.10.4
Installing ri documentation for kameleon-builder-2.10.4
Done installing documentation for progressbar, kameleon-builder after 1 seconds
2 gems installed

Make sure to have the latest version of kameleon-builder: 2.10.4 as of writing this.

Make sure to have the kameleon command in your path 

pneyron@fgrenoble:~$ which kameleon
/home/pneyron/.gem/ruby/2.5.0/bin/kameleon
Add the Grid'5000 environment recipes to your kameleon repositories
pneyron@fgrenoble:~$ kameleon repo add grid5000 https://gitlab.inria.fr/grid5000/environments-recipes
Cloning into '/home/pneyron/.kameleon.d/repos/grid5000'...
warning: redirecting to https://gitlab.inria.fr/grid5000/environments-recipes.git/
remote: Enumerating objects: 144, done.
remote: Counting objects: 100% (144/144), done.
remote: Compressing objects: 100% (96/96), done.
remote: Total 15154 (delta 67), reused 110 (delta 44), pack-reused 15010
Receiving objects: 100% (15154/15154), 1.90 MiB | 14.11 MiB/s, done.
Resolving deltas: 100% (9391/9391), done.

Create a customized environment recipe

We create a debiantesting-x64-pmem recipe, which will build on top of the debiantesting-x64-min environment, which is maintained by the Grid'5000 tech team.

Using Debian testing and not Debian 10 is important, since the support of PMEM in Debian 10 lacks the latest functionalities provided by more recent Linux kernels (e.g. kmem).

Instead of extending the debiantesting-x64-min recipe and thus building our environment from scratch (which means running the debian installer, etc), we choose to use the from_grid5000_environment/debian10-x64-custom recipe, which will let kameleon extract the already builded debiantesting-x64-min environment tarball and add our stuff on top of it. As a reminder, kameleon does all that in a VM.

Create the debiantesting-x64-pmem recipe, which extends from_grid5000_environment/debian10-x64-custom
pneyron@fgrenoble:~/debiantesting-x64-pmem$ mkdir debiantesting-x64-pmem
pneyron@fgrenoble:~/debiantesting-x64-pmem$ kameleon new debiantesting-x64-pmem grid5000/from_grid5000_environment/debian10-x64-custom
      create  grid5000/from_grid5000_environment/debian10-x64-custom.yaml
      create  grid5000/steps/backend/qemu.yaml
      create  grid5000/steps/backend/VM.yaml
      create  grid5000/steps/aliases/defaults.yaml
      create  grid5000/steps/checkpoints/qemu.yaml
      create  grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
      create  grid5000/steps/bootstrap/download_upstream_tarball.yaml
      create  grid5000/steps/bootstrap/create_appliance.yaml
      create  grid5000/steps/bootstrap/prepare_appliance.yaml
      create  grid5000/steps/bootstrap/start_qemu.yaml
      create  grid5000/steps/disable_checkpoint.yaml
      create  grid5000/steps/export/save_appliance_VM.yaml
      create  grid5000/steps/export/create_kadeploy_environment.yaml
      create  grid5000/steps/data/helpers/create_appliance.py
      create  grid5000/steps/data/qemu-sendkeys.rb
      create  grid5000/steps/data/helpers/export_appliance.py
      create  grid5000/steps/data/helpers/kaenv-customize.py
      create  grid5000/steps/env/bashrc
      create  grid5000/steps/env/functions.sh
      create  debiantesting-x64-pmem.yaml
Adapt the debiantesting-x64-pmem recipe and add our stuff

Edit the recipe file debiantesting-x64-pmem.yaml, to add the pmem tools and make the LDAP and home directories available. 

#==============================================================================
# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
#==============================================================================
#
# DESCRIPTION: debiantesting-x64-pmem
#
#==============================================================================
# This recipe extends another. To look at the step involed, run:
#   kameleon dryrun debiantesting-x64-pmem.yaml
# To see the variables that you can override, use the following command:
#   kameleon info debiantesting-x64-pmem.yaml
---
extend: grid5000/from_grid5000_environment/debian10-x64-custom.yaml

global:
  ### Grid'5000 environment information
  ## (Uncomment and change any line if needed)

  ## Frontend to run kaenv3 on
  #grid5000_frontend: "frontend"

  ## Site used in the build
  #grid5000_site: "grenoble"

  ## Environment to build from
  grid5000_environment_import_name: debiantesting-x64-min
  #grid5000_environment_import_user: "deploy"
  #grid5000_environment_import_version: ""

  ## New environment description
  #grid5000_environment_export_name: "$${kameleon_recipe_name}"
  #grid5000_environment_export_format: "tar.gz"
  grid5000_environment_export_description: "debian testing with pmem tools"

  ## Set where to store the environment and the assiated kadeploy URL base
  #grid5000_environment_export_dir: "$HOME/public/"
  #grid5000_environment_export_baseurl: "local://$HOME/public/"

  ## Optionaly, the environment postinstall script can be changed, e.g. to
  ## enable NFS homes, LDAP accounts, if not enabled in the imported env.
  grid5000_environment_export_postinstall_script: "g5k-postinstall --net debian --fstab nfs --restrict-user current"

  ### You can add below any other global variable definition
  ## See the variables which can be overloaded, by running:
  ##   kameleon info debiantesting-x64-pmem.yaml
  ## Or define any new variable you would need. e.g.:
  #my_variable: my_value

bootstrap:
  ### The bootstrap section takes in charge the import of the Grid'5000
  ## environment to customize. No modification should be needed here.
  - "@base"

setup:
  ### The setup section is where to place your customization. Add all steps
  ## required by your customization.

  ## The debiantesting-x64-min environment does not provide LDAP and home directories
  ## This step fixes this, by installing the required software package and copying the
  ## required configuration files from the Grid'5000 node where the build will be run.
  - prepare_user_access:
    - update-system:
      - exec_in: apt-get -y update && apt-get -y upgrade
    - install_nfs_service:
      - exec_in: apt-get install -y nfs-common autofs
    - install_ldap_service:
      - exec_in: apt-get install -y nslcd
    - setup_ldap:
      - exec_in: mkdir -p /etc/ldap/certificates/
      - local2in:
        - /etc/ldap/certificates/ca2019.grid5000.fr.cert
        - /etc/ldap/certificates/ca2019.grid5000.fr.cert
      - local2in:
        - /etc/ldap/ldap.conf 
        - /etc/ldap/ldap.conf 
      - local2in:
        - /etc/nsswitch.conf
        - /etc/nsswitch.conf
      - local2in:
        - /etc/nslcd.conf
        - /etc/nslcd.conf

  ## Install the PMEM tools, and some additional common tools
  - install_tools:
    - install_pmem_tools:
      - exec_in: apt-get install -y ipmctl ndctl daxctl
    - install_common_tools:
      - exec_in: apt-get install -y vim-nox hwloc-nox htop dstat

export:
  ### The export section takes in charge the export of your customized Grid'5000
  ## environment. No modification should be needed here.
  - "@base"

One can easily add more in the environment.

Build your environment

To build the environment, we use a Grid'5000 node 

pneyron@fgrenoble:~/debiantesting-x64-pmem$ oarsub -I
[ADMISSION RULE] Set default walltime to 1 hour.
[ADMISSION RULE] Modify resource description with type constraints
[ADMISSION_RULE] Resources properties : \{'property' => 'type = \'default\'','resources' => [{'resource' => 'host','value' => '1'}]}
[ADMISSION RULE] Job properties : maintenance = 'NO'
Import job key from file: /home/pneyron/.ssh/id_rsa
OAR_JOB_ID=1931031
Interactive mode: waiting...
Starting...
Connect to OAR job 1931031 via the node dahu-20.grenoble.grid5000.fr
[OAR] OAR_JOB_ID=1931031
[OAR] Your nodes are:
      dahu-20.grenoble.grid5000.fr*32

[pneyron@dahu-20 debiantesting-x64-pmem](1931031-->59mn)$

In the job shell, we run the build 

[pneyron@dahu-20 debiantesting-x64-pmem](1931031-->58mn)$ kameleon build debiantesting-x64-pmem.yaml
Creating kameleon build directory : /home/pneyron/debiantesting-x64-pmem/build/debiantesting-x64-pmem
Starting build recipe 'debiantesting-x64-pmem.yaml'
Step 1 : bootstrap/_init_bootstrap/_init_0_create_appliance
...
...
...
Step 29 : export/create_kadeploy_environment/create_kaenv_file
--> Running the step...
Step 30 : export/create_kadeploy_environment/copy_files
--> Running the step...
[local] Copying grid5000 environment files to /home/pneyron/public/
[local] '/home/pneyron/debiantesting-x64-pmem/build/debiantesting-x64-pmem/debiantesting-x64-pmem.tar.gz' -> '/home/pneyron/public/debiantesting-x64-pmem.tar.gz'
[local] 'debiantesting-x64-pmem.yaml' -> '/home/pneyron/public/debiantesting-x64-pmem.yaml'
Step create_kadeploy_environment took: 2 secs
Step 31 : export/_clean_export/_clean_1_start_vm
--> Running the step...
Step 32 : export/_clean_export/_clean_0_delete_initial_image_at_the_end
--> Running the step...
Step _clean_export took: 0 secs

Successfully built 'debiantesting-x64-pmem.yaml'
Total duration : 156 secs

Our environment is now ready to be deployed on a troll node !

We quit this job, we will create a new one on troll nodes and of type deploy.

Deploy the environment

Reserve a troll node for 2h in a deploy job
pneyron@fgrenoble:~/debiantesting-x64-pmem$ oarsub -p "cluster='troll'" -t deploy -l nodes=1,walltime=2 -I
[ADMISSION RULE] Modify resource description with type constraints
[ADMISSION_RULE] Resources properties : \{'property' => 'type = \'default\'','resources' => [{'resource' => 'host','value' => '1'}]}
[ADMISSION RULE] Job properties : ((cluster='dahu') AND deploy = 'YES') AND maintenance = 'NO'
Import job key from file: /home/pneyron/.ssh/id_rsa
OAR_JOB_ID=1931038
Interactive mode: waiting...
Starting...
Connect to OAR job 1931038 via the node frontend
[OAR] OAR_JOB_ID=1931038
[OAR] Your nodes are:
      troll-2.grenoble.grid5000.fr*32

[pneyron@fgrenoble debiantesting-x64-pmem](1931038-->119mn)$
Open the console of the node

Since the deployment is pretty long on the troll nodes, we open the console help our patience... 

pneyron@fgrenoble:~$ kaconsole3 -m troll-2
Deploy the environment on the node

Run the deployment... 

[pneyron@fgrenoble debiantesting-x64-pmem](1931038-->119mn)$ kadeploy3 -a ~/public/debiantesting-x64-pmem.yaml -f $OAR_NODE_FILE -k
Deployment #D-ea5e7cb0-da6a-4b46-8726-f9f41ee2c115 started
Grab the key file /home/pneyron/.ssh/authorized_keys
Grab the tarball file /home/pneyron/public/debiantesting-x64-pmem.tar.gz
Grab the postinstall file server:///grid5000/postinstalls/g5k-postinstall.tgz
Launching a deployment on troll-2.grenoble.grid5000.fr
...
...
...
End of step Deploy[BootNewEnvHardReboot] after 196s
End of deployment for troll-2.grenoble.grid5000.fr after 526s
End of deployment on cluster troll after 526s
Deployment #D-ea5e7cb0-da6a-4b46-8726-f9f41ee2c115 done

The deployment is successful on nodes
troll-2.grenoble.grid5000.fr
[pneyron@fgrenoble debiantesting-x64-pmem](1931038-->109mn)$

Well done, we can now ssh to the machine as root or with our user account !

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=PMEM-environment&oldid=85781"