Kadeploy-v3
From Grid5000
Caution
| Warning |
|---|---|
Kadeploy 3 is currently provided in a 3.1-4 version. Please submit bugs in the Kadeploy3 component of our bugzilla installation or send mail to support-staff@lists.grid5000.fr. If you use Kadeploy 3, please consider subscribing to kadeploy3-users@lists.gforge.inria.fr (http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/kadeploy3-users). | |
The Kadeploy 3 software suite
Kadeploy 3 is composed of 8 tools designed for the end users: kadeploy, kaenv, kareboot, kastat, kaconsole, kanodes, kapower, karights.
For the official up-to-date documentation, please refer to Official documentation of the kadeploy project on the gforge webpage.
Kadeploy client
The Kadeploy client is actually the user interface for the Kadeploy software. It can be used by using the kadeploy3 command. The CLI looks like this:
> kadeploy3 -h
Usage: kadeploy3 [options]
Contact: kadeploy3-users@lists.grid5000.fr
General options:
-a, --env-file ENVFILE File containing the environment description
-b, --block-device BLOCKDEVICE Specify the block device to use
-d, --debug-mode Activate the debug mode
-e, --env-name ENVNAME Name of the recorded environment to deploy
-f, --file MACHINELIST Files containing list of nodes (- means stdin)
-k, --key [FILE] Public key to copy in the root's authorized_keys,
if no argument is specified, use the authorized_keys
-m, --machine MACHINE Node to run on
--multi-server Activate the multi-server mode
-n, --output-ko-nodes FILENAME File that will contain the nodes not correctly deployed
-o, --output-ok-nodes FILENAME File that will contain the nodes correctly deployed
-p, --partition-number NUMBER Specify the partition number to use
-r, --reformat-tmp FSTYPE Reformat the /tmp partition with the given filesystem type
(ext[234] are allowed)
-s, --script FILE Execute a script at the end of the deployment
-u, --user USERNAME Specify the user
-v, --version Get the version
--vlan VLANID Set the VLAN"
-w, --set-pxe-profile FILE Set the PXE profile (use with caution)
--set-pxe-pattern FILE Specify a file containing the substituation of a pattern for
each node in the PXE profile (the NODE_SINGULARITY pattern must
be used in the PXE profile)
-x, --upload-pxe-files FILES Upload a list of files (file1,file2,file3) to the "tftp_images_path"
directory. Those files will be prefixed with "pxe-$username-"
--env-version NUMBER Number of version of the environment to deploy
--server STRING Specify the Kadeploy server to use
-V, --verbose-level VALUE Verbose level between 0 to 4
Advanced options:
--write-workflow-id FILE Write the workflow id in a file
--ignore-nodes-deploying Allow to deploy even on the nodes tagged as "currently deploying"
(use this only if you know what you do)
--disable-bootloader-install Disable the automatic installation of a bootloader for a
Linux based environment
--disable-disk-partitioning Disable the disk partitioning
--breakpoint MICROSTEP Set a breakpoint just before launching the give micro-step, the
syntax is macrostep:microstep (use this only if you know what you do)
--set-custom-operations FILE Add some custom operations defined in a file
--reboot-classical-timeout V Overload the default timeout for classical reboots
--reboot-kexec-timeout V Overload the default timeout for kexec reboots
--force-steps STRING Undocumented, for administration purpose only
At least, Kadeploy must be called with one node and an environment. The nodes to deploy can be specified by using several -m|--machine options, or the -f|--file options (one node per line in the file), or a mix of both.
The environment can be specified with the -e|--env-name option if you want to use an environment recorded in the environment database or with the -a|--env-file options if you want to use an environment described in a file. Here are some examples:
> kadeploy3 -m gdx-5.orsay.grid5000.fr -e lenny-x64-nfs-1.0 -o nodes_ok -n nodes_ko > kadeploy3 -m gdx-[5-12].orsay.grid5000.fr -e lenny-x64-base -o nodes_ok -n nodes_ko > kadeploy3 -f nodes -a custom_env.dsc > kadeploy3 -f nodes -m gdx-5.orsay.grid5000.fr -a custom_env.dsc > cat nodefile|kadeploy3 -f - -e lenny-x64-base
We present now several use cases.
Use case 1 - basic usage - deployment of a node
> kadeploy3 -m gdx-5.orsay.grid5000.fr \
-e lenny-x64-nfs-1.0 \
--verbose-level 4 \
-k ~/.ssh/id_rsa.pub
This command performs the deployment of the environment lenny-x64-nfs-1.0 on the node gdx-5.orsay.grid5000.fr and copies the SSH public key ~/.ssh/id_rsa.pub of the user in the deployed environment to allow a direct connection with the root account. Furthermore, the verbose level is set to 4, which means that you want the maximum verbose information.
Use case 2 - basic usage - deployment of a range of nodes
> kadeploy3 -m gdx-[45-51].orsay.grid5000.fr \
-e lenny-x64-base \
-k
This command performs the deployment of the environment lenny-x64-base on the nodes gdx-45.orsay.grid5000.fr , gdx-46.orsay.grid5000.fr , ..., gdx-51.orsay.grid5000.fr . Furthermore, it copies the entries of the ~/.ssh/authorized_keys user file in the /root/.ssh/authorized_keys of the deployed nodes.
Use case 3 - basic usage - deployment of a set of nodes
> kadeploy3 -f ~/machinefile \
-e custom_env \
-l johnsmith \
-o nodes_ok -n nodes_ko
This command uses the environment custom_env of the user johnsmith to deploy the nodes specified in ~/machinefile . The list of the nodes correctly deployed will be written in the file specified with the -o|--output-ok-nodes option. Idem for the nodes not correctly deployed with the -o|--output-ko-nodes option. Refer to the part about Kaenv to know more about the environment management.
Use case 4 - basic usage - execution of a script after deployment
> kadeploy3 -f $OAR_NODE_FILE \
-a ~/my-lenny.dsc \
-r ext3 \
-p 4 \
-s ~/launcher.sh
This command performs the deployment of the environment described by the file ~/my-lenny.dsc (useful if you don't want to share your environment with the other users) on the nodes specified in the file pointed by $OAR_NODE_FILE (typically a variable set by the resource manager). We specify here that we want the /tmp partition to be reformated. Furthermore, we specify that we want to deploy the environment on the 4th disk partition, instead of the default one. Finally, we ask to execute the script ~/launcher.sh at the end of the deployment.
Use case 5 - advanced usage - play with breakpoint
> kadeploy3 -m gdx-5.orsay.grid5000.fr \
-e lenny-x64-nfs-1.0 \
--verbose-level 4 \
--breakpoint BroadcastEnvKastafior:manage_user_post_install \
-d
This kind of command can be used for debug purpose. It performs a deployment with the maximum verbose level and it asks to stop the deployment workflow just before executing the manage_user_post_install micro-step of the BroadcastEnvKastafior macro-step. Thus you will be able to connect in the deployment environment and to debug what you want. Furthermore, the full output of the distant commands performed is shown.
Use case 6 - advanced usage - specific PXE profile
> kadeploy3 -m gdx-[5-10].orsay.grid5000.fr \
-e lenny-x64-nfs-1.0 \
-w ~/pxe_profile -x "~/custom-kernel,~/custom-initrd" \
--set-pxe-pattern ~/singularities
In some specific case, you may want to use a specific PXE profile to boot your nodes. To do this, you have to provide a PXE profile. Warning, the files used in your PXE profil (Comboot, kernel, initrd, ...) must be served by the TFTP server of Kadeploy. For that purpose, Kadeploy offers a option to stage some files in the TFTP server root directory: -x|--upload-pxe-files "your_files" , which makes the files available with the following path scheme to use in the pxelinux configuration file: kernels/pxe-username--your_file_basename (prefix added to avoid conficts with files used by other users).
Here is an example of PXE profile that uses uploaded files:
PROMPT 1
SERIAL 0 38400
DEFAULT bootlabel
DISPLAY messages
TIMEOUT 50
label bootlabel
KERNEL kernels/pxe-ejeanvoine-custom-kernel
APPEND initrd=kernels/pxe-ejeanvoine-custom-initrd root=/dev/sda3 node_id=NODE_SINGULARITY
You can notice the NODE_SINGULARITY pattern used in the PXE profile. Thanks to the --set-pxe-pattern option, you can also provide a file that defines a value in the PXE profile that depends on the node concerned. This file must define on each line a couple of value as follows : hostname,node singularity. In our example, the file ~/singularities can contains something like:
gdx-5.orsay.grid5000.fr,1 gdx-6.orsay.grid5000.fr,2 gdx-7.orsay.grid5000.fr,3 gdx-8.orsay.grid5000.fr,3 gdx-9.orsay.grid5000.fr,4 gdx-10.orsay.grid5000.fr,5
Use case 7 - advanced usage - specific bootloader requirement
> kadeploy3 -m gdx-5.orsay.grid5000.fr \
-e Custom_linux_env \
--disable-bootloader-install
If you deploy a Linux based environment and if the administrator choose to boot the nodes with the chainload fashion, Kadeploy will install automatically a bootloader on the deployment partition. In some cases, you may want to bypass this installation because you have installed at the time of a previous deployment another bootloader. This allows to avoid the overriding of the installed bootloader. However, if no bootloader is installed or if the installed bootdloader is not able to boot your environment, the won't be reachable at the end of the deployment.
Use case 8 - advanced usage - get a workflow id for an external deployment tracking
> kadeploy3 -m gdx-5.orsay.grid5000.fr \
-e Custom_linux_env \
--write-workflow-id wid_file
This command performs the deployment of the Custom_linux_env environment and write the workflow id of this deployment in the file wid_file . The aim of getting the deployment id is to monitor the deployment from an extern tool thanks to the Kanodes tool.
Use case 9 - expert usage - modify the deployment workflow
> kadeploy3 -m gdx-5.orsay.grid5000.fr \
-e "FreeBSD 7.1" \
--force-steps "SetDeploymentEnv|SetDeploymentEnvProd:2:100&
BroadcastEnv|BroadcastEnvKastafior:2:300&
BootNewEnv|BootNewEnvKexec:1:150"
If you are a power user, you can specify the full Kadeploy workflow and bypass the default configuration. Use it at your own risk since the nodes may not support all the Kadeploy features like the Kexec optimization for instance. The syntax for the --force-steps option is the same that for the macrostep field if the Kadeploy configuration. The difference is that the three macrostep are defined on the same line, with the \& character as a delimiter between the macro-steps. Warning, you must define at least one implementation for each macro-step, without newline (unlike the example).
Use case 10 - expert usage - insert custom operations in the deployment workflow
> kadeploy3 -m gdx-5.orsay.grid5000.fr \
-e lenny-x64-nfs-1.0 \
--set-custom-operations ~/custom_ops
For very specific purpose, you can add some custom operations in the deployment workflow. To do this, you have to specify these operations in a file where each line specify the operations that must be executed at the beginning of a micro-step. The syntax is: macrostep,microstep@cmd1\%arg\%dir,cmd2\%arg\%dir,...,cmdN\%arg\%dir . You can specify two kinds of operations: send and exec . For the send operation, arg is used to specify a file name and dir is used to specify the destination directory. Concerning the exec operation, arg is used to specify the command to execute and dir must be empty.
Here is an example of a file that contains custom operations:
> cat ~/custom_ops SetDeploymentEnvUntrusted,format_tmp_part@send%/tmp/script.sh%/dest,exec%/dest/script.sh% BootNewEnvKexec,reboot@exec%echo "s" > /proc/sysrq-trigger%
Kareboot
Kareboot can be used by using the kareboot3 command. The CLI looks like this:
> kareboot3 -h
Usage: kareboot3 [options]
Contact: kadeploy3-users@lists.gforge.inria.fr
General options:
-b, --block-device BLOCKDEVICE Specify the block device to use
-c, --check-prod-env Check if the production environment has been detroyed
-e, --env-name ENVNAME Name of the recorded environment
-f, --file MACHINELIST Files containing list of nodes (- means stdin)
-k, --key [FILE] Public key to copy in the root's authorized_keys,
if no argument is specified, use the authorized_keys
-l, --reboot-level VALUE Reboot level (soft, hard, very_hard)
-m, --machine MACHINE Reboot the given machines
--multi-server Activate the multi-server mode
-n, --output-ko-nodes FILENAME File that will contain the nodes not correctly rebooted
-o, --output-ok-nodes FILENAME File that will contain the nodes correctly rebooted
-p, --partition-number NUMBER Specify the partition number to use
-r, --reboot-kind REBOOT_KIND Specify the reboot kind (set_pxe, simple_reboot,
deploy_env, env_recorded)
-u, --user USERNAME Specify the user
-v, --version Get the version
-w, --set-pxe-profile FILE Set the PXE profile (use with caution)
--set-pxe-pattern FILE Specify a file containing the substituation of a pattern for
each node in the PXE profile (the NODE_SINGULARITY pattern must
be used in the PXE profile)
-x, --upload-pxe-files FILES Upload a list of files (file1,file2,file3) to the "tftp_images_path"
directory. Those files will be prefixed with "pxe-$username-"
--env-version NUMBER Specify the environment version
--no-wait Do not wait the end of the reboot
--server STRING Specify the Kadeploy server to use
-V, --verbose-level VALUE Verbose level between 0 to 4
--reboot-classical-timeout V Overload the default timeout for classical reboots
At least, Kareboot must be called with one node and a reboot kind. The nodes to reboot can be specified by using several -m|--machine options, or the -f|--file options (one node per line in the file), or a mix of both. The expected values for the -r|--reboot-kind are:
-
simple_reboot: perform a simple reboot of the nodes. Kareboot firstly tries to perform a soft reboot, then a hard reboot is performed and lastly a very hard reboot if it doesn't success before. -
set_pxe: modify the PXE profile with the one given with the-w|--set-pxe-profileoptions and perform a simple reboot. -
env_recorded: perform a reboot on an environment that is already deployed (for instance, the production environment on the production part). This operation must be used with the-eand-poptions at least. -
deploy_env: perform a reboot on the deployment environment. This can be used with the-k|--keyoption.
Here are some basic examples:
> kareboot3 -m gdx-5.orsay.grid5000.fr -r simple_reboot
> kareboot3 -m gdx-[5-8].orsay.grid5000.fr -r simple_reboot
> cat nodefile|kareboot3 -f - -r simple_reboot
> kareboot3 -m gdx-5.orsay.grid5000.fr -r simple_reboot -o reboot_ok.txt \
-n reboot_ko.txt
> kareboot3 -f nodes -r set_pxe -w ~/customized_pxe_profile
> kareboot3 -f nodes -r set_pxe -w ~/customized_pxe_profile -l hard \
-x "~/custom_kernel,~/custom_initrd" \
--set-pxe-pattern ~/singularities (Cf. Kadeploy use case 6)
> kareboot3 -f nodes -r deploy_env -k .ssh/id_rsa
> kareboot3 -r env_recorded -e production_environment \
-p 2 -u root -m gdx-5.orsay.grid5000.fr
> kareboot3 -r env_recorded -e production_environment \
-p 2 -u root -m gdx-5.orsay.grid5000.fr \
--no-wait
Kareboot can be used to manage the demolishing environment. Typically, at the end of a reservation with deployment, the resource manager will perform a reboot on the production environment. By using the -c|--check-prod-env option (for instance: kareboot -f nodes -r env_recorded -c ), Kareboot firstly checks if the deployed environment on the involved nodes is tagged like a demolishing environment. If the environment is considered as demolishing , Kareboot does not perform a reboot and returns the 2 value. In this case, the production environment has been destroyed and should be deployed again. If the environment is not considered as demolishing , the reboot is performed and a check is performed at the end of the reboot to ensure that the production environment is correctly deployed. If the nodes are correctly rebooted on the production environment, Kareboot returns the 0 value. Otherwise it returns the 1 value, what means that the production environment has been destroyed and that it should be deployed again.
An environment is considered demolishing if the number of failures after a reboot on the production environment (when the -c|--check-prod-env optionis used with Kareboot), is higher than the threshold specify in the configuration file. In the Kaenv part you can find the way to reset the demolishing counter of an environment by using an option of Kaenv.
Warning, if the --no-wait option is used, Kareboot won't wait the end of the reboot to exit. Thus, this option cannot be used with the -c|--check-prod-env , -o, --output-ok-nodes and -n, --output-ko-nodes options.
Kaenv
Command line interface
Kaenv can be used by using the kaenv3 command. The CLI looks like this:
> kaenv3 -h
Usage: kaenv3 [options]
Contact: kadeploy3-users@lists.grid5000.fr
General options:
-a, --add ENVFILE Add an environment
-d, --delete ENVNAME Delete an environment
-l, --list List environments
-m, --files-to-move FILES Files to move (src1:dst1,src2:dst2,...)
-p, --print ENVNAME Print an environment
-s, --show-all-versions Show all versions of an environment
-t, --visibility-tag TAG Set the visibility tag (private, shared, public)
-u, --user USERNAME Specify the user
--env-version NUMBER Specify the version
--server STRING Specify the Kadeploy server to use
Advanced options:
--remove-demolishing-tag ENVNAME Remove demolishing tag on an environment
--set-visibility-tag ENVNAME Set the visibility tag on an environment
--update-tarball-md5 ENVNAME Update the MD5 of the environment tarball
--update-preinstall-md5 ENVNAME Update the MD5 of the environment preinstall
--update-postinstalls-md5 ENVNAME Update the MD5 of the environment postinstalls
--move-files Move the files of the environment (for administrators only)
We present now several use cases.
Use case 1 - list the environments
> kaenv3 -l
This command lists the environment that you have previously recorded, and the public environments.
Use case 2 - list the shared environments recorded by another user
> kaenv3 -l -u johnsmith -s
This command lists the environment of the user johnsmith . If you use "*" as a user value, it lists the environments of all the users. Furthermore, the -s|--show-all-versions option is used to show all the versions of each environment. If this option is not specified, only the version is displayed.
Use case 3 - print an environment
> kaenv3 -p FreeBSD --env-version 3 -u johnsmith
This command lists prints the version 3 of the environment FreeBSD that belongs to johnsmith . If no version number is given, the last version of the environment is printed. To print an environment you own, there is no need to use the -u|--user option.
Use case 4 - add an environment described in a file
> kaenv3 -a ~/new_env.dsc
This command adds the environment defined in the file ~/new_env.dsc .
Use case 5 - add an environment described in an http file
> kaenv3 -a http://www.grid5000.fr/pub/johnsmith/env.desc
This command adds the environment defined in the file http://www.grid5000.fr/pub/johnsmith/env.desc .
Use case 6 - delete an environment
> kaenv3 -d FreeBSD --env-version 2
This command deletes the version 2 of the environment FreeBSD from the environment database. If no version number is given, all the versions are deleted.
Use case 7 - remove the demolishing property of an environment
> kaenv3 --remove-demolishing-tag FreeBSD --env-version 3
This command resets the demolishing counter of the version 3 of the environment FreeBSD . If no version number is given, the latest version of the environment is considered.
Use case 8 - update the tarball of an environment
> kaenv3 --update-tarball-md5 sidx64-base
This command is useful if you modify the tarball of the environment sidx64-base without modifying the kernel or the initrd and if you do not want to record a new environment. Thus, it will update the MD5 of the tarball file. This operation is required if something change in the tarball, otherwise the environment will be unusable.
Use case 9 - update the postinstalls of an environment
> kaenv3 --update-postinstalls-md5 sidx64-base
This command does the same thing than the precedent one but it concerns the post-install files. This operation is required if something change in the post-install files, otherwise the environment will be unusable.
Use case 10 - define the visibility of an environment
> kaenv3 --set-visibility-tag sidx64-base --env-version 3 -t private
This command allows to define the environment sidx64-base version 3 as a private environment. Note that the environment version is required and only the almighty environment users are allowed to define an environment as public.
Environment description
Each line of an environment description must follow the pattern: key : value (note that the spaces around the : are mandatory.
Here is an example of an environment description:
name : xen version : 1 description : https://www.grid5000.fr/index.php/Etch-x64-xen-1.0 author : John Smith tarball : /grid5000/etch-x64-xen-1.0.tgz|tgz preinstall : /home/john/test/pre_install.tgz|tgz|launch.sh postinstall : /home/john/test/post_install.tgz|tgz|traitement.sh kernel : /boot/vmlinuz-2.6.18-6-xen-amd64 kernel_params : console=tty0 console=ttyS1,38400n8 initrd : /boot/initrd.img-2.6.18-6-xen-amd64 hypervisor : /boot/xen-3.0.3-1-amd64.gz hypervisor_params : dom0_mem=1000000 fdisktype : 83 filesystem : ext2 environment_kind : xen visibility : shared demolishing_env : 0
Explanation of the fields used in the environment description:
-
name: name of the environment. The spaces are allowed in the name but remember to use some quotes around it when you use Kadeploy or Kaenv. -
version: version of the environment. -
author: author of the environment. -
tarball: disk image of the environment. The syntax is:file|kind. The allowed kinds aretgz,tbz2,ddgzandddbz2. -
preinstall (opt): pre-install file. The syntax is:file|kind|script [params]. The allowed kinds of files aretgzandtbz2. For debug purpose, you can use the keywordbreakpointinstead of a script. Thus, the file will be transfered, the deployment workflow will be stopped and you will be able to connect in the deployment environment to debug. Finally, the script value can be none if no script must be launched. Warning, if thepreinstallfield is fulfilled, the entireSetDeploymentEnvstep defined by the administrator will be bypassed. Refer to the part concerning build of a pre-install. -
postinstall: post-install files. The syntax is:file1|kind|script1 [params],file2|kind|script2 [params], .... The allowed kinds of files for post-installs aretgzandtbz2. For debug purpose, you can use the keywordbreakpointinstead of a script. Thus, the file will be transferred, the deployment workflow will be stopped and you will be able to connect in the deployment environment to debug. Finally, the script value can be none if no script must be launched. -
kernel: path of the kernel in the tarball. -
kernel_params: set of parameters that must be applied to the kernel for a correct boot. -
initrd: path of the initrd in the tarball. -
hypervisor (opt): path of the hypervisor in the tarball. This fields is only required for the Xen based environments. -
hypervisor_params (opt): set of parameters that must be applied to the hypervisor for a correct boot. This fields is only required for the Xen based environments. -
fdisktype: partition type in hexadecimal (83for Linux,a4for FreeBSD, ...). This is used as an input for fdisk. -
filesystem: type of filesystem wished on the deployment partition. It must be known by the mkfs command. -
environment_kind: kind of environment. Expected values are linux, xen or other. -
visibility: define the visibility level of an environment. Three levels are available:- private: only the owner of the environment can see and use it ;
- shared: the environment can be used by everybody but it must explicitly use with the owner name ; furtermore, it won't be listed unless the owner name is specified ;
- public: the environment can be used by everybody and it is listed without specifing its owner name.
-
demolishing_env: specify that the environment is demolishing (expected values are 0 if the environment is not demolishing, 10000 otherwise). SHOULD BE IMPROVED...
Kaconsole
Karights can be used by using the kaconsole command. It has only one use case that is opening a console on a given node, for instance:
> kaconsole3 -m gdx-25.orsay.grid5000.fr
Kaconsole can't be used on a node on which a user doesn't have the deployment rights. Furthermore, as soon as the deployments rights are revoked for a user, ever open console is automatically closed.
Kastat
Kastat can be used by using the kastat3 command. The CLI looks like this:
> kastat3 -h
Usage: kastat3 [options]
Contact: kadeploy3-users@lists.grid5000.fr
General options:
-a, --list-min-retries NB Print the statistics about the nodes that need several attempts
-b, --list-failure-rate Print the failure rate for the nodes
-c, --list-min-failure-rate RATE Print the nodes which have a minimum failure-rate of
RATE (0 <= RATE <= 100
-d, --list-all Print all the information
-f, --field FIELD Only print the given fields (user,hostname,step1,step2,step3,
timeout_step1,timeout_step2,timeout_step3,
retry_step1,retry_step2,retry_step3,
start,
step1_duration,step2_duration,step3_duration,
env,md5,success,error)
-m, --machine MACHINE Only print information about the given machines
-s, --step STEP Applies the retry filter on the given steps (1, 2 or 3)
-v, --version Get the version
-x, --date-min DATE Get the stats from this date (yyyy:mm:dd:hh:mm:ss)
-y, --date-max DATE Get the stats to this date
--server STRING Specify the Kadeploy server to use
We present now the use cases. Note that all the commands can be filtered with a period by using the -x|--date-min and -y|--date-max options.
Use case 1 - get the information about the deployments performed on a node
> kastat3 -d -m gdx-25.orsay.grid5000.fr
This command prints all the deployment performed on the node gdx-25.orsay.grid5000.fr .
Use case 2 - get the information about deployments performed on a range of node
> kastat3 -d -m gdx-[25-130].orsay.grid5000.fr
This command prints all the deployment performed on the nodes gdx-25.orsay.grid5000.fr , gdx-26.orsay.grid5000.fr , ..., gdx-130.orsay.grid5000.fr .
Use case 3 - print only a subset of the information about the deployments performed
> kastat3 -d -f hostname -f env -f success
This command prints all the deployment performed. Because the -f|--field option is used, only the fields hostname , env , and success are printed. If the option -f|--field is not used, all the fields are printed.
Use case 4 - print the failure rate about the nodes wrt the deployments that occurs between two dates
> kastat3 -b -x 2009:02:12:08:00:00 -y 2009:02:13:08:00:00
This command prints the failure rate of all the nodes (at least deployed one time) during the period between the 2009/02/12 - 8h00 and the 2009/02/13 - 8h00. The -x|--date-min and -y|--date-max options can be used separately or can be omitted.
Use case 5 - print the information about the nodes that have at least a given failure rate
> kastat3 -c 25 -x 2009:02:12:08:00:00
This command prints the nodes that have a failure rate of at least 25\% from the 2009/02/12 - 8h00.
Use case 6 - print the information about the nodes that require several retries to deploy correctly
> kastat3 -a 3 -s 1
This command prints the information about the deployments that requires at least 3 retries in the macro-step 1. If the -s|--step option is not set, the information about the deployments that requires at least 3 retries in any macro-step are printed.
Kanodes
Kanodes can be used by using the kanodes3 command. The CLI looks like this:
> kanodes3 -h
Usage: kanodes3 [options]
Contact: kadeploy3-users@lists.grid5000.fr
General options:
-d, --get-deploy-state Get the deploy state of the nodes
-f, --file MACHINELIST Only print information about the given machines (- means stdin)
-m, --machine MACHINE Only print information about the given machines
-v, --version Get the version
-w, --workflow-id WID Specify a workflow id (this is use with the get_yaml_dump
operation. If no wid is specified, the information of all
the running worklfows will be dumped
-y, --get-yaml-dump Get the yaml dump
--server STRING Specify the Kadeploy server to use
We present now the use cases.
Use case 1 - print the deployment state of the nodes
> kanodes3 -d
This command prints the global state of all the nodes managed by a Kadeploy server. The output is as follows 1,2,3,4,5,6, where :
- 1 is the hostname ;
- 2 is the deployment state of the node (prod_env, deployed, deploy_failed, aborted) ;
- 3 is the username who launched the last deployment ;
- 4 is the environment name ;
- 5 is the environment version ;
- 6 is the environment owner.
Use case 2 - print the deployment state of some nodes
> kanodes3 -d -m gdx-25.orsay.grid5000.fr -m netgdx-[1-30].orsay.grid5000.fr -f machine_file
This command prints the global state of the node gdx-25.orsay.grid5000.fr the nodes netgdx-1.orsay.grid5000.fr , netgdx-2.orsay.grid5000.fr , ..., netgdx-30.orsay.grid5000.fr and of the nodes listed in the file machine_file .
Use case 3 - get information about all the current deployment workflows
> kanodes3 -y
This command prints a YAML output of the deployment state of all the nodes currently in deployment. On the YAML output, the nodes are sorted according to the deployment workflow they belong to.
Use case 4 - get information about a specific deployment workflows
> kanodes3 -y -w 78
This command prints a YAML output of the deployment state of all the nodes currently in the deployment number 78. The deployment number, or workflow id, can be obtained thanks to a Kadeploy option.
Kapower
Kapower can be used by using the kapower3 command. The CLI looks like this:
> kapower3 -h
Usage: kapower3 [options]
Contact: kadeploy3-users@lists.gforge.inria.fr
General options:
-d, --debug-mode Activate the debug mode
-f, --file MACHINELIST Files containing list of nodes (- means stdin)
-l, --level VALUE Level (soft, hard, very_hard)
-m, --machine MACHINE Operate on the given machines
--multi-server Activate the multi-server mode
-n, --output-ko-nodes FILENAME File that will contain the nodes on which
the operation has not been correctly performed
-o, --output-ok-nodes FILENAME File that will contain the nodes on which the
operation has been correctly performed
--off Shutdown the nodes
--on Power on the nodes
--status Get the status of the nodes
-v, --version Get the version
--no-wait Do not wait the end of the power operation
--server STRING Specify the Kadeploy server to use
-V, --verbose-level VALUE Verbose level between 0 to 4
Use case 1 - print the power status of some nodes
> kapower3 --status -m gdx-[25-35].orsay.grid5000.fr -o nodes_up -n nodes_down
This command print the power status of the nodes gdx-25.orsay.grid5000.fr to gdx-35.orsay.grid5000.fr . Furthermore, the list of the powered up nodes is stored in nodes_up and the list of the powered off nodes is stored in nodes_down .
Use case 2 - power off some nodes
> kapower3 --off -f machine_file --server lille
This command powers off the nodes nodes contained in the machine_file file. Since the --server is used, the nodes of a distant site are concerned by the operation ; in this example, the lille site is concerned.
Use case 3 - power on some nodes
> kapower3 --on -m gdx-25.orsay.grid5000.fr --no-wait
This command powers on the node gdx-25.orsay.grid5000.fr without waiting the end of the operation to return.
Note for the administrators
While using the --no-wait option, Kapower add the --no-wait value to the command line used for the given operation. Thus the underlying power management tool must be able to handle this option if this Kapower option is required.
Karights
Karights can be used by using the karights3 command (it is designed for administrators in order to allow users to perform deployments). The CLI looks like this:
> karights3 -h
Usage: karights3 [options]
Contact: kadeploy3-users@lists.grid5000.fr
General options:
-a, --add Add some rights to a user
-d, --delete Delete some rights to a user
-f, --file FILE Machine file (- means stdin)
-m, --machine MACHINE Include the machine in the operation
-o, --overwrite-rights Overwrite existing rights
-p, --part PARTNAME Include the partition in the operation
-s, --show-rights Show the rights for a given user
-u, --user USERNAME Specify the user
-v, --version Get the version
--server STRING Specify the Kadeploy server to use
We present now the use cases.
Use case 1 - give some rights to a user on a node
> karights3 -a -m gdx-25.orsay.grid5000.fr -p /dev/sda3 -u johnsmith
This command gives some rights for a given user.
Use case 1 - give some rights to a user on several nodes
> karights3 -a -m gdx-[25-32].orsay.grid5000.fr -p /dev/sda3 -u johnsmith
This command gives some rights for a given user on a range of nodes.
Use case 3 - give all the rights to a user on all the nodes
> karights3 -a -m "*" -p "*" -u root
This command gives all the rights on all the nodes to the user root .
Use case 4 - give some rights on a node and remove existing ones
> karights3 -a -m gdx-25.orsay.grid5000.fr -p /dev/sda3 -u johnsmith -o
This command gives some rights for a given user. Furthermore, if some rights (excepted those specified with *) were previously given on the node gdx-25.orsay.grid5000.fr , they are deleted.
Use case 5 - remove som rights
> karights3 -d -m gdx-25.orsay.grid5000.fr -p /dev/sda3 -u johnsmith
This command removes some rights for a given user.
Use case 6 - show the rights of a user
> karights3 -s -u johnsmith
This command shows the rights given to user.
What you should know if you want to do kernel development on deployed nodes
Kernel development implies to know what Kadeploy do concerning the boot of the deployed environments.
Kadeploy 3 behavior
Kadeploy 3 has a different behavior depending on the kind of deployed environment. Reminder: the kind of environment is defined in the environment description.
Linux environments
On a Linux environment, Kadeploy 3 automatically installs the Grub 2 bootloader on the deployed partition once the tarball is broadcasted. Then it modifies the PXE profile of the concerned nodes in order to ask the chainload on the deployed partition. This is performed thanks to pxelinux and the comboot chain.c32.
Xen environments
On a Xen environment, Kadeploy 3 doesn't install the Grub 2 bootloader since Grub 2 there are some known issues when booting a Xen Dom0 with Grub 2. Thus Kadeploy 3 uses the old method that consists in booting the nodes in a pure PXE fashion. To do that, Kadeploy extracts the kernel, initrd and hypervisor files from the environment tarball and modifies the PXE profile of the concerned nodes in order to ask their in pure PXE. This is performed thanks to pxelinux and the comboot mboot.c32.
Other environments
On an Other environment, Kadeploy 3 assumes that a bootloader is already installed on the partition since a full partition image (dd.gz image) has been copied. Thus, it only modifies the PXE profile of the concerned nodes in order to ask the chainload on the deployed partition, like in the Linux case.
Tips to simply use your new kernel
If you do kernel development on the deployed nodes, you will probably want to update you kernel without recording a new image and redeploying it to save time, especially to perform small tests.
Linux environments
On a Linux environment, after having updated your kernel/initrd, 2 cases are imaginable:
your kernel/initrd have the same name, so you can reboot the node without modifying anything.
your kernel/initrd have a new name, so you will have to update the grub configuration file (/boot/grub/grub.cfg ) of your node in order to allow grub to select the new kernel and then you can reboot the node.
Xen environments
On a Xen environment, the things are a little bit more complicated. As far as the kernel/initrd/hypervisor are extracted by Kadeploy in a dedicated cache, changing them on the deployed nodes won't have any effect for the next reboot. So you have to use a feature of Kareboot that allows to reboot a node after having changed the PXE profile of the node. For instance:
> kareboot3 -m gdx-25.orsay.grid5000.fr -r set_pxe -w ~/pxe_profile_xen \
-x "~/custom_kernel,~/custom_initrd,~/custom_hypervisor"
Kadeploy has the same feature, so please refer to the use case about specific PXE profile for more information.
Other environments
On an Other environment, you eventually have to update your bootloader in order to boot on the new kernel.
Migrate your Kadeploy 2.1.x environment
Migrating your environments from Kadeploy 2.1.x is quite easy.
Differences
The fields used in the previous descriptions are similar. This following fields are exactly the same: name , version , description , author , fdisktype , and filesystem . You just have to replace the = character with : for these fields.
Then, the old filebase and filesite have been modified and renamed in tarball and postinstall .
Kadeploy 3 considers three kinds of environments: linux , xen , and other . The environment kind must be specified with the field environment_kind .
- If you choose the linux kind, you can fill the new
kernelfield by using the oldkernelpathfield. This is the same thing with the newinitrdfield where you can use the oldinitrdpath. The newkernel_paramsfield can be filled with the oldkernelparam. - If you choose the xen , you must use the fields of a linux environment and the new
hypervisorandhypervisor_params. You do not have to specify anymore the use of mboot.c32, this is automatically handled by Kadeploy 3. - If you choose the other kind (typically for OS like FreeBSD), you can fill the
kernel,kernel_paramsandinitrdwith any value.
The field part has been added to allow the users to specify a default deployment partition for their environment. The expected value must be an existing block device like /dev/sda3 .
Automatic translation
A small script named env_migrate has been written to ease the translation of the old Kadeploy environment. The CLI looks like this:
> kaenv2to3 -h Usage: kaenv2to3 [options] Contact: kadeploy3-users@lists.grid5000.fr General options: -e, --env-name NAME Environment name -f, --env-file FILE Environment file -k, --env-kind KIND Environment kind (linux|xen|other)
To use kaenv2to3 , you have to choose the kind of environment you wish to migrate and you have to choose the environment. The choice can be made in two ways:
- choosing the environment from its name in the old version of Kadeploy (like it can be seen with the kaenvironments output) ;
- choosing the environment from a file that contains the description in the old format.
Here are two examples:
> kaenv2to3 -e envname_in_kaenvironments_db -k linux > kaenv2to3 -f envfile_in_2_1_x_format.dsc -k xen
If everything went right, the output of kaenv2to3 shows the description in the new format. You can redirect this output to a file and use this file as an input for kaenv .
Environment variables in the pre- and post-install context
When writing a script for an admin pre-install, an admin post-install and a user post-install, you can use the following environment variables :
-
KADEPLOY_CLUSTER: cluster on which the pre/post install is launched -
KADEPLOY_ENV: environment deployed -
KADEPLOY_DEPLOY_PART: deployment partition -
KADEPLOY_ENV_EXTRACTION_DIR: path where the environment tarball is extracted -
KADEPLOY_PREPOST_EXTRACTION_DIR: path where the pre/post tarball are extracted
Build a custom pre-install
The goal of the pre-install in the Kadeploy workflow is to prepare the disk of the nodes before the copy of the environment. It can include:
- setting disk parameters (with hdparm for instance) ;
- partitioning the disk (with fdisk or parted) ;
- formating the deployment and the
/tmppartition ; - mounting partition(s).
You can do want you want in the pre-install but you must that Kadeploy will extract the environment in the directory defined by the environment_extraction_dir field of the general configuration file. Commonly, this directory is /mnt/dest . Thus, you have to mount all the partitions you need in this directory. If you wish to deploy the environment onto several partitions, you can use for instance the following map:
- /dev/sda3 --> /mnt/dest
- /dev/sda4 --> /mnt/dest/var
- /dev/sda5 --> /mnt/dest/usr
- /dev/sda6 --> /mnt/dest/tmp
If you choose to mount more than one partition in the pre-install, remember to umount all the partitions excepted the one mounted on environment_extraction_dir (/mnt/dest in principle) in the post-install step. Indeed, the common Kadeploy workflow will automatically umount the partition mounted on environment_extraction_dir . Thus, if other partitions are mounted, the umount will fail.
