ISO to kadeploy image

From Grid5000
Revision as of 20:25, 31 January 2012 by Sbadia (Talk | contribs)

Jump to: navigation, search


HowTo Create an kadeploy suitable image from an ISO file.

Contents

Other documentations

Debian

  • Same as ubuntu :)

Ubuntu

Note.png Note

Available on nancy and other (kaenv3 -p ubuntu-x64 -u sbadia)

Preparation

  • Make an OAR reservation (if possible 2h mini)
Terminal.png frontend:
oarsub -I -t deploy -l nodes=1,walltime=2 -n "ubuntu install"
  • Deploy Squeeze base image
Terminal.png frontend:
kadeploy3 -f $OAR_NODEFILE -e squeeze-x64-base
  • Download an Ubuntu ISO on you laptop, and upload it on the fresh deployed node.
Terminal.png frontend:
scp ubuntu-11.10-amd64.iso root@chinqchint-11:
  • Install qemu (necessary for install vm with ISO)
Terminal.png node:
apt-get update && apt-get install qemu
  • Umount /tmp (sda5 partition), this partition is needed for host installation
Terminal.png node:
umount /tmp

Boot vm from ISO file

  • In another terminal launch an connection to the node with X forward
Terminal.png frontend:
ssh -X root@chinqchint-11
  • And boot ISO
    • For 32bits image use simply qemu
Terminal.png node:
qemu -hda /dev/sda -cdrom ubuntu-11.10-i386.iso -boot d -k fr
    • For 64bits image use qemu-system-x86_64 (Recommended)
Terminal.png node:
qemu-system-x86_64 -hda /dev/sda -cdrom ubuntu-11.10-amd64.iso -boot d -k fr

Installation

  1. Basique ubuntu installation (if possible minimal).
  2. Skip network configuration.
  3. On taskel select base system and ssh server
  4. Remember proxy config (http://proxy:3128)
  5. Format and install distribution on sda5 partition, all in /, and with ext3 filesystem
  6. When is finish, shutdown vm (CTRL+C in qemu teminal).

Create your g5k image

Post Configurations

  • Some steps are needed
Terminal.png node:
mount /dev/sda5 /mnt/
  • Create a script in dhclient-exit-hook for set the hostname dynamicaly
cat << EOF > /mnt/etc/dhcp/dhclient-exit-hooks.d/g5k-hostname
EOF
Terminal.png node:
cat /mnt/etc/dhcp/dhclient-exit-hooks.d/g5k-hostname
#!/bin/sh

if [ -n "$new_host_name" ]; then
	if [ -n "$new_domain_name" ]; then
		hostname="${new_host_name}.${new_domain_name}"
	else
		hostname="${new_host_name}"
	fi

	echo "$hostname" > /etc/hostname 2> /dev/null
	hostname "$hostname"
	echo "$hostname" > /etc/mailname 2> /dev/null
fi
  • And make it executable
Terminal.png node:
chmod +x /mnt/etc/dhcp/dhclient-exit-hooks.d/g5k-hostname
  • Remove fstab file (this file is installed by adminpostinstall (sed -e'/sda5/sda3/' -i /mnt/etc/fstab)
Terminal.png node:
rm -r /mnt/etc/fstab
  • Remove also interfaces file, set by admin postinstall.
Terminal.png node:
rm -r /mnt/etc/network/interfaces

Image and registration

  • Finaly everythings is ok, make your tgz image.
Terminal.png frontend:
ssh root@chinqchint-11 "cd /mnt; tar --posix --numeric-owner --one-file-system -zcf - *" > ~/public/ubuntu-x64.tgz
  • Copy description file of squeeze-x64-base
Terminal.png frontend:
kaenv3 -p squeeze-x64-base -u deploy > ~/ubuntu-x64.dsc
  • Modify description
###
name : ubuntu-x64
version : 2
description : Ubuntu Oneiric 11.10
author : sebastien.badia@inria.fr
tarball : http://public.nancy.grid5000.fr/~sbadia/ubuntu-oneiric.tgz|tgz
postinstall : /grid5000/postinstalls/debian-x64-min-1.0-post.tgz|tgz|traitement.ash /rambin
kernel : /boot/vmlinuz-3.0.0-12-server
initrd : /boot/initrd.img-3.0.0-12-server
fdisktype : 83
filesystem : ext3
environment_kind : linux
visibility : shared
demolishing_env : 0
Note.png Note

Visibility tag determine visiblity for other users

  • Private: Just for you
  • Shared: You and other users (kaven3 -l -u sbadia)
  • Public: Only for reference/prod images
  • So save it on kadeploy database
Terminal.png frontend:
kaenv3 -a ubuntu-x64.dsc

Test it

  • Deploy your fresh image with debug mode (in case of :D)
Terminal.png node:
kadeploy3 -f $OAR_NODEFILE -e ubuntu-x64 -k ~/.ssh/id_dsa.pub -d -V4

Scientific Linux

Note.png Note

Available on nancy and other (kaenv3 -p sl55-x64 -u sbadia)

Preparation

  • Make an OAR reservation (if possible 2h mini)
Terminal.png frontend:
oarsub -I -t deploy -l nodes=1,walltime=2 -n "scientificlinux install"
  • Deploy Squeeze base image
Terminal.png frontend:
kadeploy3 -f $OAR_NODEFILE -e squeeze-x64-base
  • Download an Scientific Linux ISO on you laptop, and upload it on the fresh deployed node.
Terminal.png frontend:
scp scientific-linux-5.5-x86_64.iso root@graphene-45:
  • Install qemu (necessary for install vm with ISO)
Terminal.png node:
apt-get update && apt-get install qemu
  • Umount /tmp (sda5 partition), this partition is needed for host installation
Terminal.png node:
umount /tmp

Boot vm from ISO file

  • In another terminal launch an connection to the node with X forward
Terminal.png frontend:
ssh -X root@graphene-45
  • And boot ISO
Terminal.png node:
qemu-system-x86_64 -hda /dev/sda -cdrom scientific-linux-5.5-x86_64.iso -boot d -k fr

Installation

  1. Minimal install, witch ssh server
  2. No SeLinux, firewall desactivated
  3. Proxy config (http://proxy:3128)
  4. Format and install distribution on sda5 partition, all in /, and with ext3 filesystem
  5. When is finish, shutdown vm (CTRL+C in qemu teminal).

Create your g5k image

Post Configurations

  • Some steps are needed
Terminal.png node:
mount /dev/sda5 /mnt/
  • Create a script in dhclient-exit-hook for set the hostname dynamicaly
cat << EOF > /mnt/etc/dhcp/dhclient-exit-hooks.d/g5k-hostname
EOF
Terminal.png node:
cat /mnt/etc/dhcp/dhclient-exit-hooks.d/g5k-hostname
#!/bin/sh

if [ -n "$new_host_name" ]; then
	if [ -n "$new_domain_name" ]; then
		hostname="${new_host_name}.${new_domain_name}"
	else
		hostname="${new_host_name}"
	fi

	echo "$hostname" > /etc/hostname 2> /dev/null
	hostname "$hostname"
	echo "$hostname" > /etc/mailname 2> /dev/null
fi
  • And make it executable
Terminal.png node:
chmod +x /mnt/etc/dhcp/dhclient-exit-hooks.d/g5k-hostname
  • Remove fstab file (this file is installed by adminpostinstall (sed -e'/sda5/sda3/' -i /mnt/etc/fstab)
Terminal.png node:
rm -r /mnt/etc/fstab
  • Remove also interfaces file, set by admin postinstall.
Terminal.png node:
rm -r /mnt/etc/network/interfaces
Todo.png Todo

Yum and inittab for redhat

$ cat dest/etc/yum.conf 
[main]
cachedir=/var/cache/yum
keepcache=1
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=0
plugins=1
metadata_expire=72000
<[PROXY_AVAILABILITY]>proxy=http://proxy.<[SITE]>.grid5000.fr:3128/

# Default for installonlyn.  
# Uncomment and set to 0 to turn off
# installonly_limit = 4

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d

Image and registration

  • Finaly everythings is ok, make your tgz image.
Terminal.png frontend:
ssh root@graphene-45 "cd /mnt; tar --posix --numeric-owner --one-file-system -zcf - *" > ~/public/sl55-x64.tgz
  • Copy description file of squeeze-x64-base
Terminal.png frontend:
kaenv3 -p squeeze-x64-base -u deploy > ~/sl55-x64.dsc
  • Modify description
###
name : sl55-x64
version : 6
description : http://www.scientificlinux.org/distributions/5x/55/
author : sebastien.badia@inria.fr
tarball : http://public.nancy.grid5000.fr/~sbadia/scientific-linux-55-x64.tgz|tgz
postinstall : http://public.nancy.grid5000.fr/~sbadia/sl56-post.tgz|tgz|traitement.ash /rambin
kernel : /vmlinuz
initrd : /initrd.img
fdisktype : 83
filesystem : ext3
environment_kind : linux
visibility : shared
demolishing_env : 0
  • So save it on kadeploy database
Terminal.png frontend:
kaenv3 -a sl55-x64.dsc

Test it

  • Deploy your fresh image with debug mode (in case of :D)
Terminal.png node:
kadeploy3 -f $OAR_NODEFILE -e sl55-x64 -k ~/.ssh/id_dsa.pub -d -V4

ArchLinux

Note.png Note

Available on nancy and other (kaenv3 -p arch-x64 -u sbadia)

Preparation

  • Make an OAR reservation (if possible 2h mini)
Terminal.png frontend:
oarsub -I -t deploy -l nodes=1,walltime=2 -n "arch install"
  • Deploy Squeeze base image
Terminal.png frontend:
kadeploy3 -f $OAR_NODEFILE -e squeeze-x64-base
  • Download an Arch Linux ISO on you laptop, and upload it on the fresh deployed node.
Terminal.png frontend:
scp archlinux-2011.08.19-core-x86_64.iso root@graphene-56:
  • Install qemu (necessary for install vm with ISO)
Terminal.png node:
apt-get update && apt-get install qemu
  • Umount /tmp (sda5 partition), this partition is needed for host installation
Terminal.png node:
umount /tmp

Boot vm from ISO file

  • In another terminal launch an connection to the node with X forward
Terminal.png frontend:
ssh -X root@chinqchint-11
  • And boot ISO
Terminal.png node:
qemu-system-x86_64 -hda /dev/sda -cdrom archlinux-2011.08.19-core-x86_64.iso -boot d -k fr

Installation

  1. km (set your keyboard layout)
  2. /arch/setup (install arch)
  3. setup medium to local cd
  4. cfdisk -> install on sda5 (ext3), no swap
  5. install grub
  6. install base system and ssh server
  7. Configure sshd at startup, and dhcp on eth0 (rc.conf -> DAEMONS=(hwclock syslog-ng network netfs crond sshd))

Image and registration

  • Finaly everythings is ok, make your tgz image.
Terminal.png frontend:
ssh root@graphene-56 "cd /mnt; tar --posix --numeric-owner --one-file-system -zcf - *" > ~/public/arch-x64.tgz
  • Copy description file of squeeze-x64-base
Terminal.png frontend:
kaenv3 -p squeeze-x64-base -u deploy > ~/arch-x64.dsc
  • Modify description
Todo.png Todo

Copy dsc here, postinstall also


  • So save it on kadeploy database
Terminal.png frontend:
kaenv3 -a arch-x64.dsc

Test it

  • Deploy your fresh image with debug mode (in case of :D)
Terminal.png node:
kadeploy3 -f $OAR_NODEFILE -e arch-x64 -k ~/.ssh/id_dsa.pub -d -V4

Debian kfreeBSD

Note.png Note

Available on nancy and other (kaenv3 -p debkfreebsd-x64 -u sbadia)

Preparation

  • Make an OAR reservation (if possible 2h mini)
Terminal.png frontend:
oarsub -I -t deploy -l nodes=1,walltime=2 -n "debkfreebsd install"
  • Deploy Squeeze base image
Terminal.png frontend:
kadeploy3 -f $OAR_NODEFILE -e squeeze-x64-base
  • Download an GNU Debian/kfreeBSD ISO on you laptop, and upload it on the fresh deployed node.
Terminal.png frontend:
scp debian-kfreebsd-amd64.iso root@griffon-13:
  • Install qemu (necessary for install vm with ISO)
Terminal.png node:
apt-get update && apt-get install qemu
  • Umount /tmp (sda5 partition), this partition is needed for host installation
Terminal.png node:
umount /tmp

Boot vm from ISO file

  • In another terminal launch an connection to the node with X forward
Terminal.png frontend:
ssh -X root@griffon-13
  • And boot ISO
Terminal.png node:
qemu-system-x86_64 -hda /dev/sda -cdrom debian-kfreebsd-amd64.iso -boot d -k fr

Installation

  1. Note network is iface em0
  2. Setup the proxy for installation.
  3. Change the fstype of <sda3> partition to a5 (initial linux 82).
  4. Install debian kfreebsd.
  5. Reboot on <sda3> partiton, bsd <ad4s3>.
  6. Install grub on the hd0,3.

Image and registration

  • Reboot node and install grub
Terminal.png frontend:
kareboot3 -l hard -r simple_reboot -m griffon-13.nancy.grid5000.fr
Terminal.png node:
grub-setup --skip-fs-probe --force '(hd0,3)'
  • Finaly everythings is ok, make your tgz image.
Terminal.png frontend:
ssh root@griffon-13 "dd if=/dev/sda3 | gzip" | pv > ~/public/debian-kfreebsd.dd.gz
  • Copy description file of squeeze-x64-base
Terminal.png frontend:
kaenv3 -p squeeze-x64-base -u deploy > ~/debkfreebsd-x64.dsc
  • Modify description
###
name : debkfreebsd-x64
version : 2
description : GNU/Debian kFreeBSD squeeze 6.0.3
author : sebastien.badia@inria.fr
tarball : http://public.nancy.grid5000.fr/~sbadia/debian-kfreebsd.dd.gz|ddgz
postinstall : http://public.nancy.grid5000.fr/~sbadia/kfreebsd-1.0.tgz|tgz|traitement.ash /rambin
kernel : chainload
initrd : foo
fdisktype : a5
filesystem : ext2
environment_kind : other
visibility : shared
demolishing_env : 1
  • So save it on kadeploy database
Terminal.png frontend:
kaenv3 -a debkfreebsd-x64.dsc

Test it

  • Deploy your fresh image on another node with debug mode (in case of :D)
Terminal.png node:
kadeploy3 -f $OAR_NODEFILE -e debkfreebsd-x64 -k ~/.ssh/id_dsa.pub -d -V4
Terminal.png node:
uname -a
GNU/kFreeBSD griffon-14.nancy.grid5000.fr 8.1-1-amd64 #0 Tue Aug 17 23:12:29 CEST 2011 x86_64 amd64 Intel(R) Xeon(R) CPU           L5420  @ 2.50GHz GNU/kFreeBSD

FreeBSD

Tips

Some usefull tips

make a change in g5k deployment kernel

mkdir /tmp/deploy
  • Decompress initrd
gzip -dc /var/lib/tftpboot/kernels/deploy-initrd-2.6.32.1-big > /tmp/deploy.ext2.img
  • Mount
mount -o loop -t ext2 /tmp/deploy.ext2.img /tmp/deploy
  • Make your changes :)
  • And recreate inirtd
cat deploy.ext2.img | gzip -9 -c > initrd.img

Create a custom initrd

Note.png Note

Grid'5000 standard partition map

  • /etc/fstab.g5k
    • sda1: swap (4Go)
    • sda2: prod (6Go, ext3)
    • sda3: deploy (6Go, ext3)
    • sda5: tmp (Rest of disk, ext3)

mkinitrd -f --fstab=/etc/fstab.g5k --omit-lvm-modules /boot/initrd.g5k.img 2.6.9-55.EL \
--with=mptsas --with=mptspi --with=sata_nv --with=tg3 --with=sata_sil --with=sata_svw \
--with=ata_piix --with=e1000 --with=megaraid_sas  --with=mptscsih --with=cciss \
--with=libata --with=sd_mod --with=scsi_mod --with=bnx2 --with=scsi_transport_spi

Customise an initrd (redhat)

  • Decompress actual initrd
cd /root
mkdir initrd
cd initrd
echo "--> utilisation de l'initrd $(uname -r)"
gzip -dc /boot/initrd-$(uname -r).img | cpio -id --no-absolute-filenames
  • Modify your init (in my case, add some disk_drv)
  • And recompress
find . | cpio --dereference -H newc -o | gzip -9 > /boot/initrd-$(uname -r).img

Recover your deploy session

  • re-make your réservation with the deploy type.
oarsub -I -t deploy -l nodes=1,walltime=9 -p "host='griffon-$x.nancy.grid5000.fr'"
  • and kareboot the node to the deploy part.
kareboot3 -l hard -p 3 -r env_recorded -e debian-x64-5-prod -m griffon-$x.nancy.grid5000.fr
Personal tools
Namespaces

Variants
Actions
Public Portal
Users Portal
Admin portal
Wiki special pages
Toolbox