KaVLAN

From Grid5000
Jump to navigation Jump to search


Overview

The goal of Kavlan is to allow people to manage VLAN on grid5000 nodes. The benefits is complete level 2 isolation. It can be used together with OAR and Kadeploy to do some experimentations on the grid.

The following figure shows two jobs running with KaVLAN: each job has it's nodes isolated in a VLAN (purple and green). The other nodes are all in the default VLAN (red). The only way to reach the isolated nodes is to use a gateway node (kavlan-1 and kavlan-2 in the figure). The gateway has two Ethernet interfaces: one in the default VLAN and one is the dedicated VLAN. This way, you can use ssh to reach your nodes (an other way to reach an isolated node is to use the kaconsole command).

KaVLAN architecture: 2 jobs running KaVLAN


Note.png Note

The gateways are NOT doing any routing: they are only used as ssh gateways.

Usage

How to reserve a VLAN

KaVLAN only works with deploy reservations; to obtain nodes and a VLAN, simply add the -t kavlan option to oarsub. For example, if you need 3 nodes and a VLAN:

oarsub -t kavlan -t kadeploy -l /nodes=3 -I

Then you can get the id of your VLAN using the kavlan command

kavlan -V

If you run this command outside the shell started by OAR for your reservation, you must add the oar JOBID:

kavlan -V -j JOBID

You should get an integer in the <1-8> range.

You can get all the options of the command using --help:

# kavlan --help
Version 1.0rc2
USAGE : kavlan [options]
      -r|--get-network-range
      -g|--get-network-gateway
      -l|--get-nodelist
      -V|--get-vlan-id              print VLAN ID of job (needs -j JOBID)
      -d|--disable-dhcp
      -e|--enable-dhcp
      -i|--vlan_id <VLANID>
      -s                            set vlan for given node(s)
      -f|--filenode <NODEFILE>
      -j|--oar-jobid=<JOBID>
      -m|--machine <nodename>
      -q|--quiet                    quiet mode
      -h|--help                     print this help
      -v|--verbose                  verbose mode

Once you have a kavlan reservation running, you are allowed to connect to the VLAN gateway named kavlan-<ID> where ID is your vlan ID, and you can also put your nodes in your VLAN (and back into the default VLAN) at anytime during the lifetime of your job.

Since KaVLAN works only with deploy jobs, the next step is to deploy at least one node (otherwise, you won't have root acces on it and therefore can't restart it's network configuration).

Let's say you want to deploy all nodes:

kadeploy -f $OAR_NODEFILE -e <myenvironment>

Enable/disable the dhcp server of the gateway

Once the deployment is over, you are now able to change the VLAN of your nodes. First check that the DHCP server is running on the gateway, run on the frontend (add -j JOBID if needed) :

kavlan -e 

You can disable the DHCP server with kavlan -d

Change the VLAN of your nodes

# uniq $OAR_NODEFILE > ./mynodes
# taktuk -l root -f ./mynodes broadcast exec [ "echo '/etc/init.d/networking restart'| at now + 1 minute " ]
# kavlan -s
Take node list from OAR nodefile: /var/lib/oar/387465
 ... node azur-25.sophia.grid5000.fr changed to vlan KAVLAN-7
 ... node azur-28.sophia.grid5000.fr changed to vlan KAVLAN-7
 ... node azur-30.sophia.grid5000.fr changed to vlan KAVLAN-7
all nodes are configured in the vlan 7

Put your nodes back into the default VLAN

kavlan -s -i DEFAULT -f $OAR_NODEFILE

Configure ssh to easily connect to nodes in a VLAN

In order to transparently use ssh to acces to isolated nodes, you should this to your .ssh/config file:

Host *-*-kavlan-1
   ProxyCommand ssh -q -a -x kavlan-1 nc %h %p
Host *-*-kavlan-2
   ProxyCommand ssh -q -a -x kavlan-2 nc %h %p
Host *-*-kavlan-3
   ProxyCommand ssh -q -a -x kavlan-3 nc %h %p
Host *-*-kavlan-4
   ProxyCommand ssh -q -a -x kavlan-4 nc %h %p
Host *-*-kavlan-5
   ProxyCommand ssh -q -a -x kavlan-5 nc %h %p
Host *-*-kavlan-6
   ProxyCommand ssh -q -a -x kavlan-6 nc %h %p
Host *-*-kavlan-7
   ProxyCommand ssh -q -a -x kavlan-7 nc %h %p
Host *-*-kavlan-8
   ProxyCommand ssh -q -a -x kavlan-8 nc %h %p

Then you can simply use ssh <cluster>-<nodeid>-kavlan-<vlanid> to access the node (azur-2-kavlan-7 for ex.)

Advance usage

Setup a DHCP server on your nodes