Armored Node for Sensitive Data

	Note
	This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

This page documents how to secure a Grid'5000 node, making it suitable to host and process sensitive data. The process is based on a tool (g5k-armor-node.py) that runs on top of the debian10-x64-big Grid'5000 environment.

Node reservation and deployment

Identify your requirements:

• Select a cluster that suits your needs (for example using the Hardware page).
• Estimate for how long you will need the resources. If they exceed what is allowed for the default queue in the Usage Policy, maybe the production queue will match your needs. If the duration also exceeds what is allowed by the production queue (more than one week), you can follow the procedure explained on the Usage Policy page to request an exception.
• Take into consideration that all data (including data you produced) stored locally on the machine will be destroyed at the end of the reservation.
• Reserve a node and a VLAN, then deploy the node with the debian10-x64-big environment inside the VLAN (see detailed steps below).

Detailed steps for reservation and deployment

Reserve the node and the VLAN. Example for a reservation in the production queue for one node of cluster CLUSTER starting at START DATE for a duration of WALLTIME:

nancy frontend:oarsub -q production -t deploy -t destructive -l {"type='kavlan'"}/vlan=1+{"cluster='CLUSTER'"}/nodes=1,walltime=WALLTIME -r START DATE

FIXME: mention reserving additional disks

Once the job has started, connect inside the job:

frontend:oarsub -C JOB ID

Note that since it is a deploy job, the job shell opens on the frontend again.

Take note of the hostname of the reserved node for instance with oarprint:

frontend:oarprint host

Take note of the assigned VLAN number:

frontend:kavlan -V

Deploy the node with the debian10-x64-big environment, inside the VLAN:

frontend:kadeploy3 -e debian10-x64-big -f $OAR_NODEFILE --vlan `kavlan -V` -k

Now wait for the deployment to complete.

Securing the node with g5k-armor-node.py

Connect to the node from the outside of Grid'5000, using the node's hostname in the VLAN (hostname with the Kavlan suffix for the reserved VLAN, because the node was deployed inside the kavlan VLAN). After securing the node, this will be the only allowed way to connect to the node, as SSH will only be authorized from Grid'5000 access machines:

your machine:ssh -J YOUR_G5K_LOGIN@access.grid5000.fr root@node-X-kavlan-Y.site.grid5000.fr

On the node, download g5k-armor-node.py, for example with:

node:wget https://gitlab.inria.fr/grid5000/g5k-armor/-/raw/master/g5k-armor-node.py

Run it:

node:chmod a+rx g5k-armor-node.py
node:./g5k-armor-node.py

After the script finishes, disconnect from the node, and try to connect again using SSH. You should get an error message from SSH, because the node's host key changed. This is expected: the script replaced the node's SSH host key with a newly generated one. Follow the instructions to remove the old key.

Using the secured node

You must connect to the node using your Grid'5000 login directly from your workstation:

your machine:ssh -J YOUR_G5K_LOGIN@access.grid5000.fr YOUR_G5K_LOGIN@node-X-kavlan-Y.site.grid5000.fr

The node can access the Internet and you can use the sudo command on the node to install additional software if needed.

Please remember that:

• Only your home directory is encrypted (/home/<username>). You must not store sensitive data outside of it (or on other Grid'5000 machines).
• You must only use secured protocols to transfer data to/from the node as described below.
• If you reboot the node or if the node is shut down for some reason, you will no longer be able to access your data. However, if you made a copy of the encryption key when it was displayed at the end of the script's output, you can restore the encrypted storage from the node with:

echo '<paste key content here>' > /run/user/1000/key
sudo cryptsetup luksOpen --key-file /run/user/1000/key /dev/sda5 encrypted
sudo mount /dev/mapper/encrypted $HOME
exit

Then reconnect to the node.

If you prefer to avoid keeping a copy of the encryption key, it is a good idea to make intermediary backups of the processed data, in case the secured node becomes unreachable during the processing.

Transferring data to/from the node

You must transfer data directly between an external secure storage, and your Grid'5000 node. You must not use other Grid'5000 storage spaces (such as NFS spaces) in the process.

It is recommended to use rsync. Using rsync, you can specify access.grid5000.fr as a SSH JumpHost using the -e option. Alternatively, you can customize your SSH configuration as described in the Getting Started tutorial.

• To transfer files to the node:

rsync -e "ssh -J YOUR_G5K_LOGIN@access.grid5000.fr" <local path> YOUR_G5K_LOGIN@node-X-kavlan-Y.site.grid5000.fr:<remote path>

• To fetch files from the node:

rsync -e "ssh -J YOUR_G5K_LOGIN@access.grid5000.fr" YOUR_G5K_LOGIN@node-X-kavlan-Y.site.grid5000.fr:<remote path> <local path>