Storage5k

See also: Storage overview | Storage Manager | Group Storage | Ceph | Local (on nodes) disks reservation

Grid'5000 storage resources

Sites	VM Name	Disk partition	Size	status
Sophia	stock.sophia.grid5000.fr	/dev/sda	2TB
Grenoble
Lille
Lyon	mizar.lyon.grid5000.fr		20TB
Luxembourg	hera.luxembourg.grid5000.fr		1.80TB
Nancy	fgriffon2.nancy.grid5000.fr		2.14TB
Rennes	srv-bigdata.rennes.grid5000.fr		6TB

Storage5k

Overview

Storage5k is a tool for the reservation of large storage, providing much more space than the one available in a user's /home. Contrary to other resource reservations in Grid'5000, the duration of a storage5k reservation is not limited, allowing data to be used in long term experiments.

The data reserved by storage5k is directly accessible on Grid'5000 nodes, using NFS or iSCSI mounts.

	Warning
	Storage5k is an experimental tool and data integrity is not guaranteed. You MUST backup your important data.

See the Storage5k page to check on which sites Storage5k is available.

Usage

The Storage5k tool is available on Grid'5000 frontends. Check if the storage5k command is installed with:

frontend: storage5k -v

An individual Storage5k resource is called a "chunk", and represents the smallest allocatable unit of storage. A typical chunk size is 10GB but it may vary among Grid'5000 sites. To display the chunks size, use the following command:

frontend: storage5k -a chunk_size

Let's say that you need 50GB of space during one day to carry out your experiment. To reserve this storage, use:

frontend: storage5k -a add -l chunks=number,walltime=24

You can get information on your reserved storage with:

frontend: storage5k -a info

Note the Job_Id field. Actually, a Storage5k reservation is an OAR job ! You can use any of the OAR tools, such as:

frontend: oarstat -f -j storage_job_id

An other important field is Source nfs. It displays the NFS mount point where your storage is exported. Note that your storage is already available from the frontend:

frontend: ls /data/username_storage_job_id

frontend: cp my_big_data /data/username_storage_job_id/

Nodes you reserve will automatically have access to the storage. For instance, try this:

frontend: oarsub -l nodes=3 -I

node: ls /data/username_storage_job_id/
lost+found my_big_data

node: oarsh othernode

othernode: ls /data/username_storage_job_id/
lost+found my_big_data

Though there is an exception: Deployed nodes do not mount your reserved storage by default. Let's try:

frontend: oarsub -l nodes=3 -t deploy -I

frontend: kadeploy3 -e wheezy-x64-nfs -f $OAR_NODE_FILE -k

frontend: ssh node

node: ls /data
ls: cannot access /data: No such file or directory

node: exit

For this situation, Storage5k provides you with a way to mount the reserved space into your nodes. This command will mount your storage in all the nodes belonging to the job nodes_job_id

frontend: storage5k -a mount -j nodes_job_id

frontend: ssh node

node: ls /data/username_storage_job_id/
lost+found my_big_data

You can umount storage with:

frontend: storage5k -a umount -j nodes_job_id

With the knowledge of the Source nfs entry, you can manually mount your storage as well:

frontend: ssh root@node   #As root

root@node: mount storage5k.lyon.grid5000.fr:/data/username_storage_job_id /mnt

root@node: exit

frontend: ssh node        #As normal user

node: ls /mnt/
lost+found my_big_data

node: exit

	Warning
	Accessing storage in deployed nodes requires the nfs-common package. It is included by default in the nfs, big and std environments