Group Storage: Difference between revisions

From Grid5000
Jump to navigation Jump to search
 
(41 intermediate revisions by 9 users not shown)
Line 6: Line 6:
This service provides large storage spaces, possibly shared between multiple Grid'5000 users. Those storage spaces are accessible on Grid'5000 over NFS.
This service provides large storage spaces, possibly shared between multiple Grid'5000 users. Those storage spaces are accessible on Grid'5000 over NFS.


For smaller storage space (under 200GB), [https://www.grid5000.fr/w/Storage#.2Fhome asking for a quota extension for your home] may be a better solution.
For smaller storage space (under 200GB), [https://www.grid5000.fr/w/Storage#.2Fhome asking for a quota extension for your home] may be a better solution if you do not need to share data between users. You can also ask for a quota extension for your home in the range of 200-400GB if your request is limited in time (no more than one year).
 
Remind that '''Grid'5000 does NOT have a BACKUP''' service for storage it provides: it is your responsibility to save important data outside Grid'5000 (or at least to copy data to several Grid'5000 sites in order to increase redundancy).  


== Using an existing storage space ==
== Using an existing storage space ==


You must be member of the group associated to storage space to access it. Contact the storage space owner to request an access.
You must be a member of the group associated to storage space to access it. Contact the storage space owner to request an access.


=== On frontends and nodes on standard, nfs or big environment ===
Once you have been authorized, the storage space is available from nodes and frontends at:
Once you have been authorized, the storage space is available from nodes and frontends at:


<code class="host">node:</code><code class="command">cd /srv/storage/</code><code class="replace">storage_name</code>@<code class="replace">server_address</code>
<code class="host">node:</code><code class="command">cd /srv/storage/</code><code class="replace">storage_name</code>@<code class="replace">server_hostname_(fqdn)</code>
 
{{Note|text=Autofs is used to remotely mount storage. So they are automatically  mounted  when  accessed. The file systems are then automatically unmounted after a period of inactivity. That's why you may not see the directory when doing <code class="command">ls /srv/storage</code>, you have to access it before with <code class="command">cd /srv/storage/my-storage@server</code> for autofs to mount it.}}


(autofs is used to remotely mount the storage)
{{Note|text=By default, the storage is created with some specific rights and ACLs:
* <code class="replace">s</code> on group permission: New files will be created with the same group as the parent directory
* sticky bit (<code class="replace">t</code>): Prevent deletion of file not owned.
* <code class="replace">g::rwx:</code> ACL:  Sub-directory will be created with 'rwx' access rights for group, you can check the ACLs with the <code>getfacl</code> command}}


For example:
For example:
Line 22: Line 30:
<code class="host">node:</code><code class="command">cd /srv/storage/</code><code class="replace">my_storage</code>@<code class="replace">storage1.lille.grid5000.fr</code>
<code class="host">node:</code><code class="command">cd /srv/storage/</code><code class="replace">my_storage</code>@<code class="replace">storage1.lille.grid5000.fr</code>


You may want to add a symbolic link to this path for convenience:
'''We strongly advise you to add a symbolic link to this path in your home for convenience :'''


<code class="host">node:</code><code class="command">ln -s /srv/storage/</code><code class="replace">my_storage</code>@<code class="replace">storage1.lille.grid5000.fr</code><code class="command"> ~/my_storage</code>
<code class="host">node:</code><code class="command">ln -s /srv/storage/</code><code class="replace">my_storage</code>@<code class="replace">storage1.lille.grid5000.fr</code><code class="command"> ~/my_storage</code>


{{Warning|text=This will only work from the frontends or from a node with <code class="env">nfs</code>,<code class="env">big</code> or <code class="env">std</code> environment. Otherwise, you will need to use the [[Storage_Manager|storage API]] to enable access to the storage space. Contact the Grid'5000 team if needed.}}
=== On other deployed environment or [[Persistent Virtual Machine]] ===
If you use a deployed environement (other than <code class="env">nfs</code> or <code class="env">big</code>) or a [[Persistent Virtual Machine]], you will need to use the [[Storage_Manager|storage API]] to enable access to the storage space. Contact the Grid'5000 team if needed.
 
== Request for a storage space ==
 
In this section, we list different use cases which lead to different ways to ask for a storage space in Grid'5000.
 
Each user joining Grid'5000 requests to be part of a GGA (Group Granting Access). A GGA is a group managed by an authority (ie: Research Team leader, Grid'5000 Site Manager) qualified to grant an access to the platform to a user for a certain amount of time. This access can include or not associated storage spaces, depending on the fact that the GGA is also a Storage Group.
 
To know if your GGA is able to grant access to storage spaces, just check into the User account Management System [https://api.grid5000.fr/stable/users/#myaccount] in the Groups section if the "Storage" keyword is displayed just after the "Granting Access" keyword of your GGA. If it is the case, ask to your GGA managers the list of storage spaces associated with their GGA.
 
There may be many of them, on different servers of different sites, and they are usually accessible under the following path:
  /srv/storage/GGA_NAME@STORAGE_SERVER_FQDN
  /srv/storage/ALT_NAME@STORAGE_SERVER_FQDN
 
=== Your GGA is not a team level GGA ===
 
For instance you belong to a site GGA like misc, site-misc (eg: nancy-misc).
 
Then proceed as described in the section "Request a new storage group".
 
=== You need to share data with users belonging to different GGAs than yours or a subset of members of your GGA ===
 
Then proceed as described in the section "Request a new storage group".
 
=== You need to store a significant amount of data for a limited time ===
 
Then proceed as described in the section "Request a new storage group".
 
=== An existing storage space associated with your GGA matches your needs ===
 
It is fine, you can use it right away by creating a directory of yours at the top level directory of the storage space (beware, you may have to follow some team guidelines to create your data space within this area).
 
=== An existing storage space associated with your GGA almost matches your needs ===
 
Typically, there is not enough space in it, but not so much (eg: the FS has 1TB available and you need 2TB).
 
In this case, ask to your GGA manager to request to support-staff for a size extension (maybe some cleanup in this storage space will be sufficient enough).
 
Please note that this is only relevant if your do not need a significant amount of data for a limited time (if you do, your are not in this use case).
 
=== None of the existing storage spaces associated with your GGA are matching your needs and you can share your data with other members of your GGA ===
 
For instance, to be as close as possible of the cluster's nodes you are doing your experiments, you want to use a team storage space in a site where it is not yet available.
Proceed as described in the section "Request a new storage group" but specify your GGA name as the name of the group (even if it does not start with sto).


== Requesting a new storage space ==
=== Your GGA is not yet a Storage Group and you can share your data with other members of your GGA ===
 
Proceed as described in the section "Request a new storage group" but specify your GGA name as the name of the group (even if it does not start with sto).
 
In this case, we will inform your GGA managers that the GGA is promoted into a Storage Group and explain them what this imply (mostly benefits).
 
=== If nothing matches ===
 
Proceed as described in the section "Request a new storage group".
 
== Request a new storage group ==


To request a new storage space, please send an email to [mailto:support-staff@lists.grid5000.fr support-staff@lists.grid5000.fr]. Please copy/paste the following text and fill all the required information
To request a new storage space, please send an email to [mailto:support-staff@lists.grid5000.fr support-staff@lists.grid5000.fr]. Please copy/paste the following text and fill all the required information
Line 39: Line 102:


   What is your Grid'5000 login ?
   What is your Grid'5000 login ?
  <...>
    <...>
   What name do you want for your storage space ? (only lowercase letters, numbers and hyphen (-) allowed)
   What name do you want for your storage space ? (only lowercase letters and numbers are allowed)
  <...>
    <...>
   On which site do you want the storage space ? You can also specify the server.
   On which site do you want the storage space ? You can also specify the server.
  <...>
    <...>
   What is the name of the group you want to be link with this storage space ? (must start by "sto-", only lowercase letters, numbers and hyphen (-) allowed)
   Is there an existing users group that could be re-used to authorize access to this storage space ? To see your existing groups, use the account management interface (https://api.grid5000.fr/stable/users/). If Yes:
  sto-<...>
    <name of existing group>
  Does the group already exists (from another storage space) ?
  Or should we create a new group for this storage space ? (Group name must begin by "sto-" ; Only lowercase letters ; Numbers and hyphen (-) allowed)
  Yes/No
    sto-<...>
   What size do you require for the storage space?
   What size do you require for the storage space?
  XXX GBytes/TBytes
    XXX GBytes/TBytes
   What is the intended usage, in a few lines ?
   What is the intended usage, in a few lines ?
  <...>
    <...>
   What is the expiration date (We will contact you before removing your storage) ?
   What is the expiration date (We will contact you before removing your storage) ?
  <...>
    <...>


The Grid'5000 team will create the storage space, and if the group is new, will create and set you as an owner of the group.
The Grid'5000 team will create the storage space, and if the group is new, will create and set you as an owner of the group.


=== Available servers for storage spaces ===
=== Available servers for storage spaces ===
{| class="wikitable"
{{:Generated/GroupStorage}}
|-
! Site !! Server Name !! Size !! Status !! Notes
|-
| [[Lille:Home|Lille]] || storage1.lille.grid5000.fr || 90TB || [[Image:Check.png|center]] ||
|-
| [[Lille:Home|Lille]] || storage2.lille.grid5000.fr || 90TB || [[Image:Check.png|center]] ||
|-
|}
 
More servers will be added when this service will no longer be in beta.


=== Asking for storage extension ===
=== Asking for storage extension ===
You can also contact the Grid'5000 team to get more space or to extend the expiration date of your storage. Just send a mail to [mailto:support-staff@lists.grid5000.fr support-staff@lists.grid5000.fr] with <code>Group storage extension</code> as subject and include the name of your storage, what extension do you need and why do you need it.
You can also contact the Grid'5000 team to get more space or to extend the expiration date of your storage. Just send a mail to [mailto:support-staff@lists.grid5000.fr support-staff@lists.grid5000.fr] with <code>Group storage extension</code> as subject and include the name of your storage, what extension do you need and why do you need it.


== Managing the group of users allowed to access the storage ==
== Managing the group of users allowed to access the storage in [https://api.grid5000.fr/stable/users/ UMS] ==
For now, the only way to manage the group is by using the REST API.


To get information about a group you may do:
{{Note|text=Groups have '''owners''', '''delegates''' and '''members'''. Owners and delegates have basically the same access rights with respect to membership management, but managing the group's meta-data such as its description is reserved to owners. '''You must be a member to be able to access the storage. An owner/delegate is NOT automatically added as a member.'''
 
  <code class="host">frontend:</code><code class="command"><nowiki>curl 'https://api.grid5000.fr/stable/users/groups/</nowiki></code><code class="replace">sto-mygroup</code><code class="command">' | jq </code>
  {
    "uid": 1234,
    "name": "sto-mygroup",
    "manager_uid": "",
    "enabled": true,
    "system": false,
    "grants_access": false,
    "freely_join": false,
    "freely_leave": true,
    "created_at": "2019-03-06T09:11:13.000Z",
    "updated_at": "2019-03-06T09:22:12.000Z",
    "description": "My awesome group",
    "owners": [
      "dtennant"
    ],
    "delegates": [
      "msmith",
      "dtennant"
    ],
    "members": [
      "dtennant",
      "msmith",
      "pcapaldi"
    ],
    "site": false
  }
{{Note|text=The following point are relevant to basic group management:
* groups have '''owners''', '''delegates''' and '''members'''. All owners are delegates and all delegates are members. Owners and delegates have basically the same access rights with respect to membership management, but managing the group's meta-data such as its description is reserved to owners.
* '''freely_join''': if true, users can freely add themselves to the group. If not, one of the group's delegate must add the user.
* '''freely_leave''': if true, users can freely remove themselves to the group. If not, one of the group's delegate must remove the user.


}}
}}


To manage a group, for example adding <code class="replace">'jwhittaker'</code> as a member, first we get the current group configuration:
{{Note|text=You may have to wait up to 15 minutes after your mail is answered by the Grid'5000 team for the changes to take effect.}}
 
=== As a manager ===
  <code class="host">frontend:</code><code class="command"><nowiki>curl 'https://api.grid5000.fr/stable/users/groups/</nowiki></code><code class="replace">sto-mygroup</code><code class="command">' | jq '.' > group_config.json</code>
==== Accept a request ====
Under the [https://api.grid5000.fr/stable/users/#groups '''Groups''' tab] at the bottom of the page you will find the pending request. You can accept the request and specify if the user should also be a owner or a delegate


Then edit <code class="command">group_config.json</code> by adding <code class="replace">'jwhittaker'</code> to the <code>members</code> array with your favorite text editor. Now we just need to send the new configuration to the API:
==== Add a user to a group ====
Under the [https://api.grid5000.fr/stable/users/#groups '''Groups''' tab], you can see the list of groups you can manage and add user to them.
You can also add new owner and delegate


  <code class="host">frontend:</code><code class="command">curl -X PUT -H 'Content-Type: application/json' <nowiki>'https://api.grid5000.fr/stable/users/groups/</nowiki></code><code class="replace">sto-mygroup</code><code class="command">' -d '@group_config.json' | jq</code>
=== As a user ===
You can request to join an existing group storage via the '''Groups''' tab under [https://api.grid5000.fr/stable/users/#myaccount '''My Account'''].


You may have to wait up to 15 minutes for the changes to take effect.
Click on '''Join a new group''' and specify the group starting with '''sto''', the motivation and date limit.

Latest revision as of 18:23, 17 January 2024

See also: Storage overview | Storage Manager | Group Storage | Ceph | Local (on nodes) disks reservation
Note.png Note

This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

This service provides large storage spaces, possibly shared between multiple Grid'5000 users. Those storage spaces are accessible on Grid'5000 over NFS.

For smaller storage space (under 200GB), asking for a quota extension for your home may be a better solution if you do not need to share data between users. You can also ask for a quota extension for your home in the range of 200-400GB if your request is limited in time (no more than one year).

Remind that Grid'5000 does NOT have a BACKUP service for storage it provides: it is your responsibility to save important data outside Grid'5000 (or at least to copy data to several Grid'5000 sites in order to increase redundancy).

Using an existing storage space

You must be a member of the group associated to storage space to access it. Contact the storage space owner to request an access.

On frontends and nodes on standard, nfs or big environment

Once you have been authorized, the storage space is available from nodes and frontends at:

node:cd /srv/storage/storage_name@server_hostname_(fqdn)

Note.png Note

Autofs is used to remotely mount storage. So they are automatically mounted when accessed. The file systems are then automatically unmounted after a period of inactivity. That's why you may not see the directory when doing ls /srv/storage, you have to access it before with cd /srv/storage/my-storage@server for autofs to mount it.

Note.png Note

By default, the storage is created with some specific rights and ACLs:

  • s on group permission: New files will be created with the same group as the parent directory
  • sticky bit (t): Prevent deletion of file not owned.
  • g::rwx: ACL: Sub-directory will be created with 'rwx' access rights for group, you can check the ACLs with the getfacl command

For example:

node:cd /srv/storage/my_storage@storage1.lille.grid5000.fr

We strongly advise you to add a symbolic link to this path in your home for convenience :

node:ln -s /srv/storage/my_storage@storage1.lille.grid5000.fr ~/my_storage

On other deployed environment or Persistent Virtual Machine

If you use a deployed environement (other than nfs or big) or a Persistent Virtual Machine, you will need to use the storage API to enable access to the storage space. Contact the Grid'5000 team if needed.

Request for a storage space

In this section, we list different use cases which lead to different ways to ask for a storage space in Grid'5000.

Each user joining Grid'5000 requests to be part of a GGA (Group Granting Access). A GGA is a group managed by an authority (ie: Research Team leader, Grid'5000 Site Manager) qualified to grant an access to the platform to a user for a certain amount of time. This access can include or not associated storage spaces, depending on the fact that the GGA is also a Storage Group.

To know if your GGA is able to grant access to storage spaces, just check into the User account Management System [1] in the Groups section if the "Storage" keyword is displayed just after the "Granting Access" keyword of your GGA. If it is the case, ask to your GGA managers the list of storage spaces associated with their GGA.

There may be many of them, on different servers of different sites, and they are usually accessible under the following path: 

 /srv/storage/GGA_NAME@STORAGE_SERVER_FQDN
 /srv/storage/ALT_NAME@STORAGE_SERVER_FQDN

Your GGA is not a team level GGA

For instance you belong to a site GGA like misc, site-misc (eg: nancy-misc).

Then proceed as described in the section "Request a new storage group".

You need to share data with users belonging to different GGAs than yours or a subset of members of your GGA

Then proceed as described in the section "Request a new storage group".

You need to store a significant amount of data for a limited time

Then proceed as described in the section "Request a new storage group".

An existing storage space associated with your GGA matches your needs

It is fine, you can use it right away by creating a directory of yours at the top level directory of the storage space (beware, you may have to follow some team guidelines to create your data space within this area).

An existing storage space associated with your GGA almost matches your needs

Typically, there is not enough space in it, but not so much (eg: the FS has 1TB available and you need 2TB).

In this case, ask to your GGA manager to request to support-staff for a size extension (maybe some cleanup in this storage space will be sufficient enough).

Please note that this is only relevant if your do not need a significant amount of data for a limited time (if you do, your are not in this use case).

None of the existing storage spaces associated with your GGA are matching your needs and you can share your data with other members of your GGA

For instance, to be as close as possible of the cluster's nodes you are doing your experiments, you want to use a team storage space in a site where it is not yet available.

Proceed as described in the section "Request a new storage group" but specify your GGA name as the name of the group (even if it does not start with sto).

Your GGA is not yet a Storage Group and you can share your data with other members of your GGA

Proceed as described in the section "Request a new storage group" but specify your GGA name as the name of the group (even if it does not start with sto).

In this case, we will inform your GGA managers that the GGA is promoted into a Storage Group and explain them what this imply (mostly benefits).

If nothing matches

Proceed as described in the section "Request a new storage group".

Request a new storage group

To request a new storage space, please send an email to support-staff@lists.grid5000.fr. Please copy/paste the following text and fill all the required information

  • Email subject:
Group storage creation
  • Email content:
 What is your Grid'5000 login ?
    <...>
 What name do you want for your storage space ? (only lowercase letters and numbers are allowed)
    <...>
 On which site do you want the storage space ? You can also specify the server.
    <...>
 Is there an existing users group that could be re-used to authorize access to this storage space ? To see your existing groups, use the account management interface (https://api.grid5000.fr/stable/users/). If Yes:
    <name of existing group>
 Or should we create a new group for this storage space ? (Group name must begin by "sto-" ; Only lowercase letters ; Numbers and hyphen (-) allowed)
    sto-<...>
 What size do you require for the storage space?
    XXX GBytes/TBytes
 What is the intended usage, in a few lines ?
    <...>
 What is the expiration date (We will contact you before removing your storage) ?
    <...>

The Grid'5000 team will create the storage space, and if the group is new, will create and set you as an owner of the group.

Available servers for storage spaces

Site Server Name Size Link Speed Notes
Grenoble storage1.grenoble.grid5000.fr 55.0 TB 50 Gbps
Lille storage1.lille.grid5000.fr 90.0 TB 50 Gbps
Lille storage2.lille.grid5000.fr 90.0 TB 50 Gbps
Luxembourg storage1.luxembourg.grid5000.fr 40.0 TB 20 Gbps
Lyon storage1.lyon.grid5000.fr 75.0 TB 10 Gbps
Nancy storage1.nancy.grid5000.fr 200.0 TB 10 Gbps
Nancy storage2.nancy.grid5000.fr 90.0 TB 10 Gbps
Nancy storage3.nancy.grid5000.fr 180.0 TB 10 Gbps
Nancy storage4.nancy.grid5000.fr 230.0 TB 10 Gbps
Nancy storage5.nancy.grid5000.fr 50.0 TB 10 Gbps
Rennes storage2.rennes.grid5000.fr 115.0 TB 50 Gbps
Rennes storage3.rennes.grid5000.fr 115.0 TB 50 Gbps
Sophia storage1.sophia.grid5000.fr 30.0 TB 1 Gbps
Toulouse storage1.toulouse.grid5000.fr 10.0 TB 20 Gbps

Last generated from the Grid'5000 Reference API on 2024-04-23 (commit 1f5b4fc89f)

Asking for storage extension

You can also contact the Grid'5000 team to get more space or to extend the expiration date of your storage. Just send a mail to support-staff@lists.grid5000.fr with Group storage extension as subject and include the name of your storage, what extension do you need and why do you need it.

Managing the group of users allowed to access the storage in UMS

Note.png Note

Groups have owners, delegates and members. Owners and delegates have basically the same access rights with respect to membership management, but managing the group's meta-data such as its description is reserved to owners. You must be a member to be able to access the storage. An owner/delegate is NOT automatically added as a member.

Note.png Note

You may have to wait up to 15 minutes after your mail is answered by the Grid'5000 team for the changes to take effect.

As a manager

Accept a request

Under the Groups tab at the bottom of the page you will find the pending request. You can accept the request and specify if the user should also be a owner or a delegate

Add a user to a group

Under the Groups tab, you can see the list of groups you can manage and add user to them. You can also add new owner and delegate

As a user

You can request to join an existing group storage via the Groups tab under My Account.

Click on Join a new group and specify the group starting with sto, the motivation and date limit.

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Group_Storage&oldid=101582"