Grid5000:Network: Difference between revisions

From Grid5000
Jump to navigation Jump to search
← Older edit
Lnussbaum (talk | contribs)
6,377 edits
No edit summary
MTU documentation
 
(30 intermediate revisions by 10 users not shown)
Line 8: Line 8:
This page provides an overview of Grid'5000 network infrastructure. Other useful pages:
This page provides an overview of Grid'5000 network infrastructure. Other useful pages:
* [[Status#Network_Monitoring|'''Network monitoring section of the Status page''']]
* [[Status#Network_Monitoring|'''Network monitoring section of the Status page''']]
* [[Virtual_machines_on_the_production_environment]] tutorial (provides introduction to [[Subnet_reservation]])
* [[Virtualization on Grid'5000]] tutorial (provides introduction to [[Subnet reservation]])
* [[Deploying_OpenStack_using_KaVLAN]] tutorial (provides introduction to [[KaVLAN]]; there's also another [[Network_isolation_on_Grid%275000|more advanced tutorial]])
* [[Network reconfiguration tutorial]] (provides introduction to [[KaVLAN]]; there's also another [[Advanced KaVLAN|more advanced tutorial]])
}}
}}


Line 16: Line 16:


[[Image:Logo RENATER.png|150px|left]]
[[Image:Logo RENATER.png|150px|left]]
Grid'5000 backbone network infrastructure is provided by '''[http://www.renater.fr RENATER]'''. '''RENATER''' is the French National Telecommunication Network for Technology, Education and Research. More information can be found on [http://www.renater.fr renater.fr]. '''[http://www.renater.fr RENATER]''' offers about 30 POPs (Points Of Presence) in France, at least one POP for each region, which metropolitan and regional networks are connected on. More than 600 sites (Universities, Research Centers, ..) are interconnected through '''[http://www.renater.fr RENATER]'''.
Grid'5000 backbone network infrastructure is provided by '''[http://www.renater.fr RENATER]'''. '''RENATER''' is the French National Telecommunication Network for Technology, Education and Research. More information can be found on [http://www.renater.fr renater.fr].


[[Image:renater5.jpg|251px|thumbnail|Renater's WDM infrastructure]]
[[Image:renater5.jpg|251px|thumbnail|Renater's WDM infrastructure]]
[[Image:renater5-g5k.jpg|250px|thumbnail|10Gbps lambda activated for Grid'5000 in Renater's Dark Fiber infrastructure]]
[[Image:g5k-backbone.png|250px|thumbnail|10Gbps lambda activated for Grid'5000 in Renater's infrastructure]]
[[Image:Schema_Backbone.png|251px|thumbnail|Grid5000 IP Network]]




The actual phase of the network is '''RENATER-5''', the deployment has been completed by January 2009.
The actual phase of the network is '''RENATER-5''', the deployment has been completed by January 2009. Renater provides Grid'5000 a dedicated 10Gbit/s switching network  (using dedicated ''lambdas'' on Renater network infrastructure) and every Grid'5000 router devices is connected to the closest Renater's Point of Presence.
The "standard" architecture is based on '''10Gbit/s''' dark fibers and provides IP transit connectivity, interconnection with GEANT-2 (http://www.geant2.net/), overseas territories and the SFINX (Global Internet exchange).


From Grid'5000 point of view, this '''backbone''' network interconnecting its sites is a single layer-2 Ethernet network. To forward communication between two sites, IP routing is performed through the backbone network.


The IP addresses of sites' router in the backbone network are:


All Grid'5000 sites (with the exception of Reims and Nantes) are connected to dedicated 10Gbit/s ''lambdas'' on a DWDM infrastructure (see [http://en.wikipedia.org/wiki/Wavelength-division_multiplexing Wikipedia: Wavelength-division Multiplexing] for more info).
Network address:        <code class="host">192.168.4.0/24</code>
 
Sophia:                 <code class="host">192.168.4.12</code>
Reims and Nantes are connected using an L2 VPN.
Lyon:                    <code class="host">192.168.4.13</code>
 
Nancy:                  <code class="host">192.168.4.14</code>
Grenoble:                <code class="host">192.168.4.15</code>
Toulouse:               <code class="host">192.168.4.16</code>
Lille:                  <code class="host">192.168.4.18</code>
Rennes:                  <code class="host">192.168.4.19</code>
Luxembourg:              <code class="host">192.168.4.21</code>
Nantes:                  <code class="host">192.168.4.22</code>
Strasbourg:              <code class="host">192.168.4.23</code>
Louvain:                <code class="host">192.168.4.24</code>




Line 38: Line 46:
Grid'5000 sites provide access to a variety of network technologies (both Ethernet and high-performance networks such as InfiniBand).
Grid'5000 sites provide access to a variety of network technologies (both Ethernet and high-performance networks such as InfiniBand).
More information is available on the following pages:
More information is available on the following pages:
* [[Bordeaux:Network|Bordeaux]]
* [[Grenoble:Network|Grenoble]]
* [[Grenoble:Network|Grenoble]]
* [[Lille:Network|Lille]]
* [[Lille:Network|Lille]]
Line 45: Line 52:
* [[Nancy:Network|Nancy]]
* [[Nancy:Network|Nancy]]
* [[Nantes:Network|Nantes]]
* [[Nantes:Network|Nantes]]
* [[Reims:Network|Reims]]
* [[Rennes:Network|Rennes]]
* [[Rennes:Network|Rennes]]
* [[Sophia:Network|Sophia]]
* [[Sophia:Network|Sophia]]
* [[Toulouse:Network|Toulouse]]
* [[Toulouse:Network|Toulouse]]
* [[Strasbourg:Network|Strasbourg]]
* [[Louvain:Network|Louvain]]
The list of high performance networks is available on [[Hardware#Network_interconnects|this page]].


The list of high performance networks is available on [[Special:G5KHardware#High_performance_network_families|this page]].
== Network MTU ==
* A MTU (Maximum Transmission Unit) of up to 9000 is guaranteed to work on Grid'5000: within a site, between sites, and in a Kavlan
* Values of MTU above 9000 may work locally between nodes on the same site, but there is no guarantee. Contact [[Support]] if your experiment needs a MTU above 9000.
* The default MTU configured on nodes is 1500, but you can easily change the MTU during your experiments:
sudo-g5k
ip link set INTERFACE mtu 9000


=Address space for virtual machines=
=Address space for virtual machines=
Grid'5000 has two different mechanisms to provide IP addresses to virtual machines: subnet reservation and KaVLAN.
Grid'5000 has two different mechanisms to provide IP addresses to virtual machines: subnet reservation and KaVLAN.
* '''Subnet reservation''' provides a mean to allocate IP addresses in the production (= default) network. This provides a way to avoid conflicts between users, but does not provide isolation: all machines are still in the same L2 network. Subnet reservation is documented in the [[Virtual_machines_on_the_production_environment]] tutorial, and on the [[Subnet_reservation]] page.
* '''Subnet reservation''' provides a mean to allocate IP addresses in the production (= default) network. This provides a way to avoid conflicts between users, but does not provide isolation: all machines are still in the same L2 network. Subnet reservation is documented in the [[Virtualization_on_Grid'5000|Virtualization on the standard environment]] tutorial, and on the [[Subnet reservation]] page.
* '''KaVLAN''' provides network isolation through dynamic switches and routers reconfiguration, for the duration of an experiment. Inside a KaVLAN network, users are free to allocate IP addresses as needed. However, some IP ranges are routed to/from the KaVLAN network. KaVLAN is documented in the [[Deploying_OpenStack_using_KaVLAN]] tutorial (which provides an introduction to KaVLAN), in the [[Network_isolation_on_Grid%275000]] tutorial (which is a more advanced KaVLAN tutorial), and on the [[KaVLAN]] page.
* '''KaVLAN''' provides network isolation through dynamic switches and routers reconfiguration, for the duration of an experiment. Inside a KaVLAN network, users are free to allocate IP addresses as needed. However, some IP ranges are routed to/from the KaVLAN network. KaVLAN is documented in the [[Network reconfiguration tutorial]] (which provides an introduction to KaVLAN), in the [[Network isolation on Grid%275000]] tutorial (which is a more advanced KaVLAN tutorial), and on the [[KaVLAN]] page.


=Tables of available IP networks=
=Tables of available IP networks=
== Production network and subnet reservation ==
== Production network and subnet reservation ==
{|
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_Golden_rules/Addressing_plan -->
|- style="background-color: #cccccc; border:1px solid #c9c9c9;"
{{:Network_Golden_rules/Addressing_plan}}
! Site        !! State !! Production network                        !! Administration network                  !! High Perf network (ib/mx)                    !! Kavlan network                        !! Virtualization
|-
| Bordeaux    || {{Yes}} || <code class="host">172.16.0.0/20</code>  || <code class="host">172.17.0.0/20</code>  || <code class="host">172.18.0.0/20</code>  || <code class="host">10.0.0.0/14</code> || <code class="host">10.128.0.0/14</code>
|-  
| Grenoble    || {{Yes}} || <code class="host">172.16.16.0/20</code>  || <code class="host">172.17.16.0/20</code> || <code class="host">172.18.16.0/20</code> || <code class="host">10.4.0.0/14</code> || <code class="host">10.132.0.0/14</code>
|-  
| Lille        || {{Yes}} || <code class="host">172.16.32.0/20</code>  || <code class="host">172.17.32.0/20</code> || <code class="host">172.18.32.0/20</code> || <code class="host">10.8.0.0/14</code> || <code class="host">10.136.0.0/14</code>
|-
| Lyon        || {{Yes}} || <code class="host">172.16.48.0/20</code>  || <code class="host">172.17.48.0/20</code> || <code class="host">172.18.48.0/20</code> || <code class="host">10.12.0.0/14</code> || <code class="host">10.140.0.0/14</code>
|-
| Nancy        ||  {{Yes}} || <code class="host">172.16.64.0/20</code>  || <code class="host">172.17.64.0/20</code> || <code class="host">172.18.64.0/20</code> || <code class="host">10.16.0.0/14</code> || <code class="host">10.144.0.0/14</code>
|-
| <s>Orsay</s>        || {{No}} || <code class="host">172.16.80.0/20</code>  || <code class="host">172.17.80.0/20</code> || <code class="host">172.18.80.0/20</code> || <code class="host">10.20.0.0/14</code> || <code class="host">10.148.0.0/14</code> + <code class="host">10.152.0.0/14</code>
|-
| Rennes      || {{Yes}} || <code class="host">172.16.96.0/20</code>  || <code class="host">172.17.96.0/20</code> || <code class="host">172.18.96.0/20</code> || <code class="host">10.24.0.0/14</code> || <code class="host">10.156.0.0/14</code>
|-
| Toulouse    || {{Yes}} || <code class="host">172.16.112.0/20</code> || <code class="host">172.17.112.0/20</code> || <code class="host">172.18.112.0/20</code> || <code class="host">10.28.0.0/14</code> || <code class="host">10.160.0.0/14</code>
|-
| Sophia      || {{Yes}} || <code class="host">172.16.128.0/20</code> || <code class="host">172.17.128.0/20</code> || <code class="host">172.18.128.0/20</code> || <code class="host">10.32.0.0/14</code> || <code class="host">10.164.0.0/14</code>
|-
| Reims        ||{{Yes}} || <code class="host">172.16.160.0/20</code> || <code class="host">172.17.160.0/20</code> || <code class="host">172.18.160.0/20</code> || <code class="host">10.36.0.0/14</code> || <code class="host">10.168.0.0/14</code>
|-
| Luxembourg  || {{Yes}} || <code class="host">172.16.176.0/20</code> || <code class="host">172.17.176.0/20</code> || <code class="host">172.18.176.0/20</code> || <code class="host">10.40.0.0/14</code> || <code class="host">10.172.0.0/14</code>
|-
| Nantes || {{No}} || <code class="host">172.16.192.0/20</code> || <code class="host">172.17.192.0/20</code> || <code class="host">172.18.192.0/20</code> || <code class="host">10.44.0.0/14</code> || <code class="host">10.176.0.0/14</code>
|-
| <s>Orléans</s>      || {{No}} || <code class="host">172.16.208.0/20</code> || <code class="host">172.17.208.0/20</code> || <code class="host">172.18.208.0/20</code> ||
|-
| Qualification || {{Yes}} || <code class="host">172.16.240.0/20</code> || <code class="host">172.17.240.0/20</code> || <code class="host">172.18.240.0/20</code> || || <code class="host">10.252.0.0/14</code>
|}


=== Virtual subnets configuration ===
=== Virtual subnets configuration ===
{|
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_interlink/Virtual -->
|- style="background-color: #cccccc; border:1px solid #c9c9c9;"
{{:Network_interlink/Virtual}}
| '''Site''' || '''Block''' || '''Gateway''' ||''' DHCP server'''
|-
| Bordeaux || <code class="host">10.128.0.0/14</code> || 10.131.255.254 || 10.131.255.253
|-
| Grenoble || <code class="host">10.132.0.0/14</code>  || 10.135.255.254 || 10.135.255.253
|-
| Lille    || <code class="host">10.136.0.0/14</code>  || 10.139.255.254 || 10.139.255.253
|-
| Lyon    || <code class="host">10.140.0.0/14</code>  || 10.143.255.254 || 10.143.255.253
|-  
| Nancy    || <code class="host">10.144.0.0/14</code>  || 10.147.255.254 || 10.147.255.253
|-  
| Orsay I  || <code class="host">10.148.0.0/14</code> || 10.151.255.254 || 10.151.255.253
|-
| Orsay II || <code class="host">10.152.0.0/14</code>  || 10.155.255.254 || 10.155.255.253
|-
| Rennes  || <code class="host">10.156.0.0/14</code>  || 10.159.255.254 || 10.159.255.253
|-
| Toulouse || <code class="host">10.160.0.0/14</code>  || 10.163.255.254 || 10.163.255.253
|-
| Sophia  || <code class="host">10.164.0.0/14</code>  || 10.167.255.254 || 10.167.255.253
|-
| Reims    || <code class="host">10.168.0.0/14</code> || 10.171.255.254 || 10.171.255.253
|-
| Luxembourg    || <code class="host">10.172.0.0/14</code> || 10.175.255.254 || 10.175.255.253
|-
| Nantes  || <code class="host">10.176.0.0/14</code> || 10.179.255.254 || 10.179.255.253
|-
| Qualif || <code class="host">10.252.0.0/14</code>  || 10.255.255.254 || 10.255.255.253
|}


== KaVLAN ==
== KaVLAN ==
=== KaVLAN deployment status ===
=== KaVLAN deployment status ===
{| class="checks" style="width: auto;"
<!-- https://www.grid5000.fr/mediawiki/index.php/KaVLAN/Version -->
! class="left" |Sites
{{:KaVLAN/Version}}
! Version
! Status
|-
| class="left" |[[Bordeaux:Home|Bordeaux]]
|
| {{No}}
|-
| class="left" |[[Grenoble:Home|Grenoble]]
| 1.1.15-1
| {{Yes}}
|-
| class="left" |[[Lille:Home|Lille]]
| 1.1.14-1
| {{Yes}}
|-
| class="left" |[[Luxembourg:Home|Luxembourg]]
| 1.1.14-1
| {{Yes}}
|-
| class="left" |[[Lyon:Home|Lyon]]
| 1.1.14-1
| {{Yes}}
|-
| class="left" |[[Nancy:Home|Nancy]]
| 1.1.16-1
| {{Yes}}
|-
| class="left" |[[Nantes:Home|Nantes]]
|
| {{No}}
|-
| class="left" |[[Reims:Home|Reims]]
| 1.1.14-1
| {{Inprogress}} (global doesn't work -- {{Bug|4634}})
|-
| class="left" |[[Rennes:Home|Rennes]]
| 1.1.14-1
| [[Image:Check.png]]
|-
| class="left" |[[Sophia:Home|Sophia]]
| 1.1.14-1
| {{Yes}}
|-
| class="left" |[[Toulouse:Home|Toulouse]]
| 1.1.15-1
| {{Yes}}
|}


=== KaVLAN networks ===
=== KaVLAN networks ===
==== Non-routed local Vlans ====
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_Golden_rules/KaVLAN -->
{|
{{:Network_Golden_rules/KaVLAN}}
|- style="background-color: #cccccc; border:1px solid #c9c9c9;"
! Site        !! KAVLAN-1 !! KAVLAN-2 !! KAVLAN-3
|-
| All    || <code class="host">192.168.192.0/20</code> || <code class="host">192.168.208.0/20</code>  || <code class="host">192.168.224.0/20</code>
|}


==== Routed Local Vlans ====
=== KaVLAN-NG networks ===
{|
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_Golden_rules/KaVLAN-NG -->
|- style="background-color: #cccccc; border:1px solid #c9c9c9;"
{{:Network_Golden_rules/KaVLAN-NG}}
! Site        !! KAVLAN-4 !! KAVLAN-5 !! KAVLAN-6 !! KAVLAN-7 !! KAVLAN-8 !! KAVLAN-9
|-
| Bordeaux    || <code class="host">10.0.0.0/18</code> || <code class="host">10.0.64.0/18</code>  || <code class="host">10.0.128.0/18</code>  || <code class="host">10.0.192.0/18</code>  || <code class="host">10.1.0.0/18</code>  || <code class="host">10.1.64.0/18</code>
|-
| Grenoble    || <code class="host">10.4.0.0/18</code> || <code class="host">10.4.64.0/18</code>  || <code class="host">10.4.128.0/18</code>  || <code class="host">10.4.192.0/18</code>  || <code class="host">10.5.0.0/18</code>  || <code class="host">10.5.64.0/18</code>
|-  
| Lille        || <code class="host">10.8.0.0/18</code> || <code class="host">10.8.64.0/18</code>  || <code class="host">10.8.128.0/18</code>  || <code class="host">10.8.192.0/18</code>  || <code class="host">10.9.0.0/18</code>  || <code class="host">10.9.64.0/18</code>
|-
| Lyon        || <code class="host">10.12.0.0/18</code> || <code class="host">10.12.64.0/18</code>  || <code class="host">10.12.128.0/18</code>  || <code class="host">10.12.192.0/18</code>  || <code class="host">10.13.0.0/18</code>  || <code class="host">10.13.64.0/18</code>
|-  
| Nancy        || <code class="host">10.16.0.0/18</code> || <code class="host">10.16.64.0/18</code>  || <code class="host">10.16.128.0/18</code>  || <code class="host">10.16.192.0/18</code>  || <code class="host">10.17.0.0/18</code>  || <code class="host">10.17.64.0/18</code>
|-  
| Orsay        || <code class="host">10.20.0.0/18</code> || <code class="host">10.20.64.0/18</code>  || <code class="host">10.20.128.0/18</code>  || <code class="host">10.20.192.0/18</code>  || <code class="host">10.21.0.0/18</code>  || <code class="host">10.21.64.0/18</code>
|-  
| Rennes      || <code class="host">10.24.0.0/18</code> || <code class="host">10.24.64.0/18</code>  || <code class="host">10.24.128.0/18</code>  || <code class="host">10.24.192.0/18</code>  || <code class="host">10.25.0.0/18</code>  || <code class="host">10.25.64.0/18</code>
|-
| Toulouse    || <code class="host">10.28.0.0/18</code> || <code class="host">10.28.64.0/18</code>  || <code class="host">10.28.128.0/18</code>  || <code class="host">10.28.192.0/18</code>  || <code class="host">10.29.0.0/18</code>  || <code class="host">10.29.64.0/18</code>
|-
| Sophia      || <code class="host">10.32.0.0/18</code> || <code class="host">10.32.64.0/18</code>  || <code class="host">10.32.128.0/18</code>  || <code class="host">10.32.192.0/18</code>  || <code class="host">10.33.0.0/18</code>  || <code class="host">10.33.64.0/18</code>
|-
| Reims        || <code class="host">10.36.0.0/18</code> || <code class="host">10.36.64.0/18</code>  || <code class="host">10.36.128.0/18</code>  || <code class="host">10.36.192.0/18</code>  || <code class="host">10.37.0.0/18</code>  || <code class="host">10.37.64.0/18</code>
|-
| Luxembourg  || <code class="host">10.40.0.0/18</code> || <code class="host">10.40.64.0/18</code>  || <code class="host">10.40.128.0/18</code>  || <code class="host">10.40.192.0/18</code>  || <code class="host">10.41.0.0/18</code>  || <code class="host">10.41.64.0/18</code>
|-
| Nantes || <code class="host">10.44.0.0/18</code> || <code class="host">10.44.64.0/18</code>  || <code class="host">10.44.128.0/18</code>  || <code class="host">10.44.192.0/18</code>  || <code class="host">10.45.0.0/18</code>  || <code class="host">10.45.64.0/18</code>
|}


==== Global Vlans ====
= Access to the Internet =
{|
|- style="background-color: #cccccc; border:1px solid #c9c9c9;"
! Site        !! Global Vlan !! Subnet !! Router IP
|-
| Bordeaux    || <code class="host">KAVLAN-10</code> || <code class="host">10.3.192.0/18</code>  || <code class="host">10.3.255.254</code>
|-
| Grenoble    || <code class="host">KAVLAN-11</code> || <code class="host">10.7.192.0/18</code>  || <code class="host">10.7.255.254</code>
|-
| Lille        || <code class="host">KAVLAN-12</code> || <code class="host">10.11.192.0/18</code>  || <code class="host">10.11.255.254</code>
|-
| Lyon        || <code class="host">KAVLAN-13</code> || <code class="host">10.15.192.0/18</code>  || <code class="host">10.15.255.254</code>
|-
| Nancy        || <code class="host">KAVLAN-14</code> || <code class="host">10.19.192.0/18</code>  || <code class="host">10.19.255.254</code>
|-
| Orsay        || <code class="host">KAVLAN-15</code> || <code class="host">10.23.192.0/18</code>  || <code class="host">10.23.255.254</code>
|-
| Rennes      || <code class="host">KAVLAN-16</code> || <code class="host">10.27.192.0/18</code>  || <code class="host">10.27.255.254</code>
|-
| Toulouse    || <code class="host">KAVLAN-17</code> || <code class="host">10.31.192.0/18</code>  || <code class="host">10.31.255.254</code>
|-
| Sophia      || <code class="host">KAVLAN-18</code> || <code class="host">10.35.192.0/18</code>  || <code class="host">10.35.255.254</code>
|-
| Reims        || <code class="host">KAVLAN-19</code> || <code class="host">10.39.192.0/18</code>  || <code class="host">10.39.255.254</code>
|-
| Luxembourg  || <code class="host">KAVLAN-20</code> || <code class="host">10.43.192.0/18</code>  || <code class="host">10.43.255.254</code>
|-
| Nantes || <code class="host">KAVLAN-21</code> || <code class="host">10.47.192.0/18</code>  || <code class="host">10.47.255.254</code>
|}


===== Nodes assignments in global VLANs =====
See [[FAQ#Is_access_to_the_Internet_possible_from_nodes.3F]]
* Global vlan is a /18 (16382 hosts), so every site have one /23 (510 ip) in the global vlan.
* Example for the global vlan of Lille 712 (10.11.192.0/18), other example nancy bind config (nancy.db)
** Bordeaux: <code class='host'>10.11.192.1</code> → <code class='host'>10.11.193.254</code>
** Grenoble: <code class='host'>10.11.194.1</code> → <code class='host'>10.11.195.254</code>
** Lille: <code class='host'>10.11.196.1</code> → <code class='host'>10.11.197.254</code>
** Lyon: <code class='host'>10.11.198.1</code> → <code class='host'>10.11.199.254</code>
** Nancy: <code class='host'>10.11.200.1</code> → <code class='host'>10.11.201.254</code>
** Orsay: <code class='host'>10.11.202.1</code> → <code class='host'>10.11.203.254</code>
** Rennes: <code class='host'>10.11.204.1</code> → <code class='host'>10.11.205.254</code>
** Toulouse: <code class='host'>10.11.206.1</code> → <code class='host'>10.11.207.254</code>
** Sophia: <code class='host'>10.11.208.1</code> → <code class='host'>10.11.209.254</code>
** Reims: <code class='host'>10.11.210.1</code> → <code class='host'>10.11.211.254</code>
** Luxembourg: <code class='host'>10.11.212.1</code> → <code class='host'>10.11.213.254</code>
** Nantes: <code class='host'>10.11.214.1</code> → <code class='host'>10.11.215.254</code>

Latest revision as of 15:39, 28 August 2024

Network: Global | Grenoble | Lille | Louvain | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse


Note

This page provides an overview of Grid'5000 network infrastructure. Other useful pages:

Grid'5000 Network Infrastructure

Backbone Network

Grid'5000 backbone network infrastructure is provided by RENATER. RENATER is the French National Telecommunication Network for Technology, Education and Research. More information can be found on renater.fr.

Renater's WDM infrastructure
10Gbps lambda activated for Grid'5000 in Renater's infrastructure


The actual phase of the network is RENATER-5, the deployment has been completed by January 2009. Renater provides Grid'5000 a dedicated 10Gbit/s switching network (using dedicated lambdas on Renater network infrastructure) and every Grid'5000 router devices is connected to the closest Renater's Point of Presence.

From Grid'5000 point of view, this backbone network interconnecting its sites is a single layer-2 Ethernet network. To forward communication between two sites, IP routing is performed through the backbone network.

The IP addresses of sites' router in the backbone network are: 

Network address:         192.168.4.0/24
Sophia:                  192.168.4.12
Lyon:                    192.168.4.13
Nancy:                   192.168.4.14
Grenoble:                192.168.4.15
Toulouse:                192.168.4.16
Lille:                   192.168.4.18
Rennes:                  192.168.4.19
Luxembourg:              192.168.4.21
Nantes:                  192.168.4.22
Strasbourg:              192.168.4.23
Louvain:                 192.168.4.24


Grid'5000 sites networks

Grid'5000 sites provide access to a variety of network technologies (both Ethernet and high-performance networks such as InfiniBand). More information is available on the following pages:

The list of high performance networks is available on this page.

Network MTU

  • A MTU (Maximum Transmission Unit) of up to 9000 is guaranteed to work on Grid'5000: within a site, between sites, and in a Kavlan
  • Values of MTU above 9000 may work locally between nodes on the same site, but there is no guarantee. Contact Support if your experiment needs a MTU above 9000.
  • The default MTU configured on nodes is 1500, but you can easily change the MTU during your experiments:
sudo-g5k
ip link set INTERFACE mtu 9000

Address space for virtual machines

Grid'5000 has two different mechanisms to provide IP addresses to virtual machines: subnet reservation and KaVLAN.

  • Subnet reservation provides a mean to allocate IP addresses in the production (= default) network. This provides a way to avoid conflicts between users, but does not provide isolation: all machines are still in the same L2 network. Subnet reservation is documented in the Virtualization on the standard environment tutorial, and on the Subnet reservation page.
  • KaVLAN provides network isolation through dynamic switches and routers reconfiguration, for the duration of an experiment. Inside a KaVLAN network, users are free to allocate IP addresses as needed. However, some IP ranges are routed to/from the KaVLAN network. KaVLAN is documented in the Network reconfiguration tutorial (which provides an introduction to KaVLAN), in the Network isolation on Grid'5000 tutorial (which is a more advanced KaVLAN tutorial), and on the KaVLAN page.

Tables of available IP networks

Production network and subnet reservation

Here are the private IPv4 networks allocated to every site:

Site State Production network Administration network High Perf network (ib/opa) Ceph internal network Kavlan network Virtualization
Bordeaux (soon) 172.16.0.0/20 172.17.0.0/20 172.18.0.0/20 172.19.0.0/20 10.0.0.0/14 10.128.0.0/14
Grenoble 172.16.16.0/20 172.17.16.0/20 172.18.16.0/20 172.19.16.0/20 10.4.0.0/14 10.132.0.0/14
Lille 172.16.32.0/20 172.17.32.0/20 172.18.32.0/20 172.19.32.0/20 10.8.0.0/14 10.136.0.0/14
Lyon 172.16.48.0/20 172.17.48.0/20 172.18.48.0/20 172.19.48.0/20 10.12.0.0/14 10.140.0.0/14
Nancy 172.16.64.0/20 172.17.64.0/20 172.18.64.0/20 172.19.64.0/20 10.16.0.0/14 10.144.0.0/14
Orsay 172.16.80.0/20 172.17.80.0/20 172.18.80.0/20 172.19.80.0/20 10.20.0.0/14 10.148.0.0/14 + 10.152.0.0/14
Rennes 172.16.96.0/20 172.17.96.0/20 172.18.96.0/20 172.19.96.0/20 10.24.0.0/14 10.156.0.0/14
Toulouse 172.16.112.0/20 172.17.112.0/20 172.18.112.0/20 172.19.112.0/20 10.28.0.0/14 10.160.0.0/14
Sophia 172.16.128.0/20 172.17.128.0/20 172.18.128.0/20 172.19.128.0/20 10.32.0.0/14 10.164.0.0/14
Strasbourg 172.16.160.0/20 172.17.160.0/20 172.18.160.0/20 172.19.160.0/20 10.36.0.0/14 10.168.0.0/14
Luxembourg 172.16.176.0/20 172.17.176.0/20 172.18.176.0/20 172.19.176.0/20 10.40.0.0/14 10.172.0.0/14
Nantes 172.16.192.0/20 172.17.192.0/20 172.18.192.0/20 172.19.192.0/20 10.44.0.0/14 10.176.0.0/14
Louvain 172.16.208.0/20 172.17.208.0/20 172.18.208.0/20 172.19.208.0/20 10.48.0.0/14 10.180.0.0/14
Qualification 172.16.240.0/20 172.17.240.0/20 172.18.240.0/20 10.252.0.0/14

Here are the IPv6 networks allocated to every site (these are global scope networks, thus routable on the internet):

Site State Production network Administration network High Perf network (ib/opa) Ceph internal network Kavlan networks Virtualization
Grenoble 2001:0660:4406:0100::/64 2001:0660:4406:0110::/64 2001:0660:4406:0120::/64 TBD 2001:0660:4406:0180::/58 TBD
Lille 2001:0660:4406:0200::/64 2001:0660:4406:0210::/64 2001:0660:4406:0220::/64 TBD 2001:0660:4406:0280::/58 TBD
Luxembourg 2001:0660:4406:0300::/64 2001:0660:4406:0310::/64 2001:0660:4406:0320::/64 TBD 2001:0660:4406:0380::/58 TBD
Lyon 2001:0660:4406:0400::/64 2001:0660:4406:0410::/64 2001:0660:4406:0420::/64 TBD 2001:0660:4406:0480::/58 TBD
Nancy 2001:0660:4406:0500::/64 2001:0660:4406:0510::/64 2001:0660:4406:0520::/64 TBD 2001:0660:4406:0580::/58 TBD
Nantes 2001:0660:4406:0600::/64 2001:0660:4406:0610::/64 2001:0660:4406:0620::/64 TBD 2001:0660:4406:0680::/58 TBD
Rennes 2001:0660:4406:0700::/64 2001:0660:4406:0710::/64 2001:0660:4406:0720::/64 TBD 2001:0660:4406:0780::/58 TBD
Sophia 2001:0660:4406:0800::/64 2001:0660:4406:0810::/64 2001:0660:4406:0820::/64 TBD 2001:0660:4406:0880::/58 TBD
Toulouse 2001:0660:4406:0900::/64 2001:0660:4406:0910::/64 2001:0660:4406:0920::/64 TBD 2001:0660:4406:0980::/58 TBD
Strasbourg 2001:0660:4406:0a00::/64 2001:0660:4406:0a10::/64 2001:0660:4406:0a20::/64 TBD 2001:0660:4406:0a80::/58 TBD
Louvain 2001:0660:4406:0b00::/64 2001:0660:4406:0b10::/64 2001:0660:4406:0b20::/64 TBD 2001:0660:4406:0b80::/58 TBD

Virtual subnets configuration

Maximum 15 sites from 10.128.0.0/14 to 10.184.0.0/14, and one qualif site 10.252.0.0/14

Site Block Gateway DHCP server
Bordeaux 10.128.0.0/14 10.131.255.254 10.131.255.253
Grenoble 10.132.0.0/14 10.135.255.254 10.135.255.253
Lille 10.136.0.0/14 10.139.255.254 10.139.255.253
Lyon 10.140.0.0/14 10.143.255.254 10.143.255.253
Nancy 10.144.0.0/14 10.147.255.254 10.147.255.253
Orsay I 10.148.0.0/14 10.151.255.254 10.151.255.253
Orsay II 10.152.0.0/14 10.155.255.254 10.155.255.253
Rennes 10.156.0.0/14 10.159.255.254 10.159.255.253
Toulouse 10.160.0.0/14 10.163.255.254 10.163.255.253
Sophia 10.164.0.0/14 10.167.255.254 10.167.255.253
Strasbourg Reims 10.168.0.0/14 10.171.255.254 10.171.255.253
Luxembourg 10.172.0.0/14 10.175.255.254 10.175.255.253
Nantes 10.176.0.0/14 10.179.255.254 10.179.255.253
Louvain 10.180.0.0/14 10.183.255.254 10.183.255.253
available 10.184.0.0/14
Reserved for KavlanNG migration 10.188.0.0/14 to 10.248.0.0/14
Qualif 10.252.0.0/14 10.255.255.254 10.255.255.253

KaVLAN

KaVLAN deployment status

Sites Version Status
Grenoble 1.2.7-1
Lille 1.2.7-1
Luxembourg 1.2.7-1
Lyon 1.2.7-1
Nancy 1.2.7-1
Nantes 1.2.7-1
Rennes 1.2.7-1
Sophia 1.2.7-1
Toulouse 1.2.7-1
Strasbourg 1.2.7-1
Louvain 1.2.7-1

KaVLAN networks

Local VLANs (non-routed)
Site KAVLAN-1 KAVLAN-2 KAVLAN-3
All 192.168.192.0/20 192.168.208.0/20 192.168.224.0/20
Routed VLANs
Site KAVLAN-4 KAVLAN-5 KAVLAN-6 KAVLAN-7 KAVLAN-8 KAVLAN-9
Bordeaux 10.0.0.0/18 10.0.64.0/18 10.0.128.0/18 10.0.192.0/18 10.1.0.0/18 10.1.64.0/18
Grenoble 10.4.0.0/18 10.4.64.0/18 10.4.128.0/18 10.4.192.0/18 10.5.0.0/18 10.5.64.0/18
Lille 10.8.0.0/18 10.8.64.0/18 10.8.128.0/18 10.8.192.0/18 10.9.0.0/18 10.9.64.0/18
Lyon 10.12.0.0/18 10.12.64.0/18 10.12.128.0/18 10.12.192.0/18 10.13.0.0/18 10.13.64.0/18
Nancy 10.16.0.0/18 10.16.64.0/18 10.16.128.0/18 10.16.192.0/18 10.17.0.0/18 10.17.64.0/18
Orsay 10.20.0.0/18 10.20.64.0/18 10.20.128.0/18 10.20.192.0/18 10.21.0.0/18 10.21.64.0/18
Rennes 10.24.0.0/18 10.24.64.0/18 10.24.128.0/18 10.24.192.0/18 10.25.0.0/18 10.25.64.0/18
Toulouse 10.28.0.0/18 10.28.64.0/18 10.28.128.0/18 10.28.192.0/18 10.29.0.0/18 10.29.64.0/18
Sophia 10.32.0.0/18 10.32.64.0/18 10.32.128.0/18 10.32.192.0/18 10.33.0.0/18 10.33.64.0/18
Strasbourg 10.36.0.0/18 10.36.64.0/18 10.36.128.0/18 10.36.192.0/18 10.37.0.0/18 10.37.64.0/18
Luxembourg 10.40.0.0/18 10.40.64.0/18 10.40.128.0/18 10.40.192.0/18 10.41.0.0/18 10.41.64.0/18
Nantes 10.44.0.0/18 10.44.64.0/18 10.44.128.0/18 10.44.192.0/18 10.45.0.0/18 10.45.64.0/18
Louvain 10.48.0.0/18 10.48.64.0/18 10.48.128.0/18 10.48.192.0/18 10.49.0.0/18 10.49.64.0/18
Note

At the end of each network, address x.x.x.253 is used by Kavlan server

Global VLANs
Site Global Vlan Subnet Router IP
Bordeaux KAVLAN-10 10.3.192.0/18 10.3.255.254
Grenoble KAVLAN-11 10.7.192.0/18 10.7.255.254
Lille KAVLAN-12 10.11.192.0/18 10.11.255.254
Lyon KAVLAN-13 10.15.192.0/18 10.15.255.254
Nancy KAVLAN-14 10.19.192.0/18 10.19.255.254
Orsay KAVLAN-15 10.23.192.0/18 10.23.255.254
Rennes KAVLAN-16 10.27.192.0/18 10.27.255.254
Toulouse KAVLAN-17 10.31.192.0/18 10.31.255.254
Sophia KAVLAN-18 10.35.192.0/18 10.35.255.254
Strasbourg KAVLAN-19 10.39.192.0/18 10.39.255.254
Luxembourg KAVLAN-20 10.43.192.0/18 10.43.255.254
Nantes KAVLAN-21 10.47.192.0/18 10.47.255.254
Louvain KAVLAN-22 10.51.192.0/18 10.51.255.254
IP subnet assignments for the sites within a global VLANs

A global VLAN is a /18 subnet (16382 IP addresses). It is split so that every site gets one /23 (510 ip) in the global VLAN address space.

Example for the global VLAN of Lille, KAVLAN-12, whose address space is 10.11.192.0/18:

  • Bordeaux: 10.11.192.1 → 10.11.193.254
  • Grenoble: 10.11.194.1 → 10.11.195.254
  • Lille: 10.11.196.1 → 10.11.197.254
  • Lyon: 10.11.198.1 → 10.11.199.254
  • Nancy: 10.11.200.1 → 10.11.201.254
  • Orsay: 10.11.202.1 → 10.11.203.254
  • Rennes: 10.11.204.1 → 10.11.205.254
  • Toulouse: 10.11.206.1 → 10.11.207.254
  • Sophia: 10.11.208.1 → 10.11.209.254
  • Strasbourg: 10.11.210.1 → 10.11.211.254
  • Luxembourg: 10.11.212.1 → 10.11.213.254
  • Nantes: 10.11.214.1 → 10.11.215.254

KaVLAN-NG networks

During the period of transition from kavlan legacy to kavlanng:

  • maximum 15 IPv4 /14 subnets from 10.188.0.0/14 to 10.244.0.0/14
  • maximum 223 IPv6 /58 subnets from 2001:660:4406:01c0::/58 to 2001:660:4406:dfc0::/58
  • 10.248.0.0/14 and 2001:660:4406:fdc0::/58 for kavlanng devel (test site)

IPv4

Site IPv4 Block
grenoble 10.188.0.0/14
lille 10.192.0.0/14
luxembourg 10.196.0.0/14
lyon 10.200.0.0/14
nancy 10.204.0.0/14
nantes 10.208.0.0/14
rennes 10.212.0.0/14
sophia 10.216.0.0/14
toulouse 10.220.0.0/14
strasbourg 10.224.0.0/14
louvain 10.228.0.0/14
available 10.232.0.0/14 to 10.244.0.0/14 (4 sites)
devel 10.248.0.0/14

IPv6

Site IPv6 Block
grenoble 2001:660:4406:01c0::/58
lille 2001:660:4406:02c0::/58
luxembourg 2001:660:4406:03c0::/58
lyon 2001:660:4406:04c0::/58
nancy 2001:660:4406:05c0::/58
nantes 2001:660:4406:06c0::/58
rennes 2001:660:4406:07c0::/58
sophia 2001:660:4406:08c0::/58
toulouse 2001:660:4406:09c0::/58
strasbourg 2001:660:4406:0ac0::/58
louvain 2001:660:4406:0bc0::/58
available 2001:660:4406:0cc0::/58 to 2001:660:4406:eec0::/58 (227 sites)
devel 2001:660:4406:efc0::/58

Access to the Internet

See FAQ#Is_access_to_the_Internet_possible_from_nodes.3F

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Grid5000:Network&oldid=103570"