Grid5000:Network: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
No edit summary
(25 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{Portal|Network}}
{{Portal|Network}}
{{Status|Approved}}
{{Portal|User}}
{{Maintainer|Florian Le Goff}}
{{Maintainer|Lucas Nussbaum}}
{{Template:Site link|Network}}
{{Template:Site link|Network}}
__NOEDITSECTION__
__NOEDITSECTION__


=Grid'5000 Backbone Network Infrastructure=
{{Note|text=
This page provides an overview of Grid'5000 network infrastructure. Other useful pages:
* [[Status#Network_Monitoring|'''Network monitoring section of the Status page''']]
* [[Virtualization on Grid'5000]] tutorial (provides introduction to [[Subnet reservation]])
* [[Network reconfiguration tutorial]] (provides introduction to [[KaVLAN]]; there's also another [[KaVLAN use cases|more advanced tutorial]])
}}
 
=Grid'5000 Network Infrastructure=
==Backbone Network==
 
[[Image:Logo RENATER.png|150px|left]]
[[Image:Logo RENATER.png|150px|left]]
[[Image:renater5.jpg|251px|right|thumbnail|Renater's WDM infrastructure]]
Grid'5000 backbone network infrastructure is provided by '''[http://www.renater.fr RENATER]'''. '''RENATER''' is the French National Telecommunication Network for Technology, Education and Research. More information can be found on [http://www.renater.fr renater.fr]. '''[http://www.renater.fr RENATER]''' offers about 30 POPs (Points Of Presence) in France, at least one POP for each region, which metropolitan and regional networks are connected on. More than 600 sites (Universities, Research Centers, ..) are interconnected through '''[http://www.renater.fr RENATER]'''.
Grid'5000 backbone network infrastructure is provided by '''[http://www.renater.fr RENATER]'''. '''RENATER''' is the French National Telecommunication Network for Technology, Education and Research. More information can be found on [http://www.renater.fr renater.fr].


[[Image:renater5.jpg|251px|thumbnail|Renater's WDM infrastructure]]
[[Image:renater5-g5k.jpg|250px|thumbnail|10Gbps lambda activated for Grid'5000 in Renater's Dark Fiber infrastructure]]
[[Image:Schema_Backbone.png|251px|thumbnail|Grid5000 IP Network]]


'''[http://www.renater.fr RENATER]''' offers about 30 POPs (Points Of Presence) in France, at least one POP for each region, which metropolitan and regional networks are connected on.
More than 600 sites (Universities, Research Centers, ..) are interconnected through '''[http://www.renater.fr RENATER]'''.


The actual phase of the network is '''RENATER-5''', the deployment has been completed by January 2009.
The actual phase of the network is '''RENATER-5''', the deployment has been completed by January 2009.
The "standard" architecture is based on '''10Gbit/s''' dark fibers and provides IP transit connectivity, interconnection with GEANT-2 (http://www.geant2.net/), overseas territories and the SFINX (Global Internet exchange).
The "standard" architecture is based on '''10Gbit/s''' dark fibers and provides IP transit connectivity, interconnection with GEANT-2 (http://www.geant2.net/), overseas territories and the SFINX (Global Internet exchange).


For more information on the Grid5000 Backbone Network Architecture, please check the [[Network_interlink]] page.


==Grid5000 Project==
[[Image:renater5-g5k.jpg|250px|right|thumbnail|10Gbps lambda activated for Grid'5000 in Renater's Dark Fiber infrastructure]]
[[Image:Schema_Backbone.png|251px|right|thumbnail|Grid5000 IP Network]]
=== Original Design : EoMPLS tunnels ===


The initial design of Grid'5000 sites interconnection has been adressed within the RENATER backbone using a Ethernet Over MPLS (EoMPLS) solution. It used to be full mesh topology based on MPLS tunnels (LSPs) established between the RENATER PoPs on which are connected the Grid'5000 sites. Sites were interconnected through 1Gbit/s VLANs.
All Grid'5000 sites (with the exception of Nantes) are connected to dedicated 10Gbit/s ''lambdas'' on a DWDM infrastructure (see [http://en.wikipedia.org/wiki/Wavelength-division_multiplexing Wikipedia: Wavelength-division Multiplexing] for more info).


=== Current design : Dark Fibre ===
Nantes is connected using an L2 VPN.


RENATER-4 introduced a dark fibre infrastructure allowing to allocate dedicated 10Gbit/s "lambdas" for specific research projects. It also provides interconnection with GEANT-2 (see picture above), with increased capacity compared to GEANT-1 and dedicated interconnection for projects. RENATER-5 has enhanced the dark fiber infrastructure and all Grid'5000 sites are now connected to it.


Grid'5000 sites see each others inside the same VLAN at 10Gbps. A few few bottleneck still exists, like the Lyon to Paris link, where the 10Gbps are shared between all the sites above Lyon and all the sites under Lyon.


Please check [[Renater5]] for more information on the current network topology.
==Grid'5000 sites networks==


=== Monitoring and Metrology ===
Grid'5000 sites provide access to a variety of network technologies (both Ethernet and high-performance networks such as InfiniBand).
 
More information is available on the following pages:
You can find the monitoring and metrology (SNMP Graphs, Smokeping, Weathermap...) performed on Grid5000 by Renater and the Grid5000 staff on the [[Network_interlink#Monitoring]] section.
* [[Grenoble:Network|Grenoble]]
* [[Lille:Network|Lille]]
* [[Luxembourg:Network|Luxembourg]]
* [[Lyon:Network|Lyon]]
* [[Nancy:Network|Nancy]]
* [[Nantes:Network|Nantes]]
* [[Rennes:Network|Rennes]]
* [[Sophia:Network|Sophia]]


=== International Extensions ===
The list of high performance networks is available on [[Hardware#Network_interconnects|this page]].


We share a direct connection with the following networks :
=Address space for virtual machines=
Grid'5000 has two different mechanisms to provide IP addresses to virtual machines: subnet reservation and KaVLAN.
* '''Subnet reservation''' provides a mean to allocate IP addresses in the production (= default) network. This provides a way to avoid conflicts between users, but does not provide isolation: all machines are still in the same L2 network. Subnet reservation is documented in the [[Virtualization_on_Grid'5000|Virtualization on the standard environment]] tutorial, and on the [[Subnet reservation]] page.
* '''KaVLAN''' provides network isolation through dynamic switches and routers reconfiguration, for the duration of an experiment. Inside a KaVLAN network, users are free to allocate IP addresses as needed. However, some IP ranges are routed to/from the KaVLAN network. KaVLAN is documented in the [[Network reconfiguration tutorial]] (which provides an introduction to KaVLAN), in the [[Network isolation on Grid%275000]] tutorial (which is a more advanced KaVLAN tutorial), and on the [[KaVLAN]] page.


* [http://www.cs.vu.nl/das3/index.shtml DAS-3], more information is available on [[DAS3-Grid5000]]
=Tables of available IP networks=
* [http://www.naregi.org NAREGI], more information is available on [[Naregi-Grid5000]]
== Production network and subnet reservation ==
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_Golden_rules/Addressing_plan -->
{{:Network_Golden_rules/Addressing_plan}}


= Grid'5000 Sites =
=== Virtual subnets configuration ===
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_interlink/Virtual -->
{{:Network_interlink/Virtual}}


Each local network topology is different. More information is available to our users on the following pages of the Wiki :
== KaVLAN ==
=== KaVLAN deployment status ===
<!-- https://www.grid5000.fr/mediawiki/index.php/KaVLAN/Version -->
{{:KaVLAN/Version}}


* [[Bordeaux:Network|Bordeaux]]
=== KaVLAN networks ===
* [[Grenoble:Network|Grenoble]]
<!-- https://www.grid5000.fr/mediawiki/index.php/Network_Golden_rules/KaVLAN -->
* [[Lille:Network|Lille]]
{{:Network_Golden_rules/KaVLAN}}
* [[Lyon:Network|Lyon]]
* [[Nancy:Network|Nancy]]
* [[Orsay:Network|Orsay]]
* [[Rennes:Network|Rennes]]
* [[Sophia:Network|Sophia-Antipolis]]
* [[Toulouse:Network|Toulouse]]


== High Speed Networks==
= Access to the Internet =


Grid'5000 features both Myrinet and Infiniband network infrastructures. Grid5000 users may check the different cards available on [[Special:G5KHardware#High_performance_network_families|this page]].
See [[FAQ#Is_access_to_the_Internet_possible_from_nodes.3F]]

Revision as of 13:37, 16 February 2018


Note.png Note

This page provides an overview of Grid'5000 network infrastructure. Other useful pages:

Grid'5000 Network Infrastructure

Backbone Network

Logo RENATER.png

Grid'5000 backbone network infrastructure is provided by RENATER. RENATER is the French National Telecommunication Network for Technology, Education and Research. More information can be found on renater.fr. RENATER offers about 30 POPs (Points Of Presence) in France, at least one POP for each region, which metropolitan and regional networks are connected on. More than 600 sites (Universities, Research Centers, ..) are interconnected through RENATER.

Renater's WDM infrastructure
10Gbps lambda activated for Grid'5000 in Renater's Dark Fiber infrastructure
Grid5000 IP Network


The actual phase of the network is RENATER-5, the deployment has been completed by January 2009. The "standard" architecture is based on 10Gbit/s dark fibers and provides IP transit connectivity, interconnection with GEANT-2 (http://www.geant2.net/), overseas territories and the SFINX (Global Internet exchange).


All Grid'5000 sites (with the exception of Nantes) are connected to dedicated 10Gbit/s lambdas on a DWDM infrastructure (see Wikipedia: Wavelength-division Multiplexing for more info).

Nantes is connected using an L2 VPN.


Grid'5000 sites networks

Grid'5000 sites provide access to a variety of network technologies (both Ethernet and high-performance networks such as InfiniBand). More information is available on the following pages:


The list of high performance networks is available on this page.

Address space for virtual machines

Grid'5000 has two different mechanisms to provide IP addresses to virtual machines: subnet reservation and KaVLAN.

  • Subnet reservation provides a mean to allocate IP addresses in the production (= default) network. This provides a way to avoid conflicts between users, but does not provide isolation: all machines are still in the same L2 network. Subnet reservation is documented in the Virtualization on the standard environment tutorial, and on the Subnet reservation page.
  • KaVLAN provides network isolation through dynamic switches and routers reconfiguration, for the duration of an experiment. Inside a KaVLAN network, users are free to allocate IP addresses as needed. However, some IP ranges are routed to/from the KaVLAN network. KaVLAN is documented in the Network reconfiguration tutorial (which provides an introduction to KaVLAN), in the Network isolation on Grid'5000 tutorial (which is a more advanced KaVLAN tutorial), and on the KaVLAN page.

Tables of available IP networks

Production network and subnet reservation

Here are the private IPv4 networks allocated to every site:

Site State Production network Administration network High Perf network (ib/opa) Kavlan network Virtualization
Bordeaux Fail.png 172.16.0.0/20 172.17.0.0/20 172.18.0.0/20 10.0.0.0/14 10.128.0.0/14
Grenoble Check.png 172.16.16.0/20 172.17.16.0/20 172.18.16.0/20 10.4.0.0/14 10.132.0.0/14
Lille Check.png 172.16.32.0/20 172.17.32.0/20 172.18.32.0/20 10.8.0.0/14 10.136.0.0/14
Lyon Check.png 172.16.48.0/20 172.17.48.0/20 172.18.48.0/20 10.12.0.0/14 10.140.0.0/14
Nancy Check.png 172.16.64.0/20 172.17.64.0/20 172.18.64.0/20 10.16.0.0/14 10.144.0.0/14
Orsay Fail.png 172.16.80.0/20 172.17.80.0/20 172.18.80.0/20 10.20.0.0/14 10.148.0.0/14 + 10.152.0.0/14
Rennes Check.png 172.16.96.0/20 172.17.96.0/20 172.18.96.0/20 10.24.0.0/14 10.156.0.0/14
Toulouse Check.png 172.16.112.0/20 172.17.112.0/20 172.18.112.0/20 10.28.0.0/14 10.160.0.0/14
Sophia Check.png 172.16.128.0/20 172.17.128.0/20 172.18.128.0/20 10.32.0.0/14 10.164.0.0/14
Strasbourg (ex Reims) Fail.png 172.16.160.0/20 172.17.160.0/20 172.18.160.0/20 10.36.0.0/14 10.168.0.0/14
Luxembourg Check.png 172.16.176.0/20 172.17.176.0/20 172.18.176.0/20 10.40.0.0/14 10.172.0.0/14
Nantes Check.png 172.16.192.0/20 172.17.192.0/20 172.18.192.0/20 10.44.0.0/14 10.176.0.0/14
Orléans Fail.png 172.16.208.0/20 172.17.208.0/20 172.18.208.0/20
Qualification Fail.png 172.16.240.0/20 172.17.240.0/20 172.18.240.0/20 10.252.0.0/14

Here are the IPv6 networks allocated to every site (these are global scope networks, thus routable on the internet):

Site State Production network Administration network High Perf network (ib/opa) Kavlan networks Virtualization
Grenoble Check.png 2001:0660:4406:0100::/64 2001:0660:4406:0110::/64 2001:0660:4406:0120::/64 2001:0660:4406:0180::/58 TBD
Lille Check.png 2001:0660:4406:0200::/64 2001:0660:4406:0210::/64 2001:0660:4406:0220::/64 2001:0660:4406:0280::/58 TBD
Luxembourg Check.png 2001:0660:4406:0300::/64 2001:0660:4406:0310::/64 2001:0660:4406:0320::/64 2001:0660:4406:0380::/58 TBD
Lyon Check.png 2001:0660:4406:0400::/64 2001:0660:4406:0410::/64 2001:0660:4406:0420::/64 2001:0660:4406:0480::/58 TBD
Nancy Check.png 2001:0660:4406:0500::/64 2001:0660:4406:0510::/64 2001:0660:4406:0520::/64 2001:0660:4406:0580::/58 TBD
Nantes Check.png 2001:0660:4406:0600::/64 2001:0660:4406:0610::/64 2001:0660:4406:0620::/64 2001:0660:4406:0680::/58 TBD
Rennes Check.png 2001:0660:4406:0700::/64 2001:0660:4406:0710::/64 2001:0660:4406:0720::/64 2001:0660:4406:0780::/58 TBD
Sophia Check.png 2001:0660:4406:0800::/64 2001:0660:4406:0810::/64 2001:0660:4406:0820::/64 2001:0660:4406:0880::/58 TBD
Toulouse Check.png 2001:0660:4406:0900::/64 2001:0660:4406:0910::/64 2001:0660:4406:0920::/64 2001:0660:4406:0980::/58 TBD
Strasbourg Fail.png 2001:0660:4406:0a00::/64 2001:0660:4406:0a10::/64 2001:0660:4406:0a20::/64 2001:0660:4406:0a80::/58 TBD

Virtual subnets configuration

Site Block Gateway DHCP server
Bordeaux 10.128.0.0/14 10.131.255.254 10.131.255.253
Grenoble 10.132.0.0/14 10.135.255.254 10.135.255.253
Lille 10.136.0.0/14 10.139.255.254 10.139.255.253
Lyon 10.140.0.0/14 10.143.255.254 10.143.255.253
Nancy 10.144.0.0/14 10.147.255.254 10.147.255.253
Orsay I 10.148.0.0/14 10.151.255.254 10.151.255.253
Orsay II 10.152.0.0/14 10.155.255.254 10.155.255.253
Rennes 10.156.0.0/14 10.159.255.254 10.159.255.253
Toulouse 10.160.0.0/14 10.163.255.254 10.163.255.253
Sophia 10.164.0.0/14 10.167.255.254 10.167.255.253
Reims 10.168.0.0/14 10.171.255.254 10.171.255.253
Luxembourg 10.172.0.0/14 10.175.255.254 10.175.255.253
Nantes 10.176.0.0/14 10.179.255.254 10.179.255.253
VPN nets n/a n/a
Qualif 10.252.0.0/14 10.255.255.254 10.255.255.253

KaVLAN

KaVLAN deployment status

Sites Version Status
Grenoble 1.2.7-1 Check.png
Lille 1.2.7-1 Check.png
Luxembourg 1.2.7-1 Check.png
Lyon 1.2.7-1 Check.png
Nancy 1.2.7-1 Check.png
Nantes 1.2.7-1 Check.png
Rennes 1.2.7-1 Check.png
Sophia 1.2.7-1 Check.png

KaVLAN networks

Local VLANs (non-routed)
Site KAVLAN-1 KAVLAN-2 KAVLAN-3
All 192.168.192.0/20 192.168.208.0/20 192.168.224.0/20
Routed VLANs
Site KAVLAN-4 KAVLAN-5 KAVLAN-6 KAVLAN-7 KAVLAN-8 KAVLAN-9
Bordeaux 10.0.0.0/18 10.0.64.0/18 10.0.128.0/18 10.0.192.0/18 10.1.0.0/18 10.1.64.0/18
Grenoble 10.4.0.0/18 10.4.64.0/18 10.4.128.0/18 10.4.192.0/18 10.5.0.0/18 10.5.64.0/18
Lille 10.8.0.0/18 10.8.64.0/18 10.8.128.0/18 10.8.192.0/18 10.9.0.0/18 10.9.64.0/18
Lyon 10.12.0.0/18 10.12.64.0/18 10.12.128.0/18 10.12.192.0/18 10.13.0.0/18 10.13.64.0/18
Nancy 10.16.0.0/18 10.16.64.0/18 10.16.128.0/18 10.16.192.0/18 10.17.0.0/18 10.17.64.0/18
Orsay 10.20.0.0/18 10.20.64.0/18 10.20.128.0/18 10.20.192.0/18 10.21.0.0/18 10.21.64.0/18
Rennes 10.24.0.0/18 10.24.64.0/18 10.24.128.0/18 10.24.192.0/18 10.25.0.0/18 10.25.64.0/18
Toulouse 10.28.0.0/18 10.28.64.0/18 10.28.128.0/18 10.28.192.0/18 10.29.0.0/18 10.29.64.0/18
Sophia 10.32.0.0/18 10.32.64.0/18 10.32.128.0/18 10.32.192.0/18 10.33.0.0/18 10.33.64.0/18
Strasbourg Reims 10.36.0.0/18 10.36.64.0/18 10.36.128.0/18 10.36.192.0/18 10.37.0.0/18 10.37.64.0/18
Luxembourg 10.40.0.0/18 10.40.64.0/18 10.40.128.0/18 10.40.192.0/18 10.41.0.0/18 10.41.64.0/18
Nantes 10.44.0.0/18 10.44.64.0/18 10.44.128.0/18 10.44.192.0/18 10.45.0.0/18 10.45.64.0/18
Note.png Note

At the end of each network, address x.x.x.253 is used by Kavlan server

Global VLANs
Site Global Vlan Subnet Router IP
Bordeaux KAVLAN-10 10.3.192.0/18 10.3.255.254
Grenoble KAVLAN-11 10.7.192.0/18 10.7.255.254
Lille KAVLAN-12 10.11.192.0/18 10.11.255.254
Lyon KAVLAN-13 10.15.192.0/18 10.15.255.254
Nancy KAVLAN-14 10.19.192.0/18 10.19.255.254
Orsay KAVLAN-15 10.23.192.0/18 10.23.255.254
Rennes KAVLAN-16 10.27.192.0/18 10.27.255.254
Toulouse KAVLAN-17 10.31.192.0/18 10.31.255.254
Sophia KAVLAN-18 10.35.192.0/18 10.35.255.254
Strasbourg Reims KAVLAN-19 10.39.192.0/18 10.39.255.254
Luxembourg KAVLAN-20 10.43.192.0/18 10.43.255.254
Nantes KAVLAN-21 10.47.192.0/18 10.47.255.254
IP subnet assignments for the sites within a global VLANs

A global VLAN is a /18 subnet (16382 IP addresses). It is split so that every site gets one /23 (510 ip) in the global VLAN address space.

Example for the global VLAN of Lille, KAVLAN-12, whose address space is 10.11.192.0/18:

  • Bordeaux: 10.11.192.110.11.193.254
  • Grenoble: 10.11.194.110.11.195.254
  • Lille: 10.11.196.110.11.197.254
  • Lyon: 10.11.198.110.11.199.254
  • Nancy: 10.11.200.110.11.201.254
  • Orsay: 10.11.202.110.11.203.254
  • Rennes: 10.11.204.110.11.205.254
  • Toulouse: 10.11.206.110.11.207.254
  • Sophia: 10.11.208.110.11.209.254
  • Strasbourg Reims: 10.11.210.110.11.211.254
  • Luxembourg: 10.11.212.110.11.213.254
  • Nantes: 10.11.214.110.11.215.254

Access to the Internet

See FAQ#Is_access_to_the_Internet_possible_from_nodes.3F