Rennes:Network: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
 
(113 intermediate revisions by 12 users not shown)
Line 1: Line 1:
== Network topology ==
{{Template:Site link|Network}}
[[Image:reseau_35_en.png]]
{{Portal|Network}}
{{Portal|User}}
 
'''See also:''' [[Rennes:Hardware|Hardware description for Rennes]]
 
= Overview of Ethernet network topology  =
 
[[File:RennesNetwork.svg|900px]]
 
{{:Rennes:GeneratedNetwork}}
 
= More details about the network topology (deprecated)=
 
[[Image:Rennes_experiment_network.png]]


* Our network connectivity is provided by a single switch: a [http://www.cisco.com/en/US/products/hw/switches/ps708/ps711/index.html Cisco 6509]. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).
* Our network connectivity is provided by a single switch: a [http://www.cisco.com/en/US/products/hw/switches/ps708/ps711/index.html Cisco 6509]. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).
* The service processors of HP Nodes are connected with [http://www.hp.com/rnd/products/switches/2800_series/overview.htm HP ProCurve 2848] switches.
* The service processors of Sun Nodes are connected with a 10Mb hub.
* [http://www.cisco.com/en/US/products/hw/switches/ps5023/products_data_sheet09186a0080161371.html Cisco 3750] switches have a 64Gb switching capacity, and they are stacked with a 32Gb bus. The uplink of each Cisco 3750 stack is a gigabit optical fiber (They are temporarily retired).


== Node names ==
== Node names ==
* Head nodes are assigned to compilation, debugging and running reservations through [[OAR]].
* Compute nodes must be reserved before use with OAR on <code class="host">frontend.rennes.grid5000.fr</code>
* Compute nodes must be reserved before use with [https://www-secure.irisa.fr/gridprems  GridPrems] or [[OAR]].
 


{| width="100%"
{| width="100%"
Line 17: Line 25:
Cluster
Cluster
!align="left"|
!align="left"|
Head Node from grid'5000
Compute Nodes
Alias
!align="left"|
!align="left"|
Head Node from IRISA
Primary interface
!align="left"|
!align="left"|
Compute Nodes
Secondary interface (kavlan only)
!align="left"|
Third interface (kavlan only)
|-
|| Dell Poweredge R630
|| <code class="host">paravance-{1..72}.rennes.grid5000.fr</code>
|| <code class="host">paravance-{1..72}-eth0.rennes.grid5000.fr</code>
|| <code class="host">paravance-{1..72}-eth1-kavlan-*.rennes.grid5000.fr</code>
||
|-
|-
|| Apple
|| Dell Poweredge R630
|| <code class="host">tartopom-dev.rennes.grid5000.fr</code>
|| <code class="host">parasilo-{1..28}.rennes.grid5000.fr</code>
|| <code class="host">tartopom-dev.irisa.fr</code>
|| <code class="host">parasilo-{1..28}-eth0.rennes.grid5000.fr</code>
|| <code class="host">tartopom{01..32}.rennes.grid5000.fr</code>
|| <code class="host">parasilo-{1..28}-eth1-kavlan-*.rennes.grid5000.fr</code>
||
|-
|-
|| Dell
|| Dell Poweredge C6220
|| <code class="host">paraci-dev.rennes.grid5000.fr</code>
|| <code class="host">paranoia-{1..8}.rennes.grid5000.fr</code>
|| <code class="host">paraci-dev.irisa.fr</code>
|| <code class="host">paranoia-{1..8}-eth0.rennes.grid5000.fr</code>
|| <code class="host">paraci{01..64}.rennes.grid5000.fr</code>
|| <code class="host">paranoia-{1..8}-eth1-kavlan-*.rennes.grid5000.fr</code>
|| <code class="host">paranoia-{1..8}-eth2-kavlan-*.rennes.grid5000.fr</code>
|-
|-
|| HP
|| HP Proliant
|| <code class="host">paravent-dev.rennes.grid5000.fr</code>
|| <code class="host">parapluie-{1..40}.rennes.grid5000.fr</code>
|| <code class="host">paravent-dev.irisa.fr</code>
|| <code class="host">parapluie-{1..40}-eth1.rennes.grid5000.fr</code>
|| <code class="host">paravent{01..99}.rennes.grid5000.fr</code>
||
||
|-
|-
|| Sun
|-
|| <code class="host">parasol-dev.rennes.grid5000.fr</code>
|| Sun Microsystem
|| <code class="host">parasol-dev.irisa.fr</code>
|| <code class="host">parapide-{1..25}.rennes.grid5000.fr</code>
|| <code class="host">parasol{01..64}.rennes.grid5000.fr</code>
|| <code class="host">parapide-{1..25}-eth0.rennes.grid5000.fr</code>
||
||
|}
|}


== High Performance Networks ==


=== 20G Infiniband 4x QDR (parapide, parapluie) ===


= Addressing =
==== Nodes ====
Les plages d'addresses suivantes sont réservées et utilisées par Rennes :
 
; 131.254.202.0/23: Used by the ethernet and infiniband interface of nodes
; 10.254.202.0/23: These addresses are not routed (ie they can only be accessed from nodes at Rennes). They are used for the service processors (aka BMCs) or the <code class="host">parasoleil</code> and <code class="host">paravent</code> nodes.
 
The addresses assignement for routed addresses is :
* <code>131.254.202.0&nbsp;&nbsp; -> 131.254.202.253&nbsp;</code>: <code class="host">parasol</code>, <code class="host">paraci</code>, and <code class="host">tartopom</code> clusters
* <code>131.254.203.2&nbsp;&nbsp; -> 131.254.203.101&nbsp;</code>: <code class="host">paravent</code> compute nodes (Ethernet)
* <code>131.254.203.102 -> 131.254.203.201&nbsp;</code>: <code class="host">paravent</code> compute nodes (InfiniBand)
* <code>131.254.203.202 -> 131.254.203.204&nbsp;</code>: <code class="host">paravent</code> service nodes
 
 
The addresses assignement for non routed addresses is :
* <code>10.254.202.100 -> 10.254.202.165&nbsp;</code>: <code class="host">parasol</code> service processors
* <code>10.254.203.2&nbsp;&nbsp; -> 10.254.203.102&nbsp;</code>: service processors for <code class="host">paravent</code> compute nodes
* <code>10.254.203.202 -> 10.254.203.204&nbsp;</code>: service processors for <code class="host">paravent</code> service nodes


*<code class="host">parapide-1</code> to <code class="host">parapide-25</code> have a 20G Infiniband 4x DDR (ConnectX) card.
*<code class="host">parapluie-1</code> to <code class="host">parapluie-40</code> have a 20G Infiniband 4x DDR (ConnectX) card.


== High-Speed Networks ==
* sources path : <code class="file">/usr/local/src/OFED-1.5</code>
=== InfiniBand on HP nodes (paravent) ===
<code class="host">paravent01</code> to <code class="host">paravent64</code> have a 10GB Infiniband card (a Mellanox MT23108).
Infiniband nodes are connected through 7 [http://www.voltaire.com/isr_9024.htm Voltaire ISR-9024] switches.


Switchs topology :
* OAR property : ib=MT25418.
* IP over IB addressing : parapide-[1..25]-ib0.rennes.grid5000.fr (192.168.200.221 à 192.168.200.245)
* IP over IB addressing : parapluie-[1..40]-ib0.rennes.grid5000.fr (192.168.201.1 à 192.168.201.40)


[[Image:topologie_infiniband.png]]
==== Voltaire Switch ISR2012-056c ====


=== Myrinet on Dell nodes (paraci) ===
* Gateway : 192.168.200.1
Nodes <code class="host">paraci17</code> to <code class="host">paraci32</code> have a 2 gigabit Myrinet network card, that has these characteristics:
* Hardware :
* bus: PCI
* CPU: LANai9.0


These 16 cards are connected to a Myrinet switch [http://www.myricom.com/myrinet/m3switch/e.html M3-E32]. This switch contains 3 M3-SW16-8S daughterboards, each with 8 ports. 8 ports of this switch are currently unused.
L:ISR2012-056c# version show
version: 5.4.0
date:    Mar 19 2009 02:47:23 PM
build Id:958


The driver installed is GM 2.1.19, it is installed in <code class="dir">/stow-i386/gm-2.1.9</code>.
L:ISR2012-056c# front show
Device: HW Version: B02; Serial Number: CN2309001330
sFB-2012 #1: Temperature: 32C; HW Version: A05; Serial Number: CN1409000632
sFB-2012 #2: Temperature: 31C; HW Version: A05; Serial Number: CN1409000631
sFB-2012 #3: Temperature: 30C; HW Version: A05; Serial Number: CN1409000634
sFB-2012 #4: Temperature: 34C; HW Version: A05; Serial Number: CN1409000630
sMB: Temperature: 28C; HW Version: E01; Serial Number: CN2309001258
sFU-8: Temperature: 24C
sFU-4: Temperature: 22C


Myrinet-enabled nodes are tagged with the <code>lanai9</code> property in [[OAR]]. This links display their current state in Monika: [https://helpdesk.grid5000.fr/oar/Rennes/monika-paraci.cgi?props=lanai9%3DYES&Action=Display+nodes+for+these+properties&.cgifields=props monika].
L:ISR2012-056c# rear show
sLB-2024 #1: Temperature: 32C; HW Version: A03; Serial Number: CN1409000554
sRB-20210G #2: Temperature: 35C; HW Version: B00; Serial Number: CN2709001005
sLB-2024 #3: Temperature: 28C; HW Version: A04; Serial Number: CN0610001939

Latest revision as of 17:12, 8 March 2024

Network: Global | Grenoble | Lille | Louvain | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse

See also: Hardware description for Rennes

Overview of Ethernet network topology

RennesNetwork.svg


Network devices models

  • gw: Aruba 8325-48Y8C
  • parasilo-sw-1: Nexus 56128P
  • sw-1-c003: S4128F-ON
  • sw-1: Aruba 8325-48Y8C
  • sw-adm-1: Aruba 6300M
  • swcore-01-rba: WS-C6509-E
  • swdc-a035-01-02-rba: Nexus 5596
  • swdc-c003-01-02-rba: Nexus 5596

More details (including address ranges) are available from the Grid5000:Network page.

More details about the network topology (deprecated)

Rennes experiment network.png

  • Our network connectivity is provided by a single switch: a Cisco 6509. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).

Node names

  • Compute nodes must be reserved before use with OAR on frontend.rennes.grid5000.fr

Cluster

Compute Nodes Alias

Primary interface

Secondary interface (kavlan only)

Third interface (kavlan only)

Dell Poweredge R630 paravance-{1..72}.rennes.grid5000.fr paravance-{1..72}-eth0.rennes.grid5000.fr paravance-{1..72}-eth1-kavlan-*.rennes.grid5000.fr
Dell Poweredge R630 parasilo-{1..28}.rennes.grid5000.fr parasilo-{1..28}-eth0.rennes.grid5000.fr parasilo-{1..28}-eth1-kavlan-*.rennes.grid5000.fr
Dell Poweredge C6220 paranoia-{1..8}.rennes.grid5000.fr paranoia-{1..8}-eth0.rennes.grid5000.fr paranoia-{1..8}-eth1-kavlan-*.rennes.grid5000.fr paranoia-{1..8}-eth2-kavlan-*.rennes.grid5000.fr
HP Proliant parapluie-{1..40}.rennes.grid5000.fr parapluie-{1..40}-eth1.rennes.grid5000.fr
Sun Microsystem parapide-{1..25}.rennes.grid5000.fr parapide-{1..25}-eth0.rennes.grid5000.fr

High Performance Networks

20G Infiniband 4x QDR (parapide, parapluie)

Nodes

  • parapide-1 to parapide-25 have a 20G Infiniband 4x DDR (ConnectX) card.
  • parapluie-1 to parapluie-40 have a 20G Infiniband 4x DDR (ConnectX) card.
  • sources path : /usr/local/src/OFED-1.5
  • OAR property : ib=MT25418.
  • IP over IB addressing : parapide-[1..25]-ib0.rennes.grid5000.fr (192.168.200.221 à 192.168.200.245)
  • IP over IB addressing : parapluie-[1..40]-ib0.rennes.grid5000.fr (192.168.201.1 à 192.168.201.40)

Voltaire Switch ISR2012-056c

  • Gateway : 192.168.200.1
  • Hardware :
L:ISR2012-056c# version show
version: 5.4.0
date:    Mar 19 2009 02:47:23 PM
build Id:958

L:ISR2012-056c# front show 
Device: HW Version: B02; Serial Number: CN2309001330
sFB-2012 #1: Temperature: 32C; HW Version: A05; Serial Number: CN1409000632
sFB-2012 #2: Temperature: 31C; HW Version: A05; Serial Number: CN1409000631
sFB-2012 #3: Temperature: 30C; HW Version: A05; Serial Number: CN1409000634
sFB-2012 #4: Temperature: 34C; HW Version: A05; Serial Number: CN1409000630
sMB: Temperature: 28C; HW Version: E01; Serial Number: CN2309001258
sFU-8: Temperature: 24C
sFU-4: Temperature: 22C

L:ISR2012-056c# rear show
sLB-2024 #1: Temperature: 32C; HW Version: A03; Serial Number: CN1409000554
sRB-20210G #2: Temperature: 35C; HW Version: B00; Serial Number: CN2709001005
sLB-2024 #3: Temperature: 28C; HW Version: A04; Serial Number: CN0610001939
Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Rennes:Network&oldid=102263"