Rennes:Network

Network: Global | Grenoble | Lille | Louvain | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse

Physical view of the compute network

Network topology

Experiment Network

• Our network connectivity is provided by a single switch: a Cisco 6509. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).

Management Network

• The service processors of HP Nodes are connected with HP ProCurve 2848 switches.
• The service processors of Sun Nodes are connected with a 10Mb hub.
• Cisco 3750 switches have a 64Gb switching capacity, and they are stacked with a 32Gb bus.

Node names

• Head nodes are assigned to compilation, debugging and running reservations through OAR.
• Compute nodes must be reserved before use with OAR.

Cluster	Head Node from grid'5000	Head Node from IRISA	Compute Nodes
Dell	paraquad-dev.rennes.grid5000.fr (oar.paraquad.rennes.gri5000.fr)	paraquad-dev.irisa.fr	paraquad{01..64}.rennes.grid5000.fr
HP	paravent-dev.rennes.grid5000.fr (oar.paravent.rennes.gri5000.fr)	paravent-dev.irisa.fr	paravent{01..99}.rennes.grid5000.fr
Sun	parasol-dev.rennes.grid5000.fr (oar.parasol.rennes.gri5000.fr)	parasol-dev.irisa.fr	parasol{01..64}.rennes.grid5000.fr

Addressing

Les plages d'addresses suivantes sont réservées et utilisées par Rennes :

131.254.202.0/23

Used by the ethernet interface of nodes

192.168.202.0/23

These addresses are not routed (ie they can only be accessed from head nodes at Rennes). They are used for the service processors (aka BMCs) or the parasol, paravent and paraquad nodes.

192.168.200.0/23

Used for IP over High Performance Networks.

The addresses assignement for routed addresses is :

• 131.254.202.0   -> 131.254.202.253 : parasol, and paraquad clusters.
• 131.254.203.2   -> 131.254.203.101 : paravent compute nodes (Ethernet)
• 131.254.203.202 -> 131.254.203.204 : paravent service nodes
• 131.254.203.102 -> 131.254.203.201 : paravent compute nodes (InfiniBand)

The addresses assignement for non routed addresses is :

• 192.168.202.100 -> 192.168.202.165 : parasol service processors
• 192.168.203.2   -> 192.168.203.102 : service processors for paravent compute nodes
• 192.168.203.202 -> 192.168.203.204 : service processors for paravent service nodes

High Performance Networks

InfiniBand on HP nodes (paravent)

paravent01 to paravent66 have a 10GB Infiniband card (a Mellanox MT23108). Infiniband nodes are connected through 7 Voltaire ISR-9024 switches.

Switchs topology :

Note:

• The nodes are connected to ports 1 to 13 (to port 14 for node parvent66).
• There are 2x4 uplinks per switch (ports 17-20 & 20-24).
• The first uplink (ports 17-20) seems to be not very stable.

• OAR property : infiniBand (Monika). Will be standardized with OAR2.
• IP over IB addressing : paravent[01..66].ib (192.168.201.3 à 192.168.201.68)

Myrinet 10G on HP nodes (paravent)

paravent67 to paravent99 have a 10GB Myrinet card.

• Drivers : /opt/mx/
• OAR property : myri (Monika). Will be standardized with OAR2.
• IP over MX addressing : paravent[67..99].myri (192.168.201.69 à 192.168.201.101)