Nancy:Network
Jump to navigation
Jump to search
Network: Global | Grenoble | Lille | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse
IP networks in use
You have to use a public network range to run an experiment between several Grid5000 sites.
Public Networks
- computing : 172.28.52.0/22
- virtual : 10.144.0.0/14
Local Networks
- ipmi : 172.28.152.0/22
- ib : 192.168.210.0/24 : see High Performance Networks
Network
Production Network
Physical link details
The following diagram show a view of the central router, a hp procurve 5406zl, named sgravillon1.
- All internal links are 10GB CX4.
- Renater link is 10GB optical fiber.
A1 | A3 | A5 | A7 | A9 | A11 | A13 | A15 | A17 | A19 | A21 | A23 |
---|---|---|---|---|---|---|---|---|---|---|---|
fgravillon2-NIC1 | Nc (fgravillon1-NIC2) | fgriffon3-ipmi | fgriffon4-ipmi | fgrelon1-NIC2 | fgriffon1-Prod (fgrelon2-NIC1) | sgrenade1 | sgrenade3 | sgrenade5 | sgriffon2-ipmi | sgriffon3-ipmi | fgriffon1-Mgt |
A2 | A4 | A6 | A8 | A10 | A12 | A14 | A16 | A18 | A20 | A22 | A24 |
fgriffon2-ipmi | Nc (fgravillon1-NIC1) | fgravillon2-NIC2 | fgrelon1-NIC1 | fgriffon1-ipmi | Nc (fgrelon2-NIC2) | sgrenade2 | sgrenade4 | LORIA | sgriffon1-ipmi | fgriffon3-eth0 | fgriffon3-eth1 |
High Performance Networks
Infiniband 20G on griffon nodes
Nodes
griffon-1
togriffon-92
have one 20GB Infiniband card.
- Card Model : Mellanox Technologies MT26418 [ConnectX IB DDR, PCIe 2.0 5GT/s] ( ConnectX ).
- Driver :
mlx4_ib
- OAR property : ib20g=YES ( Monika ).
- IP over IB addressing :
griffon-[1..92]-ib
.nancy.grid5000.fr ( 192.168.210.[1..92] )
Switch
- Infiniband Switch 4X DDR
- Model based on Infiniscale_III
- 1 commutation card Flextronics F-X43M204
- 11 line cards 4X 12 ports DDR Flextronics F-X43M203
Interconnection
Infiniband network is physically isolated from Ethernet networks. Therefore, Ethernet network emulated over Infiniband is isolated as well. There isn't any interconnexion, neither at the data link layer nor at the network layer.
Myrinet 10G on griffon nodes
Nodes
griffon-11
togriffon-14
have one 10GB Myrinet card.
- Card Model : Myrinet 10G ( [1] ) 10G-PCIE-8B-C NIC
- Driver :
myri10ge
- OAR property : ib20g=YES ( Monika ).
- Ethernet. IP addresses :
griffon-[11..14]-myri0
.nancy.grid5000.fr ( 172.28.55.[221..224] )
Interconnection
The nodes having myrinet configured are connected to the 10G Ethernet interfaces of the switches sgriffon|1-3] as following:
- griffon-11 to sgriffon1
- griffon-12 to sgriffon1 (the configuration is not working probably due to the switch configuration)
- griffon-13 to sgriffon2
- griffon-14 to sgriffon3
Grid5000 interconnect
Interconnect type
Link details
Note: All the used fiber cables are dedicated to our Grid'5000 interconnect. When it is said trunk on the above figure it means a more rigid garter.