Hardware

From Grid5000
Revision as of 12:02, 17 December 2020 by Bjonglez (talk | contribs)

Jump to: navigation, search
Global | Grenoble | Lille | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia

Summary

  • 8 sites
  • 37 clusters
  • 806 nodes
  • 15702 CPU cores
  • 224 GPUs
  • 98.13 TiB RAM + 6.0 TiB PMEM
  • 512 SSDs and 1014 HDDs on nodes (total: 1.42 PB)

Clusters

Site Cluster Access Condition Date of arrival Nodes CPU Cores Memory Storage Network Accelerators
Grenoble dahu 2018-03-22 32 2 x Intel Xeon Gold 6130 16 cores/CPU 192 GiB 240 GB SSD + 480 GB SSD + 4.0 TB HDD 10 Gbps + 100 Gbps Omni-Path
Grenoble drac testing queue,
exotic job type		 2020-10-05 12 2 x POWER8NVL 1.0 10 cores/CPU 128 GiB 1 x 1.0 TB HDD + 1 x 1.0 TB HDD 10 Gbps + 2 x 100 Gbps InfiniBand 4 x Nvidia Tesla P100
Grenoble troll 2019-12-23 4 2 x Intel Xeon Gold 5218 16 cores/CPU 384 GiB + 1.5 TiB PMEM 480 GB SSD + 1.6 TB SSD 10 Gbps + 100 Gbps Omni-Path
Grenoble yeti 2018-01-16 4 4 x Intel Xeon Gold 6130 16 cores/CPU 768 GiB 480 GB SSD + 3 x 2.0 TB HDD* + 1.6 TB SSD 10 Gbps + 100 Gbps Omni-Path
Lille chetemi 2016-12-01 15 2 x Intel Xeon E5-2630 v4 10 cores/CPU 256 GiB [1-9,11-15]: 1 x 300 GB HDD + 1 x 300 GB HDD
10: 600 GB HDD + 300 GB HDD		 2 x 10 Gbps 
Lille chiclet 2018-08-06 8 2 x AMD EPYC 7301 16 cores/CPU 128 GiB 480 GB SSD + 2 x 4.0 TB HDD* 2 x 25 Gbps 
Lille chifflet 2016-12-01 8 2 x Intel Xeon E5-2680 v4 14 cores/CPU 768 GiB 400 GB SSD + 400 GB SSD* + 2 x 4.0 TB HDD* 2 x 10 Gbps  2 x Nvidia GTX 1080 Ti
Lille chifflot 2018-08-01 8 2 x Intel Xeon Gold 6126 12 cores/CPU 192 GiB 480 GB SSD + 480 GB SSD* + 4 x 4.0 TB HDD* 2 x 25 Gbps  [1-6]: 2 x Nvidia Tesla P100
[7-8]: 2 x Nvidia Tesla V100
Luxembourg petitprince 2013-09-10 15 2 x Intel Xeon E5-2630L 6 cores/CPU 32 GiB 250 GB HDD 2 x 10 Gbps 
Lyon gemini 2019-09-01 2 2 x Intel Xeon E5-2698 v4 20 cores/CPU 512 GiB 480 GB SSD + 4 x 1.92 TB SSD* 10 Gbps + 3 x 100 Gbps InfiniBand 8 x Nvidia Tesla V100
Lyon hercule 2012-10-02 4 2 x Intel Xeon E5-2620 6 cores/CPU 32 GiB 1 x 2.0 TB HDD + 2 x 2.0 TB HDD 10 Gbps 
Lyon nova 2016-12-01 23 2 x Intel Xeon E5-2620 v4 8 cores/CPU 64 GiB 598 GB HDD 10 Gbps 
Lyon orion 2012-09-14 4 2 x Intel Xeon E5-2630 6 cores/CPU 32 GiB 598 GB HDD 10 Gbps  Nvidia Tesla M2075
Lyon pyxis exotic job type 2020-01-06 4 2 x ThunderX2 99xx 32 cores/CPU 256 GiB 1 x 250 GB SSD + 1 x 250 GB SSD 10 Gbps + 100 Gbps InfiniBand
Lyon sagittaire 2006-07-01 21 2 x AMD Opteron 250 1 core/CPU 2 GiB 73 GB HDD 1 Gbps 
Lyon taurus 2012-09-14 14 2 x Intel Xeon E5-2630 6 cores/CPU 32 GiB 598 GB HDD 10 Gbps 
Nancy graffiti production queue 2019-06-07 13 2 x Intel Xeon Silver 4110 8 cores/CPU 128 GiB 479 GB SSD 10 Gbps  4 x Nvidia RTX 2080 Ti
Nancy graoully production queue 2016-01-04 16 2 x Intel Xeon E5-2630 v3 8 cores/CPU 128 GiB 1 x 600 GB HDD + 1 x 600 GB HDD 10 Gbps + 56 Gbps InfiniBand
Nancy graphique production queue 2015-05-12 6 2 x Intel Xeon E5-2620 v3 6 cores/CPU 64 GiB 299 GB HDD 10 Gbps + 56 Gbps InfiniBand 1: 2 x Nvidia Titan Black
[2-6]: 2 x Nvidia GTX 980
Nancy graphite 2013-12-05 4 2 x Intel Xeon E5-2650 8 cores/CPU 256 GiB 1 x 300 GB SSD + 1 x 300 GB SSD 10 Gbps + 56 Gbps InfiniBand Intel Xeon Phi 7120P
Nancy grappe production queue 2020-08-20 16 2 x Intel Xeon Gold 5218R 20 cores/CPU 96 GiB 480 GB SSD + 8.0 TB HDD* 25 Gbps 
Nancy grcinq production queue 2013-04-09 47 2 x Intel Xeon E5-2650 8 cores/CPU 64 GiB 1.0 TB HDD 1 Gbps + 56 Gbps InfiniBand
Nancy grele production queue 2017-06-26 14 2 x Intel Xeon E5-2650 v4 12 cores/CPU 128 GiB 1 x 299 GB HDD + 1 x 299 GB HDD 10 Gbps + 100 Gbps Omni-Path 2 x Nvidia GTX 1080 Ti
Nancy grimani production queue 2016-08-30 6 2 x Intel Xeon E5-2603 v3 6 cores/CPU 64 GiB 1.0 TB HDD 10 Gbps + 100 Gbps Omni-Path 2 x Nvidia Tesla K40M
Nancy grimoire 2016-01-22 8 2 x Intel Xeon E5-2630 v3 8 cores/CPU 128 GiB 600 GB HDD + 4 x 600 GB HDD* + 200 GB SSD* 4 x 10 Gbps + 56 Gbps InfiniBand
Nancy grisou 2016-01-04 51 2 x Intel Xeon E5-2630 v3 8 cores/CPU 128 GiB 1 x 600 GB HDD + 1 x 600 GB HDD [1-48]: 1 Gbps + 4 x 10 Gbps 
49: 4 x 10 Gbps 
[50-51]: 4 x 10 Gbps + 56 Gbps InfiniBand
Nancy gros 2019-09-04 124 Intel Xeon Gold 5220 18 cores/CPU 96 GiB 480 GB SSD + 960 GB SSD* 2 x 25 Gbps 
Nancy grue production queue 2019-11-25 5 2 x AMD EPYC 7351 16 cores/CPU 128 GiB 479 GB SSD 10 Gbps  4 x Nvidia Tesla T4
Nancy grvingt production queue 2018-04-11 64 2 x Intel Xeon Gold 6130 16 cores/CPU 192 GiB 1.0 TB HDD 10 Gbps + 100 Gbps Omni-Path
Nantes econome 2014-04-16 22 2 x Intel Xeon E5-2660 8 cores/CPU 64 GiB 2.0 TB HDD 10 Gbps 
Nantes ecotype 2017-10-16 48 2 x Intel Xeon E5-2630L v4 10 cores/CPU 128 GiB 400 GB SSD 2 x 10 Gbps 
Rennes paranoia 2014-02-21 8 2 x Intel Xeon E5-2660 v2 10 cores/CPU 128 GiB 1 x 600 GB HDD + 4 x 600 GB HDD 1 Gbps + 2 x 10 Gbps 
Rennes parapide 2010-01-25 17 2 x Intel Xeon X5570 4 cores/CPU 24 GiB 500 GB HDD 1 Gbps + 20 Gbps InfiniBand
Rennes parapluie 2010-11-02 16 2 x AMD Opteron 6164 HE 12 cores/CPU 48 GiB 250 GB HDD 1 Gbps + 20 Gbps InfiniBand
Rennes parasilo 2015-01-13 28 2 x Intel Xeon E5-2630 v3 8 cores/CPU 128 GiB 600 GB HDD + 4 x 600 GB HDD* + 200 GB SSD* 2 x 10 Gbps 
Rennes paravance 2015-01-13 72 2 x Intel Xeon E5-2630 v3 8 cores/CPU 128 GiB 1 x 600 GB HDD + 1 x 600 GB HDD 2 x 10 Gbps 
Sophia uvb 2011-01-04 43 2 x Intel Xeon X5670 6 cores/CPU 96 GiB 250 GB HDD 1 Gbps + 40 Gbps InfiniBand

Processors

Processors counts per families

Processor family Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Processors total
AMD EPYC 16 10 26
AMD Opteron 42 32 74
Intel Xeon 88 62 30 94 614 140 250 86 1364
POWER8NVL 24 24
ThunderX2 8 8
Sites total 112 78 30 144 624 140 282 86 1496

Processors counts per models

Microarchitecture Processor model Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Processors total
K8 AMD Opteron 250 42 42
K10 AMD Opteron 6164 HE 32 32
Nehalem Intel Xeon X5570 34 34
Westmere Intel Xeon X5670 86 86
Sandy Bridge Intel Xeon E5-2620 8 8
Sandy Bridge Intel Xeon E5-2630 36 36
Sandy Bridge Intel Xeon E5-2630L 30 30
Sandy Bridge Intel Xeon E5-2650 102 102
Sandy Bridge Intel Xeon E5-2660 44 44
Sandy Bridge Intel Xeon E5-2660 v2 16 16
Haswell Intel Xeon E5-2603 v3 12 12
Haswell Intel Xeon E5-2620 v3 12 12
Haswell Intel Xeon E5-2630 v3 150 200 350
POWER8 POWER8NVL 1.0 24 24
Broadwell Intel Xeon E5-2620 v4 46 46
Broadwell Intel Xeon E5-2630 v4 30 30
Broadwell Intel Xeon E5-2630L v4 96 96
Broadwell Intel Xeon E5-2650 v4 28 28
Broadwell Intel Xeon E5-2680 v4 16 16
Broadwell Intel Xeon E5-2698 v4 4 4
Skylake Intel Xeon Gold 6126 16 16
Skylake Intel Xeon Gold 6130 80 128 208
Skylake Intel Xeon Silver 4110 26 26
Zen AMD EPYC 7301 16 16
Zen AMD EPYC 7351 10 10
Vulcan ThunderX2 99xx 8 8
Cascade Lake-SP Intel Xeon Gold 5218 8 8
Cascade Lake-SP Intel Xeon Gold 5218R 32 32
Cascade Lake-SP Intel Xeon Gold 5220 124 124
Sites total 112 78 30 144 624 140 282 86 1496

Cores counts per models

Microarchitecture Core model Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Cores total
K8 AMD Opteron 250 42 42
K10 AMD Opteron 6164 HE 384 384
Nehalem Intel Xeon X5570 136 136
Westmere Intel Xeon X5670 516 516
Sandy Bridge Intel Xeon E5-2620 48 48
Sandy Bridge Intel Xeon E5-2630 216 216
Sandy Bridge Intel Xeon E5-2630L 180 180
Sandy Bridge Intel Xeon E5-2650 816 816
Sandy Bridge Intel Xeon E5-2660 352 352
Sandy Bridge Intel Xeon E5-2660 v2 160 160
Haswell Intel Xeon E5-2603 v3 72 72
Haswell Intel Xeon E5-2620 v3 72 72
Haswell Intel Xeon E5-2630 v3 1200 1600 2800
POWER8 POWER8NVL 1.0 240 240
Broadwell Intel Xeon E5-2620 v4 368 368
Broadwell Intel Xeon E5-2630 v4 300 300
Broadwell Intel Xeon E5-2630L v4 960 960
Broadwell Intel Xeon E5-2650 v4 336 336
Broadwell Intel Xeon E5-2680 v4 224 224
Broadwell Intel Xeon E5-2698 v4 80 80
Skylake Intel Xeon Gold 6126 192 192
Skylake Intel Xeon Gold 6130 1280 2048 3328
Skylake Intel Xeon Silver 4110 208 208
Zen AMD EPYC 7301 256 256
Zen AMD EPYC 7351 160 160
Vulcan ThunderX2 99xx 256 256
Cascade Lake-SP Intel Xeon Gold 5218 128 128
Cascade Lake-SP Intel Xeon Gold 5218R 640 640
Cascade Lake-SP Intel Xeon Gold 5220 2232 2232
Sites total 1648 972 180 1010 7784 1312 2280 516 15702

Memory

RAM size per node

RAM size Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Nodes total
2 GiB 21 21
24 GiB 17 17
32 GiB 15 22 37
48 GiB 16 16
64 GiB 23 59 22 104
96 GiB 140 43 183
128 GiB 12 8 107 48 108 283
192 GiB 32 8 64 104
256 GiB 15 4 4 23
384 GiB 4 4
512 GiB 2 2
768 GiB 4 8 12
Sites total 52 39 15 72 374 70 141 43 806

PMEM size per node

PMEM size Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Nodes total
1.5 TiB 4 4
Sites total 4 4

Networking

Network interconnects

Interconnect Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Cards total
Ethernet 1 Gbps 21 95 41 43 200
Ethernet 10 Gbps 52 46 30 51 364 118 216 877
Ethernet 25 Gbps 32 264 296
InfiniBand 20 Gbps 33 33
InfiniBand 40 Gbps 43 43
InfiniBand 56 Gbps 83 83
InfiniBand 100 Gbps 24 10 34
Omni-Path 100 Gbps 40 84 124
Sites total 116 78 30 82 890 118 290 86 1690

Nodes with several Ethernet interfaces

Site Cluster Nodes 25G interfaces 10G interfaces 1G interfaces Interfaces (throughput)
Lille chetemi 15 2 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps)
Lille chiclet 8 2 eth0/enp98s0f0 (25 Gbps), eth1/enp98s0f1 (25 Gbps)
Lille chifflet 8 2 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps)
Lille chifflot 8 2 eth0/ens5f0 (25 Gbps), eth1/ens5f1 (25 Gbps)
Luxembourg petitprince 15 2 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps)
Nancy grimoire 8 4 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps), eth2/enp129s0f0 (10 Gbps), eth3/enp129s0f1 (10 Gbps)
Nancy grisou-49 1 4 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps), eth2/enp3s0f0 (10 Gbps), eth3/enp3s0f1 (10 Gbps)
Nancy grisou-[1-48] 48 4 1 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps), eth2/enp3s0f0 (10 Gbps), eth3/enp3s0f1 (10 Gbps), eth4/eno3 (1 Gbps)
Nancy grisou-[50-51] 2 4 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps), eth2/enp129s0f0 (10 Gbps), eth3/enp129s0f1 (10 Gbps)
Nancy gros 124 2 eth0/eno1 (25 Gbps), eth1/eno2 (25 Gbps)
Nantes ecotype 48 2 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps)
Rennes paranoia 8 2 1 eth0/enp3s0f0 (10 Gbps), eth1/enp3s0f1 (10 Gbps), eth2/eno1 (1 Gbps)
Rennes parasilo 28 2 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps)
Rennes paravance 72 2 eth0/eno1 (10 Gbps), eth1/eno2 (10 Gbps)

Network interface models

Type Model Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Cards total
Ethernet Broadcom NetXtreme BCM5704 Gigabit Ethernet 21 21
Ethernet Broadcom NetXtreme II BCM57800 1/10 Gigabit Ethernet 12 6 18
Ethernet Broadcom NetXtreme II BCM57810 10 Gigabit Ethernet 15 15
Ethernet Intel 82575EB Gigabit Network Connection 17 17
Ethernet Intel 82576 Gigabit Network Connection 16 43 59
Ethernet Intel 82599ES 10-Gigabit SFI/SFP+ Network Connection 23 4 99 70 108 304
Ethernet Intel Ethernet 10G 2P X520 Adapter 41 59 100
Ethernet Intel Ethernet Controller 10-Gigabit X540-AT2 2 2
Ethernet Intel Ethernet Controller X710 for 10GbE SFP+ 36 82 118
Ethernet Intel Ethernet Controller XXV710 for 25GbE SFP28 16 16 32
Ethernet Intel I350 Gigabit Network Connection 95 8 103
Ethernet Mellanox Technologies MT27710 Family [ConnectX-4 Lx] 4 124 128
Ethernet QLogic Corp. FastLinQ QL41000 Series 10/25/40/50GbE Controller 4 4
InfiniBand Mellanox Technologies MT25418 [ConnectX VPI PCIe 2.0 2.5GT/s - IB DDR / 10GigE] 33 33
InfiniBand Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] 43 43
InfiniBand Mellanox Technologies MT27500 Family [ConnectX-3] 83 83
InfiniBand Mellanox Technologies MT27700 Family [ConnectX-4] 12 12
InfiniBand Mellanox Technologies MT27800 Family [ConnectX-5] 2 2
InfiniBand Mellanox Technologies MT28908 Family [ConnectX-6] 4 4
Omni-Path Intel Omni-Path HFI Silicon 100 Series [discrete] 40 84 124
Sites total 104 39 15 78 648 70 182 86 1222

Storage

Nodes with several disks

Site Cluster Number of nodes Main disk Additional HDDs Additional SSDs
Grenoble dahu 32 SSD 240 GB 1 (4.0 TB) 1 (480 GB)
Grenoble drac 12 HDD 1.0 TB 1 (1.0 TB) 0
Grenoble troll 4 SSD 480 GB 0 1 (1.6 TB)
Grenoble yeti 4 SSD 480 GB 3 (2.0 TB*, 2.0 TB*, 2.0 TB*) 1 (1.6 TB)
Lille chetemi-10 1 HDD 600 GB 1 (300 GB) 0
Lille chetemi-[1-9,11-15] 14 HDD 300 GB 1 (300 GB) 0
Lille chiclet 8 SSD 480 GB 2 (4.0 TB*, 4.0 TB*) 0
Lille chifflet 8 SSD 400 GB 2 (4.0 TB*, 4.0 TB*) 1 (400 GB*)
Lille chifflot 8 SSD 480 GB 4 (4.0 TB*, 4.0 TB*, 4.0 TB*, 4.0 TB*) 1 (480 GB*)
Lyon gemini 2 SSD 480 GB 0 4 (1.92 TB*, 1.92 TB*, 1.92 TB*, 1.92 TB*)
Lyon hercule 4 HDD 2.0 TB 2 (2.0 TB, 2.0 TB) 0
Lyon pyxis 4 SSD 250 GB 0 1 (250 GB)
Nancy graoully 16 HDD 600 GB 1 (600 GB) 0
Nancy graphite 4 SSD 300 GB 0 1 (300 GB)
Nancy grappe 16 SSD 480 GB 1 (8.0 TB*) 0
Nancy grele 14 HDD 299 GB 1 (299 GB) 0
Nancy grimoire 8 HDD 600 GB 4 (600 GB*, 600 GB*, 600 GB*, 600 GB*) 1 (200 GB*)
Nancy grisou 51 HDD 600 GB 1 (600 GB) 0
Nancy gros 124 SSD 480 GB 0 1 (960 GB*)
Rennes paranoia 8 HDD 600 GB 4 (600 GB, 600 GB, 600 GB, 600 GB) 0
Rennes parasilo 28 HDD 600 GB 4 (600 GB*, 600 GB*, 600 GB*, 600 GB*) 1 (200 GB*)
Rennes paravance 72 HDD 600 GB 1 (600 GB) 0

*: disk is reservable

Accelerators (GPU, Xeon Phi)

Accelerator families

Accelerator family Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Accelerators total
Intel Xeon Phi 4 4
Nvidia 48 32 20 124 224
Sites total 48 32 20 128 228

Accelerator models

Accelerator model Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Accelerators total
Intel Xeon Phi 7120P 4 4
Nvidia GTX 1080 Ti 16 28 44
Nvidia GTX 980 10 10
Nvidia RTX 2080 Ti 52 52
Nvidia Tesla K40M 12 12
Nvidia Tesla M2075 4 4
Nvidia Tesla P100 48 12 60
Nvidia Tesla T4 20 20
Nvidia Tesla V100 4 16 20
Nvidia Titan Black 2 2
Sites total 48 32 20 128 228

Accelerator cores

Accelerator model Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Cores total
Intel Xeon Phi 7120P 244 244
Nvidia GTX 1080 Ti 57344 100352 157696
Nvidia GTX 980 20480 20480
Nvidia RTX 2080 Ti 226304 226304
Nvidia Tesla K40M 34560 34560
Nvidia Tesla M2075 1792 1792
Nvidia Tesla P100 172032 43008 215040
Nvidia Tesla T4 51200 51200
Nvidia Tesla V100 20480 81920 102400
Nvidia Titan Black 5760 5760
Sites total 172032 120832 83712 438900 815476

Nodes models

Nodes model Grenoble Lille Luxembourg Lyon Nancy Nantes Rennes Sophia Nodes total
Dell PowerEdge C6100 43 43
Dell PowerEdge C6220 4 47 22 73
Dell PowerEdge C6220 II 8 8
Dell PowerEdge C6420 32 64 96
Dell PowerEdge M620 15 15
Dell PowerEdge R430 23 23
Dell PowerEdge R630 15 75 48 100 238
Dell PowerEdge R640 4 140 144
Dell PowerEdge R720 18 10 28
Dell PowerEdge R730 8 20 28
Dell PowerEdge R740 8 8
Dell PowerEdge R7425 8 5 13
Dell PowerEdge R940 4 4
Dell PowerEdge T640 13 13
HP ProLiant DL165 G7 16 16
IBM PowerNV S822LC (8335-GTB) 12 12
Nvidia DGX-1 2 2
R181-T92-00 4 4
SUN FIRE X2270 17 17
Sun Fire V20z 21 21
Sites total 52 39 15 72 374 70 141 43 806
Last generated from the Grid'5000 Reference API on 2020-12-17 (commit 7a21cf4c8)
Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Hardware&oldid=89832"