From Grid5000
Revision as of 15:05, 2 September 2022 by Pneyron (talk | contribs) (Using the FPGA)

Jump to: navigation, search

As of August 2022, Grid'5000 features 2 nodes, each equipped with one AMD/Xilinx FPGA.

AMD Xilinx Alveo U200

Hardware description

The Grenoble site of Grid'5000 hosts 2 servers (Servan cluster) equipped with an AMD/Xilinx FPGA. FPGA are AMD/Xilinx Alveo U200, which are PCIe Accelerator card.

Detailed specifications are provided here (cards with passive thermal cooling).

Technically, the installation of those FPGAs in the Servan nodes has the following characteristics:

  • JTAG
    • JTAG programming is provided on Xilinx Alveo U200 via a USB port on the card. In the Grid'5000 installation, it is cabled back to the hosting machine itself. Thus, programming of the FPGA of e.g. servan-1 can be done (e.g. with Vivado) from e.g. servan-1 itself.
  • Network
    • Both Ethernet ports of the FPGA are cabled to the site network along with all servers of the site (including the servan server NICs).
      Grenoble site network
    • Ethernet ports are not shown as NICs in the operating system of the hosting machine (unless FPGA is programmed as to do so).
    • Ethernet ports are cabled to 100Gbps ports on Grenoble site router/switch. Switch ports are configured with Auto-Negotiation disabled and Speed forced to 100Gbps (not working otherwise, as far as we tested).
    • Kavlan is supported on the FPGA Ethernet ports just like any NIC of a server of the site (including the servan servers NICs). FPGA ports are named servan-1-fpga0, servan-1-fpga1, servan-2-fpga0, servan-2-fpga1 in kavlan. IP addresses are provided via DHCP to the FPGA ports in kavlan where the DHCP service is available.
    • Note: using the 100Gbps capability of the FPGA ports requires acquiring a free-of-charge Xilinx licence.
  • Wattmeter
    • Each servan node energy consumption is measured by a wattmeter. Measures are available in Kwollect. (Work-in-progress)
    • Energy consumption can also be retrieved using Xilinx tools (e.g. xbutil, xbtop) from the host operating system when the FPGA is running XRT (see below).

Using the FPGA

The servan nodes just like all Grid'5000 nodes are running Debian by default. No support is provided in that default operating system environment.

AMD/Xilinx supports a limited list of OS to operate the FPGA, see here. A kadeploy Ubuntu 20.04 image is provided by Pierre Neyron, which includes the Xilinx tools. Anyone may deploy it using:

kadeploy -u pneyron -e ubuntu2004-fpga

That ubuntu2004-fpga system environment is built using a kameleon recipe available in https://gitlab.inria.fr/neyron/ubuntu2004-fpga.

Vitis and Vivado are installed on a NFS shared storage (/tools/ in the deployed system), which access can be requested to Pierre Neyron.