Grid'5000 user report for Olivier Gluck

Jump to: navigation, search

User information

Olivier Gluck (users, user, account-manager, lyon, ml-users user)
More user information in the user management interface.

Experiments

  • MPI'5000 (Middleware) [in progress]
    Description:

    MPI5000 is a new transparent layer placed between MPI and TCP allowing application composed of several tasks to be correctly distributed on available node regarding the grid topology and the application scheme. Thus, our layer needs two data files: a file describing the grid topology including available nodes, both latency and bandwidth between the nodes and between sites; another file describing the application communication patterns with the size and the amount of messages sent between MPI processes. Using these two informations, our layer should realise an efficient placement of tasks on grid nodes.

    Our layer also propose to transparently slipt TCP connections between MPI processes in order to take into account the grid topology. This new architecture is based on a system of relays placed at the LAN/WAN interface. We replace each end-to-end TCP connection by three connections (two on the LAN between a node and a relay, one on the WAN between two relays). Thus, we expect a faster lost recovery on LAN as well as a reduction of memory used because for local TCP buffers (they depend on RTT latency of a connection). On the relay, we planned to use different TCP implementations or different protocols for local and distant communications. The relays could also implement a different scheduling strategy of the messages in function of the data size, for example we could give priority to small message (usually control messages). Finally, as MPI applications are mostly using small messages, they are more penalised if the network is congestionned by large flows. We planned to reserve bandwidth in order to optimise MPI communications on the long distance shared link. The implementation of our proposition is based both on a library between MPI and system calls and relays daemon. Thus, the architecture is independant of MPI implementations.


    Results: For the moment, relays and library are in a test phase. We are now testing our architecture in Grid'5000. Finally, we will implement the optimisations proposed previously.
  • Optimization of Long-distance communications for MPICH-Madeleine (Networking) [achieved]
    Description: To be executed in a grid, applications needs a support like MPI. But MPI was created for clusters. In grids, there is at least two more constraints to manage: heterogeneity and long distance management.

    MPICH-Madeleine manage heterogeneity properly but no experiments was done with long-distance.
    I use Grid'5000 to see how MPICH-Madeleine behave in the grid. I compare local perfomances and long distant ones in order to adapt this implementation for the grid. Principally, I try to optimize long-distance communications.
    Results: Optimizations done lead to obtain a bandwith of 600 Mbps instead of 95 Mbps for sending MPI messages over a Wide Area Network.

Publications

Collaborations

    Success stories and benefits from Grid'5000

    • Overall benefits
    • GRID'5000 is composed of several and heterogeneous high speed interconnects. Furthermore, it supplies Gigabit or 10-Gigabit dedicated links between its sites which makes us hope to obtain good performance in the WAN interconnect. It is a research platform which allows us to reserve nodes and links and to deploy our own system image on it.

    last update: 2008-11-16 14:59:00

    Personal tools
    Namespaces

    Variants
    Views
    Actions
    Public Portal
    Users Portal
    Admin portal
    Wiki special pages
    Toolbox