Performance testing of Distributed Filesystems

From Grid5000
Jump to: navigation, search



Tsung is a multi-protocol distributed load testing tool. It includes a plugin for file system testing. In this page, we show how to use this plugin to do some NFS performance testing. This can be used to test other distributed file systems that NFS, since we use the reguler read/write operation on the clients instead of raw protocol.


The idea here is to simulate the workload of many jobs doing concurrently reading and writing files. For example, this can emulate image processing jobs: each job read file, then do some processing, and write files (each file use several MB of disk space). In this example, the files are first written and the read from another 'client' node (we want to avoid caching effects on the client side)

<?xml version="1.0"?>
<!DOCTYPE tsung SYSTEM "/home/nniclaus/share/tsung/tsung-1.0.dtd">
<tsung loglevel="notice" dumptraffic="false">

<!-- Get clients from oar batch manager. Ideally, we need 256 cores to do this test. -->
    <client type="batch" batch="oar" maxusers="8000"/>

<!-- server is not used by fs plugin, so this is meaningless -->
    <server host="" port="5432" type="erlang"/>

   <monitor batch="true" host="torque" type="erlang"/>
   <monitor host="nef-server" type="munin"/>

<!-- 4 phases, from 32 to 256 simultaneous users -->
 <arrivalphase phase="1" duration="10" unit="minute">
    <users maxnumber="32" arrivalrate="10" unit="second"/>
  <arrivalphase phase="2" duration="20" unit="minute">
    <users maxnumber="64" arrivalrate="10" unit="second"/>
  <arrivalphase phase="3" duration="30" unit="minute">
    <users maxnumber="128" arrivalrate="10" unit="second"/>
  <arrivalphase phase="4" duration="45" unit="minute">
    <users maxnumber="256" arrivalrate="10" unit="second"/>

  <!-- Use the same seed for each test -->
  <option name="seed" value="43"/>

  <session probability="100" name="fs-example" type="ts_fs">
    <!-- First create a directory for the current user -->
    <transaction name="mkdir">
    <request subst="true">
      <fs cmd="make_dir" path="/mnt/bench/%%_tsung_userid%%"/>

    <!-- Then write 100 files (6.5MB per file) sequentially-->
    <for from="1" to="100" var="file" incr="1">
    <transaction name="write_file">
      <request subst="true">
        <fs cmd="open" mode="write" path="/mnt/bench/%%_tsung_userid%%/tsung-write.bin.%%_file%%"/>
      <for from="0" to="100" var="chunk" incr="1">
        <request><fs cmd="write_chunk" size="65536"/></request>
      <request><fs cmd="close"/></request>
    <thinktime random='false' value="1"/>

    <!-- Get an id of a users from another node to avoid cache effects on the
         client side. Here we assume that each client machine has 8 cores -->

     <setdynvars sourcetype="eval" code="fun({Pid,DynVars})->
                                    {ok,Val} = ts_dynvars:lookup(tsung_userid,DynVars),
                                    {Id,Mod} = case list_to_integer(Val) of
                                       I when I > 128 -> {I,256};
                                       I when I > 64  -> {I,128};
                                       I when I > 32  -> {I,64};
                                       I              -> {I,32}
                                    case Id < 9 of
                                     true -> (abs(Id+8) rem Mod) +1;
                                     _ ->
                                             (abs(Id-9) rem Mod) +1
        <var name="previd"/>

    <!-- Wait 30 sec  -->
    <thinktime min="30" max="31"/>

    <!-- Read 100 files from another tsung user, using the computed id -->
      <for from="1" to="100" var="file" incr="1">
        <transaction name="read_file">
          <request subst="true">
            <fs cmd="open" mode="read" path="/mnt/bench/%%_previd%%/tsung-write.bin.%%_file%%"/>
       <for from="1" to="100" var="chunk" incr="1">
         <request><fs cmd="read_chunk" size="65536"/></request>
       <request><fs cmd="close"/></request>
        <thinktime random='false' value="1"/>


How to run a test

You need to install erlang and tsung 1.4.0. You can install it in your home directory or use kadeploy and apt (tsung won't work on lenny because of the old version of erlang; a backport is needed on lenny)

Sources and binaries are available here:

Once you have defined a configuration file, you can run the test with:

tsung -f ~/.tsung/nfs.xml  start

cd to the test directory (~/.tsung/YYYYMMDD-HHMM) and then generate the graphical output (periodically during the test, or at the end) with ~/lib/tsung/bin/

In the example, the filesystem must be mounted in /mnt/bench/ on the clients, and tsung gets it's node list from OAR. If you need to run tsung on a deploy environement without OAR, use a machine file and :

 OAR_NODEFILE=machinefile tsung -f ~/.tsung/nfs.xml  start


See an examples here: test NFS4-EXT4

Personal tools

Public Portal
Users Portal
Admin portal
Wiki special pages