kea/tests/tools/dhcp-ubench/dhcp-perf-guide.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
<!ENTITY mdash  "&#x2014;" >
<!ENTITY % version SYSTEM "version.ent">
%version;
]>

<!--
 - Copyright (C) 2012  Internet Systems Consortium, Inc. ("ISC")
 -
 - Permission to use, copy, modify, and/or distribute this software for any
 - purpose with or without fee is hereby granted, provided that the above
 - copyright notice and this permission notice appear in all copies.
 -
 - THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
 - REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
 - AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
 - INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
 - LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
 - OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 - PERFORMANCE OF THIS SOFTWARE.
-->

<?xml-stylesheet href="bind10-guide.css" type="text/css"?>

<book>
  <bookinfo>
    <title>DHCP Performance Guide</title>
    <!-- <subtitle>Various aspects of DHCP Performance in BIND 10</subtitle> -->

    <copyright>
      <year>2012</year>
      <holder>Internet Systems Consortium, Inc. ("ISC")</holder>
    </copyright>
    <author>
      <firstname>Tomasz</firstname>
      <surname>Mrugalski</surname>
    </author>
    <author>
      <firstname>Marcin</firstname>
      <surname>Siodelski</surname>
    </author>
    <abstract>
      <para>BIND 10 is a framework that features Domain Name System
      (DNS) and Dynamic Host Configuration Protocol (DHCP)
      software with development managed by Internet Systems Consortium (ISC).
      This document describes various aspects of DHCP performance,
      measurements and tuning. It covers BIND 10 DHCP (codename Kea),
      existing ISC DHCP4 software, perfdhcp (a DHCP performance
      measurement tool) and other related topics.</para>
    </abstract>

    <releaseinfo>This is a companion document for BIND 10 version
    &__VERSION__;.</releaseinfo>

  </bookinfo>

  <preface>
    <title>Preface</title>

    <section id="acknowledgements">
      <title>Acknowledgements</title>

      <para>ISC would like to acknowledge generous support for
      BIND 10 development of DHCPv4 and DHCPv6 components provided
      by <ulink url="http://www.comcast.com/">Comcast</ulink>.</para>

    </section>

  </preface>

  <chapter id="intro">
    <title>Introduction</title>
    <para>
      This document is in the early stages of development. It is
      expected to grow significantly in the near future. It will
      cover topics like database backend perfomance measurements,
      tools, and the pros an cons of various optimization techniques.
    </para>

  </chapter>

  <chapter id="dhcp4">
    <title>ISC DHCP 4.x</title>
    <para>
      TODO: Write something about ISC DHCP4 here.
    </para>
  </chapter>

  <chapter id="kea">
    <title>Kea</title>
    <para>

    </para>

    <section>
      <title>Backend performance evaluation</title>
      <para>
        Kea will support several different database backends, using
        both popular databases (like MySQL or SQLite) and
        custom-developed solutions (such as an in-memory database).
        To aid in the choice of backend, the BIND 10
        source code features a set of performance microbenchmarks.
         Written in C/C++, these are small tools that simulate expected
        DHCP server behaviour and evaluate the performance of
        considered databases. As implemented, the benchmarks are not really
        simulating DHCP operation, but rather use set of primitives
        that can be used by a real server.  For this reason, they are called
        micro-benchmarks.
      </para>

      <para>Although there are many operations and data types that
      server could store in a database, the most frequently used data
      type is lease information. Although the information held for IPv4
      and IPv6 leases differs slightly, it is expected that the performance
      differences will be minimal between IPv4 and IPv6 lease operations.
      Therefore each test uses the lease4 table (in which IPv4 leases are stored)
      for performance measurements.
      </para>

      <para>All benchmarks are implemented as single threaded applications
      that take advantage of a single database connection.</para>

      <para>
        Those benchmarks are stored in tests/tools/dhcp-ubench directory of the
        BIND 10 source tree. This directory contains simplified prototypes for
        the various database back-ends that are planned or considered as a
        possibly for BIND10 DHCP.  These benchmarks are expected to evolve into
        useful tools that will allow users to measure performance in their
        specific environment.
      </para>

    <para>
      Currently the following benchmarks are implemented:
      <itemizedlist>
        <listitem><para>In memory + flat file</para></listitem>
        <listitem><para>SQLite</para></listitem>
        <listitem><para>MySQL</para></listitem>
      </itemizedlist>
    </para>

    <para>
      As the benchmarks require additional (sometimes heavy) dependencies, they
      are not built by default. Actually, their build system is completely
      separate from that of the rest of BIND 10.  It will be eventually merged
      with the main BIND 10 build system.
    </para>

    <para>
      All benchmarks will follow the same pattern:
      <orderedlist>
        <listitem><para>Prepare operation (connect to a database, create a file etc.)</para></listitem>
        <listitem><para>Measure timestamp 0</para></listitem>
        <listitem><para>Commit new lease4 record (repeated N times)</para></listitem>
        <listitem><para>Measure timestamp 1</para></listitem>
        <listitem><para>Search for random lease4 record (repeated N times)</para></listitem>
        <listitem><para>Measure timestamp 2</para></listitem>
        <listitem><para>Update existing lease4 record (repeated N times)</para></listitem>
        <listitem><para>Measure timestamp 3</para></listitem>
        <listitem><para>Delete existing lease4 record (repeated N times)</para></listitem>
        <listitem><para>Measure timestamp 4</para></listitem>
        <listitem><para>Print out statistics, based on N and measured timestamps.</para></listitem>
      </orderedlist>

      Although this approach does not attempt to simulate actual DHCP server
      operation that has mix of all steps, it answers the
      questions about basic database strengths and weak points. In particular
      it can show what is the impact of specific database optimizations, such as
      changing engine, optimizing for writes/reads etc.
    </para>

    <para>
      The framework attempts to do the same amount of work for every
      backend thus allowing fair comparison between them.
    </para>
    </section>

    <section id="mysql-backend">
      <title>MySQL backend</title>
      <para>The MySQL backend requires the MySQL client development libraries. It uses
      the mysql_config tool (similar to pkg-config) to discover required
      compilation and linking options. To install required packages on Ubuntu,
      use the following command:

      <screen>$ <userinput>sudo apt-get install mysql-client mysql-server libmysqlclient-dev</userinput></screen>

      Make sure that MySQL server is running. Make sure that you have your setup
      configured so there is a user that is able to modify used database.</para>

      <para>Before running tests, you need to initialize your database. You can
      use mysql.schema script for that purpose.</para>

      <para><emphasis>WARNING: It will drop existing
      Kea database. Do not run this on your production server. </emphasis></para>

      <para>Assuming your
      MySQL user is "kea", you can initialize your test database by:

      <screen>$ <userinput>mysql -u kea -p &lt; mysql.schema</userinput></screen>
      </para>

      <para>After the database is initialized, you are ready to run the test:
      <screen>$ <userinput>./mysql_ubench</userinput></screen>

      or

      <screen>$ <userinput>./mysql_ubench &gt; results-mysql.txt</userinput></screen>

      Redirecting output to a file is important, because for each operation
      there is a single character printed to show progress. If you have a slow
      terminal, this may considerably affect test performance. On the other hand,
      printing something after each operation is required as poor database settings
      may slow down operations to around 20 per second. (The observant user is expected
      to note that the initial dots are printed too slowly and abort the test.)</para>

      <para>Currently all default parameters are hardcoded. Default values can be
      overridden using command line switches. Although all benchmarks take
      the same list of parameters, some of them are specific to a given backend.
      To get a list of supported parameters, run the benchmark with the "-h" option:

      <screen>$ <userinput>./mysql_ubench -h</userinput></screen>
      </para>

      <para>Synchronous operation requires database backend to
      physically store changes to disk before proceeding. This
      property ensures that no data is lost in case of the server
      failure.  Unfortunately, it slows operation
      considerably. Asynchronous mode allows database to write data at
      a later time (usually controlled by the database engine on OS
      disk buffering mechanism).</para>

      <section>
        <title>MySQL tweaks</title>

        <para>To modify the default mysql_ubench parameters, command line
        switches can be used. The currently supported switches are
        (default values specified in brackets):
        <orderedlist>
          <listitem><para>-f name - name of the database ("kea")</para></listitem>
          <listitem><para>-m hostname - name of the database host ("localhost")</para></listitem>
          <listitem><para>-u user - MySQL username  ("root")</para></listitem>
          <listitem><para>-p password - MySQL password ("secret")</para></listitem>
          <listitem><para>-n num - number of iterations (100)</para></listitem>
          <listitem><para>-s yes|no - should the operations be performed in a synchronous (yes)
          or asynchronous (no) manner (yes)</para></listitem>
          <listitem><para>-v yes|no - verbose mode. Should the test print out progress? (yes)</para></listitem>
          <listitem><para>-c yes|no - precompiled statements. Should the SQL statements be precompiled? (yes)</para></listitem>
        </orderedlist>
        </para>


        <para>One parameter that has huge impact on performance is the choice of backend engine.
        You can get a list of engines of your MySQL implementation by using

        <screen>&gt; <userinput>show engines;</userinput></screen>

        in your mysql client. Two notable engines are MyISAM and InnoDB. mysql_ubench uses
        use MyISAM for synchronous mode and InnoDB for asynchronous. Please use
        '-s yes|no' to choose whether you want synchronous or asynchronous operations.</para>

        <para>Another parameter that affects performance are precompiled statements.
        In a basic approach, the actual SQL query is passed as a text string that is
        then parsed by the database engine. Alternative is a so called precompiled
        statement. In this approach the SQL query is compiled an specific values are being
        bound to it. In the next iteration the query remains the same, only bound values
        are changing (e.g. searching for a different address). Usage of basic or precompiled
        statements is controlled with '-c no|yes'.</para>
    </section>
    </section>


    <section id="sqlite-ubench">
      <title>SQLite-ubench</title>
      <para>The SQLite backend requires both the sqlite3 development and run-time packages. Their
      names may vary from system to system, but on Ubuntu 12.04 they are called
      sqlite3 libsqlite3-dev. To install them, use the following command:

      <screen>&gt; <userinput>sudo apt-get install sqlite3 libsqlite3-dev</userinput></screen>

      Before running the test the database has to be created. Use the following command for that:
      <screen>&gt; <userinput>cat sqlite.schema | sqlite3 sqlite.db</userinput></screen>

      A new database called sqlite.db will be created. That is the default name used
      by sqlite_ubench test. If you prefer other name, make sure you update
      sqlite_ubench.cc accordingly.</para>

      <para>Once the database is created, you can run tests:
      <screen>&gt; <userinput>./sqlite_ubench</userinput></screen>
      or
      <screen>&gt; <userinput>./sqlite_ubench > results-sqlite.txt</userinput></screen>
      </para>

      <section id="sqlite-tweaks">
        <title>SQLite tweaks</title>
        <para>To modify default sqlite_ubench parameters, command line
        switches can be used. The currently supported switches are
        (default values specified in brackets):
        <orderedlist>
          <listitem><para>-f filename - name of the database file ("sqlite.db")</para></listitem>
          <listitem><para>-n num - number of iterations (100)</para></listitem>
          <listitem><para>-s yes|no - should the operations be performed in a synchronous (yes)
          or asynchronous (no) manner (yes)</para></listitem>
          <listitem><para>-v yes|no - verbose mode. Should the test print out progress? (yes)</para></listitem>
          <listitem><para>-c yes|no - precompiled statements. Should the SQL statements be precompiled? (yes)</para></listitem>
        </orderedlist>
        </para>

        <para>SQLite can run in asynchronous or synchronous mode. This
        mode can be controlled by using "synchronous" parameter. It is set
        using the SQLite command:</para>

        <para><command>PRAGMA synchronous = ON|OFF</command></para>

        <para>Another tweakable feature is journal mode. It can be
        turned to several modes of operation. Its value can be
        modified in SQLite_uBenchmark::connect().  See
        http://www.sqlite.org/pragma.html#pragma_journal_mode for
        a detailed explanation.</para>

        <para>sqlite_bench supports precompiled statements. Please use
        '-c no|yes' to define which should be used: basic SQL query (no) or
        precompiled statement (yes).</para>
      </section>
    </section>

    <section id="memfile-ubench">
      <title>memfile-ubench</title> <para>The memfile backend is a
      custom backend that somewhat mimics operation of ISC DHCP4. It
      implements in-memory storage using standard C++ and boost
      mechanisms (std::map and boost::shared_ptr&lt;&gt;). All
      database changes are also written to a lease file, which is
      strictly write-only. This approach takes advantage of the fact
      that file append operation is faster than modifications introduced
      in the middle of the file (as it often requires moving all data
      after modified point, effectively requiring writing large parts of
      the whole file, not just changed fragment).</para>

      <para>There are no preparatory steps required for memfile benchmark.
      The only requirement is the ability to create and write specified lease
      file (dhcpd.leases in the current directory). The tests can be run
      as follows:
      <screen>&gt; <userinput>./memfile_ubench</userinput></screen>
      or
      <screen>&gt; <userinput>./memfile_ubench > results-memfile.txt</userinput></screen>
      </para>

      <section id="memfile-tweaks">
        <title>memfile tweaks</title>
        <para>To modify default memfile_ubench parameters, command line
        switches can be used. Currently supported switches are
        (default values specified in brackets):
        <orderedlist>
          <listitem><para>-f filename - name of the database file ("dhcpd.leases")</para></listitem>
          <listitem><para>-n num - number of iterations (100)</para></listitem>
          <listitem><para>-s yes|no - should the operations be performend in a synchronous (yes)
          or asynchronous (no) manner (yes)</para></listitem>
          <listitem><para>-v yes|no - verbose mode. Should the test print out progress? (yes)</para></listitem>
        </orderedlist>
        </para>

        <para>memfile can run in asynchronous or synchronous mode. This
        mode can be controlled by using sync parameter. It uses
        fflush() and fsync() in synchronous mode to make sure that
        data is not buffered and physically stored on disk.</para>
      </section>
    </section>

    <section>
      <title>Basic performance measurements</title>
      <para>This section contains sample results for backend performance measurements,
      taken using microbenchmarks. Tests were conducted on reasonably powerful machine:
      <screen>
CPU: Quad-core Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz (8 logical cores)
HDD: 1,5TB Seagate Barracuda ST31500341AS 7200rpm, ext4 partition
OS: Ubuntu 12.04, running kernel 3.2.0-26-generic SMP x86_64
compiler: g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
MySQL version: 5.5.24
SQLite version: 3.7.9sourceid version is 2011-11-01 00:52:41 c7c6050ef060877ebe77b41d959e9df13f8c9b5e</screen>
      </para>

      <para>The benchmarks were run without using precompiled statements.
      The code was compiled with the -O0 flag (no code optimizations).
      Each run was executed once.</para>

      <para>Two series of measures were made, synchronous and
      asynchronous. As those modes offer radically different
      performances, synchronous mode was conducted for one
      thousand repetitions and asynchronous mode was conducted for
      one hundred thousand repetitions.</para>

      <!-- raw results sync -->
      <table><title>Synchronous results (basic)</title>
      <tgroup cols='6' align='center' colsep='1' rowsep='1'>
        <colspec colname='Backend'/>
        <colspec colname='Num' />
        <colspec colname='Create'/>
        <colspec colname='Search'/>
        <colspec colname='Update'/>
        <colspec colname='Delete'/>
        <colspec colname='Average'/>
        <thead>
          <row>
            <entry>Backend</entry>
            <entry>Operations</entry>
            <entry>Create [s]</entry>
            <entry>Search [s]</entry>
            <entry>Update [s]</entry>
            <entry>Delete [s]</entry>
            <entry>Average [s]</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>MySQL</entry>
            <entry>1,000</entry>
            <entry>31.604</entry>
            <entry> 0.117</entry>
            <entry>27.964</entry>
            <entry>27.695</entry>
            <entry>21.845</entry>
          </row>

          <row>
            <entry>SQLite</entry>
            <entry>1,000</entry>
            <entry>61.421</entry>
            <entry> 0.033</entry>
            <entry>59.477</entry>
            <entry>56.034</entry>
            <entry>44.241</entry>
          </row>

          <row>
            <entry>memfile</entry>
            <entry>1,000</entry>
            <entry>38.224</entry>
            <entry> 0.001</entry>
            <entry>38.041</entry>
            <entry>38.017</entry>
            <entry>28.571</entry>
          </row>

        </tbody>
      </tgroup>
      </table>

      <para>The following parameters were measured for asynchronous mode.
      MySQL and SQLite were run with one hundred thousand repetitions.</para>

      <!-- raw results async -->
      <table><title>Asynchronous results (basic)</title>
      <tgroup cols='6' align='center' colsep='1' rowsep='1'>
        <colspec colname='Backend'/>
        <colspec colname='Num' />
        <colspec colname='Create'/>
        <colspec colname='Search'/>
        <colspec colname='Update'/>
        <colspec colname='Delete'/>
        <colspec colname='Average'/>
        <thead>
          <row>
            <entry>Backend</entry>
            <entry>Operations</entry>
            <entry>Create [s]</entry>
            <entry>Search [s]</entry>
            <entry>Update [s]</entry>
            <entry>Delete [s]</entry>
            <entry>Average [s]</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>MySQL</entry>
            <entry>100,000</entry>
            <entry>10.585</entry>
            <entry>10.386</entry>
            <entry>10.062</entry>
            <entry> 8.890</entry>
            <entry> 9.981</entry>
          </row>

          <row>
            <entry>SQLite</entry>
            <entry>100,000</entry>
            <entry> 3.710</entry>
            <entry> 3.159</entry>
            <entry> 2.865</entry>
            <entry> 2.439</entry>
            <entry> 3.044</entry>
          </row>

          <row>
            <entry>memfile</entry>
            <entry>100,000</entry>
            <entry> 1.300</entry>
            <entry> 0.039</entry>
            <entry> 1.307</entry>
            <entry> 1.278</entry>
            <entry> 0.981</entry>
          </row>

        </tbody>
      </tgroup>
      </table>

      <para>The presented performance results can be converted into operations per second metrics.
      It should be noted that due to large differences between various operations (sometimes
      over three orders of magnitude), it is difficult to create a simple, readable chart with
      that data.</para>

      <table id="tbl-basic-perf-results"><title>Estimated basic performance</title>
      <tgroup cols='6' align='center' colsep='1' rowsep='1'>
        <colspec colname='Backend'/>
        <colspec colname='Create'/>
        <colspec colname='Search'/>
        <colspec colname='Update'/>
        <colspec colname='Delete'/>
        <colspec colname='Average'/>
        <thead>
          <row>
            <entry>Backend</entry>
            <entry>Create [oper/s]</entry>
            <entry>Search [oper/s]</entry>
            <entry>Update [oper/s]</entry>
            <entry>Delete [oper/s]</entry>
            <entry>Average [oper/s]</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>MySQL (async)</entry>
            <entry>9447.47</entry>
            <entry>9627.97</entry>
            <entry>9938.00</entry>
            <entry>11248.34</entry>
            <entry>10065.45</entry>
          </row>

          <row>
            <entry>SQLite (async)</entry>
            <entry>26951.59</entry>
            <entry>31654.29</entry>
            <entry>34899.70</entry>
            <entry>40993.59</entry>
            <entry>33624.79</entry>
          </row>

          <row>
            <entry>memfile (async)</entry>
            <entry>76944.27</entry>
            <entry>2542588.35</entry>
            <entry>76504.54</entry>
            <entry>78269.25</entry>
            <entry>693576.60</entry>
          </row>


          <row>
            <entry>MySQL (sync)</entry>
            <entry>31.64</entry>
            <entry>8575.45</entry>
            <entry>35.76</entry>
            <entry>36.11</entry>
            <entry>2169.74</entry>
          </row>

          <row>
            <entry>SQLite (sync)</entry>
            <entry>16.28</entry>
            <entry>20045.37</entry>
            <entry>16.81</entry>
            <entry>17.85</entry>
            <entry>7524.08</entry>
          </row>

          <row>
            <entry>memfile (sync)</entry>
            <entry>26.16</entry>
            <entry>1223990.21</entry>
            <entry>26.29</entry>
            <entry>26.30</entry>
            <entry>306017.24</entry>
          </row>

        </tbody>
      </tgroup>
      </table>

      <!-- that is obsolete approach that is going to be removed in docbook 5.0.
      Its only advantage is that it actually works with docbook2pdf -->
      <!--
      <figure>
        <title>Graphical representation of the basic performance
        results presented in table <xref linkend="tbl-basic-perf-results" />.</title>
        <graphic scale="100" fileref="performance-results-graph1.png" />
      </figure>-->

      <!-- this works great for HTML export, but is silently ignored by docbook2pdf
      and docbook2ps tools. -->
      <mediaobject>
        <imageobject>
          <imagedata fileref="performance-results-graph1.png" format="PNG" />
        </imageobject>
        <caption>
          <para>Graphical representation of the basic performance results
          presented in table <xref linkend="tbl-basic-perf-results" />.</para>
        </caption>
      </mediaobject>

    </section>

    <section>
      <title>Optimized performance measurements</title>
      <para>This section contains sample results for backend performance measurements,
      taken using microbenchmarks. Tests were conducted on reasonably powerful machine:
      <screen>
CPU: Quad-core Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz (8 logical cores)
HDD: 1,5TB Seagate Barracuda ST31500341AS 7200rpm, ext4 partition
OS: Ubuntu 12.04, running kernel 3.2.0-26-generic SMP x86_64
compiler: g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
MySQL version: 5.5.24
SQLite version: 3.7.9sourceid version is 2011-11-01 00:52:41 c7c6050ef060877ebe77b41d959e9df13f8c9b5e</screen>
      </para>

      <para>The benchmarks were run with precompiled statements enabled.
      The code was compiled with the -Ofast flag (optimize compilation for speed).
      Each run was repeated three times and measured values were averaged.</para>

      <para>Again the benchmarks were run in two series, synchronous and
      asynchronous. As those modes offer radically different
      performances, synchronous mode was conducted for one
      thousand repetitions and asynchronous mode was conducted for
      one hundred thousand repetitions.</para>

      <!-- raw results sync -->
      <table><title>Synchronous results (optimized)</title>
      <tgroup cols='6' align='center' colsep='1' rowsep='1'>
        <colspec colname='Backend'/>
        <colspec colname='Num' />
        <colspec colname='Create'/>
        <colspec colname='Search'/>
        <colspec colname='Update'/>
        <colspec colname='Delete'/>
        <colspec colname='Average'/>
        <thead>
          <row>
            <entry>Backend</entry>
            <entry>Operations</entry>
            <entry>Create [s]</entry>
            <entry>Search [s]</entry>
            <entry>Update [s]</entry>
            <entry>Delete [s]</entry>
            <entry>Average [s]</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>MySQL</entry>
            <entry>1,000</entry>
            <entry>27.887</entry>
            <entry> 0.106</entry>
            <entry>28.223</entry>
            <entry>27.696</entry>
            <entry>20.978</entry>
          </row>

          <row>
            <entry>SQLite</entry>
            <entry>1,000</entry>
            <entry>61.299</entry>
            <entry> 0.015</entry>
            <entry>59.648</entry>
            <entry>61.098</entry>
            <entry>45.626</entry>
          </row>

          <row>
            <entry>memfile</entry>
            <entry>1,000</entry>
            <entry>39.564</entry>
            <entry> 0.001</entry>
            <entry>39.543</entry>
            <entry>39.326</entry>
            <entry>29.608</entry>
          </row>

        </tbody>
      </tgroup>
      </table>

      <para>The following parameters were measured for asynchronous mode.
      MySQL and SQLite were run with one hundred thousand repetitions.</para>

      <!-- raw results async -->
      <table><title>Asynchronous results (optimized)</title>
      <tgroup cols='6' align='center' colsep='1' rowsep='1'>
        <colspec colname='Backend'/>
        <colspec colname='Num' />
        <colspec colname='Create'/>
        <colspec colname='Search'/>
        <colspec colname='Update'/>
        <colspec colname='Delete'/>
        <colspec colname='Average'/>
        <thead>
          <row>
            <entry>Backend</entry>
            <entry>Operations</entry>
            <entry>Create [s]</entry>
            <entry>Search [s]</entry>
            <entry>Update [s]</entry>
            <entry>Delete [s]</entry>
            <entry>Average [s]</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>MySQL</entry>
            <entry>100,000</entry>
            <entry>8.507</entry>
            <entry>9.698</entry>
            <entry>7.785</entry>
            <entry>8.326</entry>
            <entry>8.579</entry>
          </row>

          <row>
            <entry>SQLite</entry>
            <entry>100,000</entry>
            <entry> 1.562</entry>
            <entry> 0.949</entry>
            <entry> 1.513</entry>
            <entry> 1.502</entry>
            <entry> 1.382</entry>
          </row>

          <row>
            <entry>memfile</entry>
            <entry>100,000</entry>
            <entry>1.302</entry>
            <entry>0.038</entry>
            <entry>1.306</entry>
            <entry>1.263</entry>
            <entry>0.977</entry>
          </row>

        </tbody>
      </tgroup>
      </table>

      <para>The presented performance results can be converted into operations per second metrics.
      It should be noted that due to large differences between various operations (sometime
      over three orders of magnitude), it is difficult to create a simple, readable chart with
      the data.</para>

      <table id="tbl-optim-perf-results"><title>Estimated optimized performance</title>
      <tgroup cols='6' align='center' colsep='1' rowsep='1'>
        <colspec colname='Backend'/>
        <colspec colname='Create'/>
        <colspec colname='Search'/>
        <colspec colname='Update'/>
        <colspec colname='Delete'/>
        <colspec colname='Average'/>
        <thead>
          <row>
            <entry>Backend</entry>
            <entry>Create [oper/s]</entry>
            <entry>Search [oper/s]</entry>
            <entry>Update [oper/s]</entry>
            <entry>Delete [oper/s]</entry>
            <entry>Average [oper/s]</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>MySQL (async)</entry>
            <entry>11754.84</entry>
            <entry>10311.34</entry>
            <entry>12845.35</entry>
            <entry>12010.24</entry>
            <entry>11730.44</entry>
          </row>

          <row>
            <entry>SQLite (async)</entry>
            <entry>64005.90</entry>
            <entry>105391.29</entry>
            <entry>66075.51</entry>
            <entry>66566.43</entry>
            <entry>75509.78</entry>
          </row>

          <row>
            <entry>memfile (async)</entry>
            <entry>76832.16</entry>
            <entry>2636018.56</entry>
            <entry>76542.50</entry>
            <entry>79188.81</entry>
            <entry>717145.51</entry>
          </row>


          <row>
            <entry>MySQL (sync)</entry>
            <entry>35.86</entry>
            <entry>9461.10</entry>
            <entry>35.43</entry>
            <entry>36.11</entry>
            <entry>2392.12</entry>
          </row>

          <row>
            <entry>SQLite (sync)</entry>
            <entry>16.31</entry>
            <entry>67036.11</entry>
            <entry>16.76</entry>
            <entry>16.37</entry>
            <entry>16771.39</entry>
          </row>

          <row>
            <entry>memfile (sync)</entry>
            <entry>25.28</entry>
            <entry>3460207.61</entry>
            <entry>25.29</entry>
            <entry>25.43</entry>
            <entry>865070.90</entry>
          </row>

        </tbody>
      </tgroup>
      </table>

      <mediaobject>
        <imageobject>
          <imagedata fileref="performance-results-graph2.png" format="PNG"/>
        </imageobject>
        <textobject>
          <phrase>Optimized performance measurements</phrase>
        </textobject>
        <caption>
          <para>Graphical representation of the optimized performance
          results presented in table <xref linkend="tbl-optim-perf-results"
          />.</para>
        </caption>
      </mediaobject>

    </section>

    <section>
      <title>Conclusions</title>
      <para>
        Improvements gained by introducing support for precompiled
        statements in MySQL is somewhat disappointing - between 6 and
        29%.  On the other hand, the improvement in SQLite is
        surprisingly high - the efficiency is more than doubled.
      </para>
      <para>
        Compiled statements do not have any measureable impact on
        synchronous operations. That is as expected, because the major
        bottleneck is the disk performance.
      </para>
      <para>
        Compilation flags yield surprisingly high improvements for C++
        STL code. The memfile backend is in some operations is almost
        twice as fast.
      </para>

      <para>
        If synchronous operation is required, the current performance
        results are likely to be deemed inadequate. The limiting
        factor here is a disk access time. Even migrating to high
        performance 15,000 rpm disk is expected to only roughly double
        number of leases per second, compared to the current results.
        The reason is that to write a file to disk, at least two disk
        sector writes
        are required: the new content and i-node modification of the
        file. The easiest way to boost synchronous performance is to
        switch to SSD disks. Memory-backed RAM disks are also a viable
        solution. However, care should be taken to properly engineer
        backup strategy for such RAM disks.
      </para>

      <para>
        While the custom made backend (memfile) provides the best
        perfomance, it carries over all the limitations existing in
        the ISC DHCP4 code: there are no external tools to query or
        change database, the maintenance requires deep knowledge etc.
        Those flaws are not shared by usage of a proper database
        backend, like MySQL and SQLite. They both offer third party
        tools for administrative tasks, they are well documented and
        maintained. However, SQLite support for concurrent access is
        limiting in certain cases. Since all three investigated
        backends more than meet expected performance results, it is
        recommended to use MySQL as a first concrete database backend.
        Should this choice be rejected for any reason, the second
        recommended choice is SQLite.
      </para>

      <para>
        It should be emphasized that obtained measurements indicate
        only database performance and they cannot be directly
        translated to expected leases per second or queries per second
        performance by an actual server. The DHCP server must do much
        more than just query the database to properly process client's
        message. The provided results should be considered as only rough
        estimates. They can also be used for relative comparisons
        between backends.
      </para>

    </section>

    <section>
      <title>Possible further optimizations</title>
      <para>
        For basic measurements the code was compiled with -g -O0
        flags. For optimized measurements the benchmarking code was
        compiled with -Ofast (optimize for speed). In both cases, the
        same backend (MySQL or SQLite) library was used. It may be
        useful to recompile the libraries (or the whole server in case
        of MySQL) with -Ofast.
      </para>
      <para>
        There are many MySQL parameters that various sources recommend
        to improve performance. They were not investigated further.
      </para>
      <para>
        Currently all operations are conducted on one by one
        basis. Each operation is treated as a separate
        transaction. Grouping N operations together will potentially
        bring almost N fold increase in synchronous operations. Such a
        feature is present in ISC DHCP4 and is called cache-threshold.
        Extension for this benchmark in this regard should be
        considered.  That affects only write operations (insert,
        update and delete). Read operations (search) are expected to
        be barely affected.
      </para>
      <para>
        Multi-threaded or multi-process benchmark may be considered in
        the future. It may be somewhat difficult as only some backends
        support concurrent access.
      </para>
    </section>

  </chapter>

  <chapter id="perfdhcp">
    <title>perfdhcp</title>
    <section>
      <title>Purpose</title>
      <para>
        Evaluation of the performance of a DHCP server requires that it
        be tested under varying traffic loads.  perfdhcp is a testing
        tool with the capability to create traffic loads
        and generate statistics from the results.  Additional features,
        such as the ability to send customised DHCP packets, allow it to
        be used in a wide range of functional testing.
      </para>
    </section>
    <section id="perfdhcp-key-features">
      <title>Key features</title>
      <para>
        perfdhcp has a number of command line switches to
        control DHCP message exchanges. Currently they fall into
        the following categories:
        <itemizedlist>
          <listitem>
            <para>
              Rate control - control how many DHCP exchanges
              are initiated within a period of time. The tool can also simulate
              best effort conditions by attempting to initiate as many DHCP
              packet exchanges as possible within a unit of time.
            </para>
          </listitem>
          <listitem>
            <para>
              Test exit specifiers - control the conditions for test
              completion, including the number of initiated exchanges,
              the test period orthe maximum number of dropped packets.
            </para>
          </listitem>
          <listitem>
            <para>
              Packet templates - specify files containing packet templates that
              are used by perfdhcp to create custom DHCP messages. The tool
              allows the specification of a number of values indicating
              offsets of values within a packet that are set by the tool.
            </para>
          </listitem>
          <listitem>
            <para>
              Reporting - for each test produce a set of performance data
              including the achieved packet exchange rate (server performance).
              There are a number of diagnostic selectors available that
              enable periodic (intermediate) reporting, printing of packet timestamps,
              and the listing of detailed information about internal perfdhcp
              states (for debugging).
            </para>
          </listitem>
          <listitem>
            <para>
              Different mode of operations - specify the DHCP protocol used
              (v4 or v6), two-way or four-way exchanges, use of the
              Rapid Commit option for DHCPv6.
            </para>
          </listitem>
          <listitem>
            <para>
              IP layer options - specify the local/remote address, local interface
              and local port to be used for communication with DHCP server.
            </para>
          </listitem>
        </itemizedlist>
      </para>
    </section>
    <section id="perfdhcp-command-line">
      <title>Command line options</title>
      <para>
        The following "help" output from the tool describes the
        command line switches.  This summary also lists the tool's
        possible exit codes as well as describing the
        error counters printed when the test is complete:
        <screen>$ <userinput>perfdhcp -h</userinput>
<![CDATA[perfdhcp [-hv] [-4|-6] [-r<rate>] [-t<report>] [-R<range>] [-b<base>]
    [-n<num-request>] [-p<test-period>] [-d<drop-time>] [-D<max-drop>]
    [-l<local-addr|interface>] [-P<preload>] [-a<aggressivity>]
    [-L<local-port>] [-s<seed>] [-i] [-B] [-c] [-1]
    [-T<template-file>] [-X<xid-offset>] [-O<random-offset]
    [-E<time-offset>] [-S<srvid-offset>] [-I<ip-offset>]
    [-x<diagnostic-selector>] [-w<wrapped>] [server]

The [server] argument is the name/address of the DHCP server to
contact.  For DHCPv4 operation, exchanges are initiated by
transmitting a DHCP DISCOVER to this address.

For DHCPv6 operation, exchanges are initiated by transmitting a DHCP
SOLICIT to this address.  In the DHCPv6 case, the special name 'all'
can be used to refer to All_DHCP_Relay_Agents_and_Servers (the
multicast address FF02::1:2), or the special name 'servers' to refer
to All_DHCP_Servers (the multicast address FF05::1:3).  The [server]
argument is optional only in the case that -l is used to specify an
interface, in which case [server] defaults to 'all'.

The default is to perform a single 4-way exchange, effectively pinging
the server.
The -r option is used to set up a performance test, without
it exchanges are initiated as fast as possible.

Options:
-1: Take the server-ID option from the first received message.
-4: DHCPv4 operation (default). This is incompatible with the -6 option.
-6: DHCPv6 operation. This is incompatible with the -4 option.
-a<aggressivity>: When the target sending rate is not yet reached,
    control how many exchanges are initiated before the next pause.
-b<base>: The base mac, duid, IP, etc, used to simulate different
    clients.  This can be specified multiple times, each instance is
    in the <type>=<value> form, for instance:
    (and default) mac=00:0c:01:02:03:04.
-d<drop-time>: Specify the time after which a request is treated as
    having been lost.  The value is given in seconds and may contain a
    fractional component.  The default is 1 second.
-E<time-offset>: Offset of the (DHCPv4) secs field / (DHCPv6)
    elapsed-time option in the (second/request) template.
    The value 0 disables it.
-h: Print this help.
-i: Do only the initial part of an exchange: DO or SA, depending on
    whether -6 is given.
-I<ip-offset>: Offset of the (DHCPv4) IP address in the requested-IP
    option / (DHCPv6) IA_NA option in the (second/request) template.
-l<local-addr|interface>: For DHCPv4 operation, specify the local
    hostname/address to use when communicating with the server.  By
    default, the interface address through which traffic would
    normally be routed to the server is used.
    For DHCPv6 operation, specify the name of the network interface
    via which exchanges are initiated.
-L<local-port>: Specify the local port to use
    (the value 0 means to use the default).
-O<random-offset>: Offset of the last octet to randomize in the template.
-P<preload>: Initiate first <preload> exchanges back to back at startup.
-r<rate>: Initiate <rate> DORA/SARR (or if -i is given, DO/SA)
    exchanges per second.  A periodic report is generated showing the
    number of exchanges which were not completed, as well as the
    average response latency.  The program continues until
    interrupted, at which point a final report is generated.
-R<range>: Specify how many different clients are used. With 1
    (the default), all requests seem to come from the same client.
-s<seed>: Specify the seed for randomization, making it repeatable.
-S<srvid-offset>: Offset of the server-ID option in the
    (second/request) template.
-T<template-file>: The name of a file containing the template to use
    as a stream of hexadecimal digits.
-v: Report the version number of this program.
-w<wrapped>: Command to call with start/stop at the beginning/end of
    the program.
-x<diagnostic-selector>: Include extended diagnostics in the output.
    <diagnostic-selector> is a string of single-keywords specifying
    the operations for which verbose output is desired.  The selector
    keyletters are:
   * 'a': print the decoded command line arguments
   * 'e': print the exit reason
   * 'i': print rate processing details
   * 'r': print randomization details
   * 's': print first server-id
   * 't': when finished, print timers of all successful exchanges
   * 'T': when finished, print templates
-X<xid-offset>: Transaction ID (aka. xid) offset in the template.

DHCPv4 only options:
-B: Force broadcast handling.

DHCPv6 only options:
-c: Add a rapid commit option (exchanges will be SA).

The remaining options are used only in conjunction with -r:

-D<max-drop>: Abort the test if more than <max-drop> requests have
    been dropped.  Use -D0 to abort if even a single request has been
    dropped.  If <max-drop> includes the suffix '%', it specifies a
    maximum percentage of requests that may be dropped before abort.
    In this case, testing of the threshold begins after 10 requests
    have been expected to be received.
-n<num-request>: Initiate <num-request> transactions.  No report is
    generated until all transactions have been initiated/waited-for,
    after which a report is generated and the program terminates.
-p<test-period>: Send requests for the given test period, which is
    specified in the same manner as -d.  This can be used as an
    alternative to -n, or both options can be given, in which case the
    testing is completed when either limit is reached.
-t<report>: Delay in seconds between two periodic reports.

Errors:
- tooshort: received a too short message
- orphans: received a message which doesn't match an exchange
   (duplicate, late or not related)
- locallimit: reached to local system limits when sending a message.

Exit status:
The exit status is:
0 on complete success.
1 for a general error.
2 if an error is found in the command line arguments.
3 if there are no general failures in operation, but one or more
  exchanges are not successfully completed.
]]>
        </screen>
      </para>
    </section>
    <section id="starting-perfdhcp">
      <title>Starting perfdhcp</title>
      <para>
        In order to run a performance test, at least two separate systems
        have to be installed: client and server. The first one must have
        perfdhcp installed, and the latter must be running the DHCP server
        (either v4 or v6). If only single system is available the client
        and server can be run on virtual machines (running on the same
        physical system) but in this case performance data may be heavily
        impacted by the overhead involved in running such the virtual
        machines.
      </para>
      <para>
        Currently, perfdhcp is seen from the server perspective as relay agent.
        This simplifies its implementation: specifically there is no need to
        receive traffic sent to broadcast addresses. However, it does impose
        a requirement that the IPv4
        address has to be set manually on the interface that will be used to
        communicate with the server. For example, if the DHCPv4 server is listening
        on the interface connected to the 172.16.1.0 subnet, the interface on client
        machine has to have network address assigned from the same subnet, e.g.
        <screen><userinput>#ifconfig eth3 172.16.1.2. netmask 255.255.255.0 up</userinput></screen>
      </para>
      <para>
        As DHCP uses low port numbers (67 for DHCPv4 relays and
        547 for DHCPv6), running perfdhcp with non-root privileges will
        usually result in the error message similar to this:
        <screen><userinput>$ perfdhcp -4 -l eth3 -r 100 all</userinput>
Error running perfdhcp: Failed to bind socket 3 to 172.16.1.2/port=67
        </screen>
        The '-L' command line switch allows the use of a custom local port.
        However, although the following command line will work:
        <screen><userinput>$ perfdhcp -4 -l eth3 -r 100 -L 10067 all</userinput></screen>
        in the standard configuration no responses will be received
        from the DHCP server because the server responds to default relay
        port 67.  A way to overcome this issue is to run
        perfdhcp as root.
      </para>

    </section>
    <section id="perfdhcp-commandline-examples">
      <title>perfdhcp command line examples</title>
      <para>
        In this section, a number of perfdhcp command line examples
        are presented as a quick start guide for new users. For the
        detailed list of command line options refer to
        <xref linkend="perfdhcp-command-line"/>.
      </para>
      <section id="perfdhcp-basic-usage">
        <title>Example: basic usage</title>
        <para>
          If server is listening on interface with IPv4 address 172.16.1.1,
          the simplest perfdhcp command line will look like:
          <screen><userinput># perfdhcp 172.16.1.1</userinput>
***Rate statistics***
Rate: 206.345

***Statistics for: DISCOVER-OFFER***
sent packets: 21641
received packets: 350
drops: 21291
orphans: 0

min delay: 9.022 ms
avg delay: 143.100 ms
max delay: 259.303 ms
std deviation: 56.074 ms
collected packets: 30

***Statistics for: REQUEST-ACK***
sent packets: 350
received packets: 268
drops: 82
orphans: 0

min delay: 3.010 ms
avg delay: 152.470 ms
max delay: 258.634 ms
std deviation: 56.936 ms
collected packets: 0
          </screen>
          Here, perfdhcp uses remote address 172.16.1.1 as a
          destination address and will use a suitable local interface for
          communication. Since, no rate control parameters have been specified,
          it will initiate DHCP exchanges at the maximum possible rate. Due to the server's
          performance limitation, it is likely that many of the packets will be dropped.
          The performance test will continue running until it is
          interrupted by the user (with ^C).
        </para>
        <para>
          The default performance statistics reported by perfdhcp have the
          following meaning:
          <itemizedlist>
            <listitem><para>Rate - number of packet exchanges (packet sent
            to the server and matching response received from the server)
            completed within a second.</para></listitem>
            <listitem><para>sent packets - total number of DHCP packets of
            a specific type sent to the server.</para></listitem>
            <listitem><para>received packets - total number of DHCP packets
            of specific type received from the server.</para></listitem>
            <listitem><para>drops - number of dropped packets for the
            particular exchange. The number of dropped packets is calculated as
            the difference between the number of sent packets and number of
            response packets received from the server.  In some cases, the
            server will have sent a reponse but perfdhcp execution ended before
            the reponse arrived. In such case this packet will be counted
            as dropped.</para></listitem>
            <listitem><para>orphans - number of packets that have been
            received from the server and did not match any packet sent by
            perfdhcp. This may occur if received packet has been sent
            to some other host or if then exchange timed out and
            the sent packet was removed from perfdhcp's list of packets
            awaiting a response.</para></listitem>
            <listitem><para>min delay - minimum delay that occured between
            sending the packet to the server and receiving a reponse from
            it.</para></listitem>
            <listitem><para>avg delay - average delay between sending the
            packet of the specific type the server and receiving a response
            from it.</para></listitem>
            <listitem><para>max delay - maximum delay that occured between
            sending the packet to the server and receiving a response from
            it.</para></listitem>
            <listitem><para>std deviation - standard deviation of the delay
            between sending the packet of a specific type to the server and
            receiving response from it.</para></listitem>
            <listitem><para>collected packets - number of sent packets that
            were garbage collected. Packets may get garbage collected when
            the waiting time for server a response exceeds value set with the
            '-d' (drop time) switch.
            <![CDATA[-d<drop-time>]]>.</para></listitem>
          </itemizedlist>
        </para>
        <para>
          Note: should multiple interfaces on the system running perfdhcp be
          connected to the same subnet, the interface to be used for the test
          can be specified using either the interface name:
          <screen><userinput># perfdhcp -l eth3</userinput></screen>
          or a local address assigned to it:
          <screen><userinput># perfdhcp -l 172.16.1.2</userinput></screen>
        </para>
      </section>
      <section id="perfdhcp-rate-control">
        <title>Example: rate control</title>
        <para>
          In the examples above perfdhcp initiates new exchanges with a best
          effort rate. With this setting, many packets are expected to be dropped
          by the server due to performance limitations. In many cases though, it is
          desired to verify that the server can handle an expected (reasonable) rate
          without dropping any packets. The following command is an example of such
          a test: it causes perfdhcp to initiate 300 four-way exchanges
          per second, and runs the test for 60 seconds:
          <screen><userinput># perfdhcp -l eth3 -p 60 -r 300</userinput>
***Rate statistics***
Rate: 256.683 exchanges/second, expected rate: 300 exchanges/second

***Statistics for: DISCOVER-OFFER***
sent packets: 17783
received packets: 15401
drops: 2382
orphans: 0

min delay: 0.109 ms
avg delay: 75.756 ms
max delay: 575.614 ms
std deviation: 60.513 ms
collected packets: 11

***Statistics for: REQUEST-ACK***
sent packets: 15401
received packets: 15317
drops: 84
orphans: 0

min delay: 0.536 ms
avg delay: 72.072 ms
max delay: 576.749 ms
std deviation: 58.189 ms
collected packets: 0
          </screen>
          Note that here, the packet drops for the DISCOVER-OFFER
          exchange have been significantly reduced (when compared with the
          output from the previous example) thanks to the setting of a
          reasonable rate. The non-zero number of packet drops and achieved
          rate (256/s) indicate that server's measured performance is lower than 300 leases
          per second. A further rate decrease should eliminate most of the packet
          drops and bring the achieved rate close to expected rate:
          <screen><userinput># perfdhcp -l eth3 -p 60 -r 100 -R 30</userinput>
***Rate statistics***
Rate: 99.8164 exchanges/second, expected rate: 100 exchanges/second

***Statistics for: DISCOVER-OFFER***
sent packets: 5989
received packets: 5989
drops: 0
orphans: 0

min delay: 0.023 ms
avg delay: 2.198 ms
max delay: 181.760 ms
std deviation: 9.429 ms
collected packets: 0

***Statistics for: REQUEST-ACK***
sent packets: 5989
received packets: 5989
drops: 0
orphans: 0

min delay: 0.473 ms
avg delay: 2.355 ms
max delay: 189.658 ms
std deviation: 5.876 ms
collected packets: 0
          </screen>
          There are now no packet drops, confirming that the server is able to
          handle a load of 100 leases/second.
          Note that the last parameter (-R 30) configures perfdhcp to simulate
          traffic from 30 distinct clients.
        </para>
      </section>
      <section id="perfdhcp-templates">
        <title>Example: templates</title>
        <para>
          By default the DHCP messages are formed with default options.  With
          template files, it is possible to define a custom packet format.
        </para>
        <para>
          The template file names are specified with <![CDATA[-T<template-file>]]>
          command line option. This option can be specified zero, one or two times.
          The first occurence of this option refers to DISCOVER or SOLICIT message
          and the second occurence refers to REQUEST (DHCPv4 or DHCPv6) message.
          If -T option occurs only once the DISCOVER or SOLICIT message will be
          created from the template and the REQUEST message will be created
          dynamically (without the template). Note that each template file
          holds data for exactly one DHCP message type. Templates for multiple
          message types must not be combined in the single file.
          The content in template files is encoded as series of ASCII hexadecimal
          digits (each byte represented by two ASCII chars 00..FF). Data in a
          template file is laid in network byte order and it can be used on the
          systems with different endianness.
          perfdhcp forms the packet by replacing parts of the message buffer read
          from the file with variable data such as elapsed time, hardware address, DUID
          etc. The offsets where such variable data is placed is specific to the
          template file and have to be specified from the command line. Refer to
          <xref linkend="perfdhcp-command-line"/> to find out how to
          specify offsets for particular options and fields. With the following
          command line the DHCPv6 SOLICIT and REQUEST packets will be formed from
          solicit.hex and request6.hex packets:
          <screen><userinput># perfdhcp -6 -l eth3 -r 100 -R 20 -T solicit.hex -T request6.hex -O 21 -E 84 -S 22 -I 40 servers</userinput>
***Rate statistics***
Rate: 99.5398 exchanges/second, expected rate: 100 exchanges/second

***Statistics for: SOLICIT-ADVERTISE***
sent packets: 570
received packets: 569
drops: 1
orphans: 0

min delay: 0.259 ms
avg delay: 0.912 ms
max delay: 6.979 ms
std deviation: 0.709 ms
collected packets: 0

***Statistics for: REQUEST-REPLY***
sent packets: 569
received packets: 569
drops: 0
orphans: 0

min delay: 0.084 ms
avg delay: 0.607 ms
max delay: 6.490 ms
std deviation: 0.518 ms
collected packets: 0
          </screen>
          where the switches have the following meaning:
          <itemizedlist>
            <listitem><para>two occurences of -O 21 - DUID's last octet
            positions in SOLICIT and REQUEST respectively.</para></listitem>
            <listitem><para>-E 84 - elapsed time option position in the
            REQUEST template</para></listitem>
            <listitem><para>-S 22 - server id position in the REQUEST
            template</para></listitem>
            <listitem><para>-I 40 - IA_NA option position in the REQUEST
            template</para></listitem>
          </itemizedlist>
        </para>
        <para>
          The offsets of options indicate where they begin in the packet.
          The only exception from this rule is <![CDATA[-O<random-offset>]]>
          option that specifies the end of the DUID (DHCPv6) or MAC address
          (DHCPv4). Depending on the number of simulated clients
          (see <![CDATA[-R<random-range>]]> command line option) perfdhcp
          will be randomizing bytes in packet buffer starting from this
          position backwards. For the number of simulated clients
          <![CDATA[<=]]> 256 only one octet (at random-offset position)
          will be ranomized, for the number of clients <![CDATA[<=]]> 65536
          two octets (at random-offset and random-offset-1)
          will be randmized etc.
        </para>
      </section>
    </section>
  </chapter>
</book>