[2040] README migrated to dhcp-perf-guide

tests/tools/dhcp-ubench/Makefile

@@ -8,6 +8,8 @@ SQLITE_LDFLAGS=`pkg-config sqlite3 --libs`
 
 all: mysql_ubench sqlite_ubench memfile_ubench
 
+doc: dhcp-perf-guide.html dhcp-perf-guide.pdf
+
 mysql_ubench.o: mysql_ubench.cc mysql_ubench.h
 	$(CXX) $< -c $(CFLAGS) $(MYSQL_CFLAGS)
 

tests/tools/dhcp-ubench/README

@@ -1,149 +0,0 @@
-
- DHCP micro-benchmarks
------------------------
-
- Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC")
-
- by Tomasz Mrugalski
-
-This directory contains simplified prototypes for various DB back-ends
-that are planned or considered as a backend engine for BIND10 DHCP.
-Athough trivial now, they are expected to evolve into useful tools
-that will allow users to measure performance in their specific environment.
-
-Currently the following benchmarks are planned:
-- in memory+flat file
-- SQLite
-- MySQL
-
-As they require additional (sometimes heavy) dependencies, they are not
-built by default. Actually, their build system completely separated.
-It will be eventually merged with the main BIND10 makefile system, but
-that is a low priority for now.
-
-All planned benchmarks will follow the same pattern.
-
-1. prepare operation (connect to a database, create a file etc.)
-2. Measure timestamp 0
-3. commit new lease4 (step repeated X times)
-4. Measure timestamp 1
-5. search for random lease4 (step repeated X times)
-6. Measure timestamp 2
-7. update existing lease4 (step repeated X times)
-8. Measure timestamp 3
-9. delete existing lease4 (step repeated X times)
-10. Measure timestamp 4
-11. Print out statistics, based on X and measured timestamps.
-
-Although this approach does not attempt to simulate actual DHCP server
-operation that has mix of all steps intervening, it answers the
-questions about basic database strenghts and weak points. In particular
-it can show what is the impact of specific DB optimizations, like
-changing engine, optimizing for writes/reads etc.
-
-The framework attempts to do the same amount of operations for every
-backend thus allowing fair complarison between them.
-
- MySQL backend
----------------
-
-MySQL backend requires MySQL client development libraries. It uses
-mysql_config tool (that works similar to pkg-config) to discover
-required compilation and linking options. To install required packages
-on Ubuntu, use the following command:
-
-sudo apt-get install mysql-client mysql-server libmysqlclient-dev
-
-Running MySQL server is required. Make sure that you have your setup
-configured so there is a user that is able to create databases.
-
-Before running tests, you need to initialize your database. You can
-use mysql-schema.sql script for that purpose.
-
-WARNING: It will drop existing Kea database. Do not run this on
-your production server. Assuming your MySQL user is kea, you can
-initialize your test database by:
-
-mysql -u kea -p < mysql.schema
-
-After that step, you are ready to run the test:
-
-./mysql_ubench
-
-or
-
-./mysql_ubench > results-mysql.txt
-
-Redirecting output to a file is important, because for each operation
-there is a single character printed to show progress. If you have a slow
-terminal, this may considerably affect test perfromance. On the other hand,
-printing something after each operation is required, as poor DB setting
-may slow down operations to around 20 per second. Observant user is expected
-to note that initial dots are printed too slowly and abort the test.
-
-Currently all parameters are hardcoded. To modify them, one needs to
-modify source code and recompile. Fortunately, that is quite easy.
-To modify MySQL parameters, see main() method at the end of mysql_ubench.c
-file. That is the plase where one can modify MySQL connection
-parameters (MySQL server hostname, user and password and database name).
-
-One parameter that has huge impact on performance is a a backend engine.
-You can get a list of engines of your MySQL implementation by using
-
-> show engines;
-
-in your mysql client. Two notable engines are MyISAM and InnoDB. You
-can tweak it by using the following command in mysql:
-
-> alter table lease4 engine=MyISAM;
-
-or
-
-> alter table lease4 engine=InnoDB;
-
- SQLite backend
-----------------
-SQLite backend requires both sqlite3 development and run-time package. Their
-names may vary from system to system, but on Ubuntu 12.04 they are called
-sqlite3 libsqlite3-dev. To install them, use the following command:
-
-sudo apt-get install sqlite3 libsqlite3-dev
-
-To run SQLite3 tests, first create the database:
-
-cat sqlite.schema | sqlite3 sqlite.db
-
-A new database called sqlite.db will be created. That is the default name used
-by sqlite_ubench test. If you prefer other name, make sure you update
-sqlite_ubench.cc accordingly. Once the database is created, you can run
-tests:
-
-./sqlite_ubench > results-sqlite.txt
-
-Make sure to tweak number of iterations in sqlite_ubench.cc. See num variable
-in main() function near the end of sqlite_ubench.cc file.
-
-To improve performance, asynchronous mode is used (PRAGMA synchronous = OFF).
-If you do not like it, modify SQLite_uBenchmark::connect() by commenting it out
-and suffer the pain of having around 10 inserts per seconds.
-
-Another tweakable feature is journal mode. It can be turned to several modes
-of operation. Its value can be modified in SQLite_uBenchmark::connect().
-See http://www.sqlite.org/pragma.html#pragma_journal_mode for detailed
-explanantion.
-
- Performance optimizations
----------------------------
-
-The following section is a collection of loose notes, ideas and comments that
-could be investigated to possibly improve (or thrash) performance. Please
-consider them as developer's scratchpad only. However, if you are experienced
-programmer or a DB admin, you are welcome to try some of them. ISC engineers
-are more than welcome to discuss pros and cons of various approaches on
-bind10-dev@lists.isc.org list.
-
-- SQLite: Excellent article that discusses performance optimizations:
-  - http://stackoverflow.com/questions/1711631/how-do-i-improve-the-performance-of-sqlite
-  - http://blog.quibb.org/2010/08/fast-bulk-inserts-into-sqlite/
-  - use prepared statements
-  - use journal mode in memory (or disable it completely)

tests/tools/dhcp-ubench/dhcp-perf-guide.xml

@@ -32,6 +32,10 @@
     <copyright>
       <year>2012</year><holder>Internet Systems Consortium, Inc.</holder>
     </copyright>
+    <author>
+      <firstname>Tomasz</firstname>
+      <surname>Mrugalski</surname>
+    </author>
 
     <abstract>
       <para>BIND 10 is a framework that features Domain Name System
@@ -71,6 +75,43 @@
       pros an cons of various optimization techniques and
       tools.
     </para>
+
+  </chapter>
+
+  <chapter id="dhcp4">
+    <title>ISC DHCP 4.x</title>
+    <para>
+      TODO: Write something about ISC DHCP4 here.
+    </para>
+  </chapter>
+
+  <chapter id="kea">
+    <title>Kea</title>
+    <para>
+
+    </para>
+
+    <section>
+      <title>Backend performance evaluation</title>
+      <para>
+        Kea will support several different database backends, using
+        both popular databases (like MySQL or SQLite) and
+        custom-developed solutions (like in-memory database).  BIND 10
+        source code features set of performance microbenchmarks.
+        These are small tools written in C/C++ that simulate expected
+        DHCP server behaviour and evaluate the performance of
+        considered databases.
+      </para>
+
+      <para>
+        Those benchmarks are stored in tests/tools/dhcp-ubench
+        directory.  This directory contains simplified prototypes for
+        various DB back-ends that are planned or considered as a
+        backend engine for BIND10 DHCP.  Athough trivial now, they are
+        expected to evolve into useful tools that will allow users to
+        measure performance in their specific environment.
+      </para>
+
     <para>
       Currently the following benchmarks are implemented:
       <itemizedlist>
@@ -82,7 +123,7 @@
 
     <para>
       As they require additional (sometimes heavy) dependencies, they are not
-      built by default. Actually, their build system completely separated.
+      built by default. Actually, their build system is completely separated.
       It will be eventually merged with the main BIND10 makefile system, but
       that is a low priority for now.
     </para>
@@ -92,13 +133,13 @@
       <orderedlist>
         <listitem><para>prepare operation (connect to a database, create a file etc.)</para></listitem>
         <listitem><para>Measure timestamp 0</para></listitem>
-        <listitem><para>Commit new lease4 (step repeated X times)</para></listitem>
+        <listitem><para>Commit new lease4 (repeated X times)</para></listitem>
         <listitem><para>Measure timestamp 1</para></listitem>
-        <listitem><para>Search for random lease4 (step repeated X times)</para></listitem>
+        <listitem><para>Search for random lease4 (repeated X times)</para></listitem>
         <listitem><para>Measure timestamp 2</para></listitem>
-        <listitem><para>Update existing lease4 (step repeated X times)</para></listitem>
+        <listitem><para>Update existing lease4 (repeated X times)</para></listitem>
         <listitem><para>Measure timestamp 3</para></listitem>
-        <listitem><para>Delete existing lease4 (step repeated X times)</para></listitem>
+        <listitem><para>Delete existing lease4 (repeated X times)</para></listitem>
         <listitem><para>Measure timestamp 4</para></listitem>
         <listitem><para>Print out statistics, based on X and measured timestamps.</para></listitem>
       </orderedlist>
@@ -114,44 +155,135 @@
       The framework attempts to do the same amount of operations for every
       backend thus allowing fair complarison between them.
     </para>
+    </section>
 
-  </chapter>
+    <section id="mysql-backend">
+      <title>MySQL backend</title>
+      <para>MySQL backend requires MySQL client development libraries. It uses
+      mysql_config tool (that works similar to pkg-config) to discover required
+      compilation and linking options. To install required packages on Ubuntu,
+      use the following command:
 
-  <chapter id="dhcp4">
-    <title>ISC DHCP 4.x</title>
-    <para>
-      TODO: Write something about ISC DHCP4 here.
-    </para>
-  </chapter>
+      <screen>$ <userinput>sudo apt-get install mysql-client mysql-server libmysqlclient-dev</userinput></screen>
 
-  <chapter id="kea">
-    <title>Kea</title>
-    <para>
+      Make sure that MySQL server is running. Make sure that you have your setup
+      configured so there is a user that is able to modify used database.</para>
 
-    </para>
+      <para>Before running tests, you need to initialize your database. You can
+      use mysql.schema script for that purpose. WARNING: It will drop existing
+      Kea database. Do not run this on your production server. Assuming your
+      MySQL user is kea, you can initialize your test database by:
 
-    <section>
-      <title>Backend performance evaluation</title>
-      <para>
-        Kea will support several different database backends, using
-        both popular databases (like MySQL or SQLite) and
-        custom-developed solutions (like in-memory database).  BIND 10
-        source code features set of performance microbenchmarks.
-        These are small tools written in C/C++ that simulate expected
-        DHCP server behaviour and evaluate the performance of
-        considered databases.
+      <screen>$ <userinput>mysql -u kea -p &lt; mysql.schema</userinput></screen>
       </para>
 
-      <para>
-        Those benchmarks are stored in tests/tools/dhcp-ubench
-        directory.  This directory contains simplified prototypes for
-        various DB back-ends that are planned or considered as a
-        backend engine for BIND10 DHCP.  Athough trivial now, they are
-        expected to evolve into useful tools that will allow users to
-        measure performance in their specific environment.
+      <para>After database is initialized, you are ready to run the test:
+      <screen>$ <userinput>./mysql_ubench</userinput></screen>
+
+      or
+
+      <screen>$ <userinput>./mysql_ubench &gt; results->mysql.txt</userinput></screen>
+
+      Redirecting output to a file is important, because for each operation
+      there is a single character printed to show progress. If you have a slow
+      terminal, this may considerably affect test perfromance. On the other hand,
+      printing something after each operation is required, as poor DB setting
+      may slow down operations to around 20 per second. Observant user is expected
+      to note that initial dots are printed too slowly and abort the test.</para>
+
+      <para>Currently all parameters are hardcoded. To modify them, one needs to
+      modify source code and recompile. Fortunately, that is quite easy.
+      To modify MySQL parameters, see main() method at the end of mysql_ubench.c
+      file. That is the plase where one can modify MySQL connection
+      parameters (MySQL server hostname, user and password and database name).</para>
+
+      <section>
+        <title>MySQL tweaks</title>
+        <para>To reconfigure mysql_ubench parameters, a modification to the source
+        code and recompilation is required. All parameters are listed in main()
+        function in mysql_ubench.cc file, near the end of the file. Currently supported
+        parameter are (default values specified in brackets):
+        <orderedlist>
+          <listitem><para>hostname - name of the host to connect to ("localhost")</para></listitem>
+          <listitem><para>user - MySQL username ("root")</para></listitem>
+          <listitem><para>passwd - MySQL password ("secret")</para></listitem>
+          <listitem><para>dbname - MySQL database name ("kea")</para></listitem>
+          <listitem><para>num - number of iterations (100)</para></listitem>
+          <listitem><para>sync - should the operations be performend in synchronous (true)
+          or asynchronous (false) manner (true)</para></listitem>
+          <listitem><para>verbose - should the test print out progress? (true)</para></listitem>
+        </orderedlist>
+        </para>
+
+        <para>One parameter that has huge impact on performance is a a backend engine.
+        You can get a list of engines of your MySQL implementation by using
+
+        <screen>&gt; <userinput>show engines;</userinput></screen>
+
+        in your mysql client. Two notable engines are MyISAM and InnoDB. mysql_ubench will
+        use MyISAM for synchronous mode and InnoDB for asynchronous.</para>
+    </section>
+    </section>
+
+
+    <section id="sqlite-ubench">
+      <title>SQLite-ubench</title>
+      <para>SQLite backend requires both sqlite3 development and run-time package. Their
+      names may vary from system to system, but on Ubuntu 12.04 they are called
+      sqlite3 libsqlite3-dev. To install them, use the following command:
+
+      <screen>&gt; <userinput>sudo apt-get install sqlite3 libsqlite3-dev</userinput></screen>
+
+      Before running the test the database has to be created. Use the following command for that:
+      <screen>&gt; <userinput>cat sqlite.schema | sqlite3 sqlite.db</userinput></screen>
+
+      A new database called sqlite.db will be created. That is the default name used
+      by sqlite_ubench test. If you prefer other name, make sure you update
+      sqlite_ubench.cc accordingly.</para>
+
+      <para>Once the database is created, you can run tests:
+      <screen>&gt; <userinput>./sqlite_ubench</userinput></screen>
+      or
+      <screen>&gt; <userinput>./sqlite_ubench > results-sqlite.txt</userinput></screen>
       </para>
 
+      <section id="sqlite-tweaks">
+        <title>SQLite tweaks</title>
+        <para>To reconfigure sqlite_ubench parameters, a modification to the source
+        code and recompilation is required. All parameters are listed in main()
+        function in sqlite_ubench.cc file, near the end of the file. Currently supported
+        parameter are (default values specified in brackets):
+        <orderedlist>
+          <listitem><para>filename - name of the database file ("sqlite.db")</para></listitem>
+          <listitem><para>num - number of iterations (100)</para></listitem>
+          <listitem><para>sync - should the operations be performend in synchronous (true)
+          or asynchronous (false) manner (true)</para></listitem>
+          <listitem><para>verbose - should the test print out progress? (true)</para></listitem>
+        </orderedlist>
+        </para>
+
+        <para>SQLite can run in asynchronous or synchronous mode. This
+        mode can be controlled by using sync parameter. It is set
+        using (PRAGMA synchronous = ON or OFF).</para>
+
+        <para>Another tweakable feature is journal mode. It can be
+        turned to several modes of operation. Its value can be
+        modified in SQLite_uBenchmark::connect().  See
+        http://www.sqlite.org/pragma.html#pragma_journal_mode for
+        detailed explanantion.</para>
+      </section>
     </section>
+
+    <section id="memfile-ubench">
+      <title>memfile-ubench</title>
+      <para> TODO </para>
+
+      <section id="memfile-tweaks">
+        <title>SQLite tweaks</title>
+        <para> ... </para>
+      </section>
+    </section>
+
   </chapter>
 
   <chapter id="perfdhcp">