Sync with #327

git-svn-id: svn://bind10.isc.org/svn/bind10/branches/vorner-recursor-timeouts@3606 e5f2f494-b856-4b98-b285-d166d9295462

ChangeLog

@@ -1,6 +1,156 @@
+  TBD.  [func]      vorner
+	New temporary logging function available in isc::log. It is used by
+	b10-recurse.
+	(Trac #393, r3602)
+
+  TBD.  [func]      vorner
+	The b10-recursive is configured through config manager.
+	It has "listen_on" and "forward_addresses" options.
+	(Trac #389, r3448)
+
+  116.	[bug]		jerry
+	src/bin/xfrout: Xfrout and Auth will communicate by long tcp
+	connection, Auth needs to make a new connection only on the first
+	time or if an error occurred.
+	(Trac #299, svn r3482)
+
+  115.	[func]*		jinmei
+	src/lib/dns: Changed DNS message flags and section names from
+	separate classes to simpler enums, considering the balance between
+	type safety and usability.  API has been changed accordingly.
+	More documentation and tests were provided with these changes.
+	(Trac #358, r3439)
+
+  114.	[build]		jinmei
+	Supported clang++.  Note: Boost >= 1.44 is required.
+	(Trac #365, svn r3383)
+
+  113.	[func]*		zhanglikun
+	Folder name 'utils'(the folder in /src/lib/python/isc/) has been
+	renamed	to 'util'. Programs that used 'import isc.utils.process'
+	now need to use 'import isc.util.process'. The folder
+	/src/lib/python/isc/Util is removed since it isn't used by any
+	program. (Trac #364, r3382)
+
+  112.	[func]		zhang likun
+	Add one mixin class to override the naive serve_forever() provided
+	in python library socketserver. Instead of polling for shutdwon
+	every poll_interval seconds, one socketpair is used to wake up
+	the waiting server.(Trac #352, svn r3366)
+
+  111.	[bug]*   zhanglikun, Michal Vaner
+	Make sure process xfrin/xfrout/zonemgr/cmdctl can be stoped
+	properly when user enter "ctrl+c" or 'Boss shutdown' command
+	through	bindctl.
+
+	The ZonemgrRefresh.run_timer and NotifyOut.dispatcher spawn
+	a thread themselves.
+	(Trac #335, svn r3273)
+
+  110.  [func]      Michal Vaner
+	Added isc.net.check module to check ip addresses and ports for
+	correctness and isc.net.addr to hold IP address. The bind10, xfrin
+	and cmdctl programs are modified to use it.
+	(Trac #353, svn r3240)
+
+  109.  [func]		naokikambe
+	Added the initial version of the stats module for the statistics
+	feature of BIND 10, which supports the restricted features and
+	items and reports via bindctl command (Trac #191, r3218)
+	Added the document of the stats module, which is about how stats
+	module collects the data (Trac #170, [wiki:StatsModule])
+
+  108.	[func]		jerry
+	src/bin/zonemgr: Provide customizable configurations for
+	lowerbound_refresh, lowerbound_retry, max_transfer_timeout and
+	jitter_scope. (Trac #340, r3205)
+
+  107.  [func]       zhang likun
+	Remove the parameter 'db_file' for command 'retransfer' of
+	xfrin module. xfrin.spec will not be generated by script.
+	(Trac #329, r3171)
+
+  106.  [bug]       zhang likun
+	When xfrin can't connect with one zone's master, it should tell
+	the bad news to zonemgr, so that zonemgr can reset the timer for
+	that zone. (Trac #329, r3170)
+
+  105.  [bug]       Michal Vaner
+	Python processes: they no longer take 100% CPU while idle
+	due to a busy loop in reading command session in a nonblocking way.
+	(Trac #349, svn r3153), (Trac #382, svn r3294)
+
+  104.	[bug]		jerry
+	bin/zonemgr: zonemgr should be attempting to refresh expired zones.
+	(Trac #336, r3139)
+				   
+  103.	[bug]		jerry
+	lib/python/isc/log: Fixed an issue with python logging,
+	python log shouldn't die with OSError.(Trac #267, r3137)
+				   
+  102.	[build]		jinmei
+	Disable threads in ASIO to minimize build time dependency.
+	(Trac #345, r3100)
+
+  101.	[func]		jinmei
+	src/lib/dns: Completed Opcode and Rcode implementation with more
+	tests and documentation.  API is mostly the same but the
+	validation was a bit tightened. (Trac #351, svn r3056)
+
+  100.  [func]      Michal Vaner
+	Python processes: support naming of python processes so
+	they're not all called python3.
+	(Trac #322, svn r3052)
+
+  99.	[func]*		jinmei
+	Introduced a separate EDNS class to encapsulate EDNS related
+	information more cleanly.  The related APIs are changed a bit,
+	although it won't affect most of higher level applications.
+	(Trac #311, svn r3020)
+
+  98.	[build]		jinmei
+	The ./configure script now tries to search some common include
+	paths for boost header files to minimize the need for explicit
+	configuration with --with-boost-include. (Trac #323, svn r3006)
+
+  97.	[func]		jinmei
+	Added a micro benchmark test for query processing of b10-auth.
+	(Trac #308, svn r2982)
+
+  96.	[bug]		jinmei
+	Fixed two small issues with configure: Do not set CXXFLAGS so that
+	it can be customized; Make sure --disable-static works.
+	(Trac #325, r2976)
+
+bind10-devel-20100917 released on September 17, 2010 
+
+  95.	[doc]		jreed
+	Add b10-zonemgr manual page. Update other docs to introduce
+	this secondary manager. (Trac #341, svn r2951)
+
+  95.	[bug]		jreed
+	bin/xfrout and bin/zonemgr: Fixed some stderr output.
+	(Trac #342, svn r2949)
+
+  94.	[bug]		jelte
+  	bin/xfrout:  Fixed a problem in xfrout where only 2 or 3 RRs
+	were used per DNS message in the xfrout stream.
+	(Trac #334, r2931)
+
+  93.	[bug]		jinmei
+	lib/datasrc: A DS query could crash the library (and therefore,
+	e.g. the authoritative server) if some RR of the same apex name
+	is stored in the hot spot cache.  (Trac #307, svn r2923)
+
+  92.	[func]*		jelte
+	libdns_python (the python wrappers for libdns++) has been renamed
+	to pydnspp (Python DNS++). Programs and libraries that used
+	'import libdns_python' now need to use 'import pydnspp'.
+	(Trac #314, r2902)
+
   91.	[func]*		jinmei
 	lib/cc: Use const pointers and const member functions for the API
-	as much as possible for safer operations.  Basically this does
+	as much as possible for safer operations.  Basically this does not
 	change the observable behavior, but some of the API were changed
 	in a backward incompatible manner.  This change also involves more
 	copies, but at this moment the overhead is deemed acceptable.
@@ -29,8 +179,8 @@
 	zone axfr/ixfr finishing, the server will notify its slaves.
 	(Trac #289, svn r2737)
 
-  86.   [func]		jerry
+  86.	[func]		jerry
-    	bin/zonemgr: Added zone manager module. The zone manager is one 
+	bin/zonemgr: Added zone manager module. The zone manager is one 
 	of the co-operating processes of BIND10, which keeps track of 
 	timers and other information necessary for BIND10 to act as a 
 	slave. (Trac #215, svn r2737)

README

@@ -17,7 +17,8 @@ This release includes the bind10 master process, b10-msgq message
 bus, b10-auth authoritative DNS server (with SQLite3 backend),
 b10-cmdctl remote control daemon, b10-cfgmgr configuration manager,
 b10-xfrin AXFR inbound service, b10-xfrout outgoing AXFR service,
-and a new libdns++ library for C++ with a python wrapper.
+b10-zonemgr secondary manager, and a new libdns++ library for C++
+with a python wrapper.
 
 Documentation is included and also available via the BIND 10
 website at http://bind10.isc.org/

configure.ac

@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.59])
-AC_INIT(bind10-devel, 20100701, bind10-dev@isc.org)
+AC_INIT(bind10-devel, 20101013, bind10-dev@isc.org)
 AC_CONFIG_SRCDIR(README)
 AM_INIT_AUTOMAKE
 AC_CONFIG_HEADERS([config.h])
@@ -17,6 +17,8 @@ AC_LANG([C++])
 # Identify the compiler: this check must be after AC_PROG_CXX and AC_LANG.
 AM_CONDITIONAL(USE_GXX, test "X${GXX}" = "Xyes")
 AC_CHECK_DECL([__SUNPRO_CC], [SUNCXX="yes"], [SUNCXX="no"])
+AC_CHECK_DECL([__clang__], [CLANGPP="yes"], [CLANGPP="no"])
+AM_CONDITIONAL(USE_CLANGPP, test "X${CLANGPP}" = "Xyes")
 
 # Linker options
 
@@ -40,6 +42,20 @@ AC_HELP_STRING([--enable-static-link],
   [enable_static_link=yes], [enable_static_link=no])
 AM_CONDITIONAL(USE_STATIC_LINK, test $enable_static_link = yes)
 
+# Check validity about some libtool options
+if test $enable_static_link = yes -a $enable_static = no; then
+	AC_MSG_ERROR([--enable-static-link requires --enable-static])
+fi
+if test $enable_shared = no; then
+	AC_MSG_ERROR([BIND 10 requires shared libraries to be built])
+fi
+
+# allow configuring without setproctitle.
+AC_ARG_ENABLE(setproctitle-check,
+AC_HELP_STRING([--disable-setproctitle-check],
+  [do not check for python setproctitle module (used to give nice names to python processes)]),
+  setproctitle_check=$enableval, setproctitle_check=yes)
+
 # OS dependent configuration
 SET_ENV_LIBRARY_PATH=no
 ENV_LIBRARY_PATH=LD_LIBRARY_PATH
@@ -154,6 +170,18 @@ fi
 AC_SUBST(PYTHON_LIB)
 LDFLAGS=$LDFLAGS_SAVED
 
+# Check for the setproctitle module
+if test "$setproctitle_check" = "yes" ; then
+    AC_MSG_CHECKING(for setproctitle module)
+    if "$PYTHON" -c 'import setproctitle' 2>/dev/null ; then
+        AC_MSG_RESULT(ok)
+    else
+        AC_MSG_RESULT(missing)
+        AC_MSG_ERROR([Missing setproctitle module. Either install it or provide --disable-setproctitle-check.
+In that case we will continue, but naming of python processes will not work.])
+    fi
+fi
+
 # TODO: check for _sqlite3.py module
 
 # Compiler dependent settings: define some mandatory CXXFLAGS here.
@@ -167,7 +195,6 @@ LDFLAGS=$LDFLAGS_SAVED
 # specify the default warning flags in CXXFLAGS and let specific modules
 # "override" the default.
 
-CXXFLAGS=-g
 werror_ok=0
 
 # SunStudio compiler requires special compiler options for boost
@@ -179,7 +206,6 @@ fi
 # gcc specific settings:
 if test "X$GXX" = "Xyes"; then
 B10_CXXFLAGS="-Wall -Wextra -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare"
-UNUSED_PARAM_ATTRIBUTE='__attribute__((unused))'
 
 # Certain versions of gcc (g++) have a bug that incorrectly warns about
 # the use of anonymous name spaces even if they're closed in a single
@@ -198,7 +224,6 @@ CXXFLAGS="$CXXFLAGS_SAVED"
 fi				dnl GXX = yes
 
 AM_CONDITIONAL(GCC_WERROR_OK, test $werror_ok = 1)
-AC_DEFINE_UNQUOTED(UNUSED_PARAM, $UNUSED_PARAM_ATTRIBUTE, Define to compiler keyword indicating a function argument is intentionally unused)
 
 # produce PIC unless we disable shared libraries. need this for python bindings.
 if test $enable_shared != "no" -a "X$GXX" = "Xyes"; then
@@ -236,7 +261,6 @@ AC_ARG_WITH(gtest,
 [  --with-gtest=PATH       specify a path to gtest header files (PATH/include) and library (PATH/lib)],
     gtest_path="$withval", gtest_path="no")
 
-
 USE_LCOV="no"
 if test "$lcov" != "no"; then
 	# force gtest if not set
@@ -270,14 +294,33 @@ if test "$lcov" != "no"; then
 fi
 AC_SUBST(USE_LCOV)
 
+#
+# Configure Boost header path
+#
+# If explicitly specified, use it.
 AC_ARG_WITH([boost-include],
   AC_HELP_STRING([--with-boost-include=PATH],
     [specify exact directory for Boost headers]),
     [boost_include_path="$withval"])
+# If not specified, try some common paths.
+if test -z "$with_boost_include"; then
+	boostdirs="/usr/local /usr/pkg /opt /opt/local"
+	for d in $boostdirs
+	do
+		if test -f $d/include/boost/shared_ptr.hpp; then
+			boost_include_path=$d/include
+			break
+		fi
+	done
+fi
+CPPFLAGS_SAVES="$CPPFLAGS"
 if test "${boost_include_path}" ; then
 	BOOST_INCLUDES="-I${boost_include_path}"
 	CPPFLAGS="$CPPFLAGS $BOOST_INCLUDES"
 fi
+AC_CHECK_HEADERS([boost/shared_ptr.hpp boost/foreach.hpp],,
+  AC_MSG_ERROR([Missing required header files.]))
+CPPFLAGS="$CPPFLAGS_SAVES"
 AC_SUBST(BOOST_INCLUDES)
 
 #
@@ -352,6 +395,9 @@ AC_SUBST(PTHREAD_LDFLAGS)
 #
 CPPFLAGS="$CPPFLAGS -I\$(top_srcdir)/ext/asio"
 #
+# Disable threads: Currently we don't use them.
+CPPFLAGS="$CPPFLAGS -DASIO_DISABLE_THREADS=1"
+#
 # kqueue portability: ASIO uses kqueue by default if it's available (it's
 # generally available in BSD variants).  Unfortunately, some public
 # implementation of kqueue forces a conversion from a pointer to an integer,
@@ -389,12 +435,8 @@ if test "X$ac_cv_have_devpoll" = "Xyes" -a "X$GXX" = "Xyes"; then
 	CPPFLAGS="$CPPFLAGS -DASIO_DISABLE_DEV_POLL=1"
 fi
 
-# Check for headers from required devel kits.
-AC_CHECK_HEADERS([boost/shared_ptr.hpp boost/foreach.hpp],,
-  AC_MSG_ERROR([Missing required header files.]))
-
 AC_ARG_ENABLE(man, [AC_HELP_STRING([--enable-man],
-  [regenerate man pages [default=no]])] ,enable_man=yes, enable_man=no)
+  [regenerate man pages [default=no]])], enable_man=yes, enable_man=no)
 
 AM_CONDITIONAL(ENABLE_MAN, test x$enable_man != xno)
 
@@ -423,6 +465,7 @@ AC_CONFIG_FILES([Makefile
                  src/bin/msgq/tests/Makefile
                  src/bin/auth/Makefile
                  src/bin/auth/tests/Makefile
+                 src/bin/auth/benchmarks/Makefile
                  src/bin/recurse/Makefile
                  src/bin/recurse/tests/Makefile
                  src/bin/xfrin/Makefile
@@ -431,7 +474,15 @@ AC_CONFIG_FILES([Makefile
                  src/bin/xfrout/tests/Makefile
                  src/bin/zonemgr/Makefile
                  src/bin/zonemgr/tests/Makefile
+                 src/bin/stats/Makefile
+                 src/bin/stats/tests/Makefile
+                 src/bin/stats/tests/isc/Makefile
+                 src/bin/stats/tests/isc/cc/Makefile
+                 src/bin/stats/tests/isc/config/Makefile
+                 src/bin/stats/tests/isc/util/Makefile
+                 src/bin/stats/tests/testdata/Makefile
                  src/bin/usermgr/Makefile
+                 src/bin/tests/Makefile
                  src/lib/Makefile
                  src/lib/asiolink/Makefile
                  src/lib/asiolink/tests/Makefile
@@ -442,6 +493,8 @@ AC_CONFIG_FILES([Makefile
                  src/lib/cc/tests/Makefile
                  src/lib/python/Makefile
                  src/lib/python/isc/Makefile
+                 src/lib/python/isc/util/Makefile
+                 src/lib/python/isc/util/tests/Makefile
                  src/lib/python/isc/datasrc/Makefile
                  src/lib/python/isc/cc/Makefile
                  src/lib/python/isc/cc/tests/Makefile
@@ -449,13 +502,16 @@ AC_CONFIG_FILES([Makefile
                  src/lib/python/isc/config/tests/Makefile
                  src/lib/python/isc/log/Makefile
                  src/lib/python/isc/log/tests/Makefile
+                 src/lib/python/isc/net/Makefile
+                 src/lib/python/isc/net/tests/Makefile
                  src/lib/python/isc/notify/Makefile
                  src/lib/python/isc/notify/tests/Makefile
                  src/lib/config/Makefile
                  src/lib/config/tests/Makefile
-                 src/lib/config/testdata/Makefile
+                 src/lib/config/tests/testdata/Makefile
                  src/lib/dns/Makefile
                  src/lib/dns/tests/Makefile
+                 src/lib/dns/tests/testdata/Makefile
                  src/lib/dns/python/Makefile
                  src/lib/dns/python/tests/Makefile
                  src/lib/exceptions/Makefile
@@ -463,6 +519,9 @@ AC_CONFIG_FILES([Makefile
                  src/lib/datasrc/Makefile
                  src/lib/datasrc/tests/Makefile
                  src/lib/xfr/Makefile
+                 src/lib/log/Makefile
+                 src/lib/testutils/Makefile
+                 src/lib/testutils/testdata/Makefile
                ])
 AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            src/bin/cfgmgr/tests/b10-cfgmgr_test.py
@@ -472,7 +531,6 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            src/bin/cmdctl/cmdctl.spec.pre
            src/bin/xfrin/tests/xfrin_test
            src/bin/xfrin/xfrin.py
-           src/bin/xfrin/xfrin.spec.pre
            src/bin/xfrin/run_b10-xfrin.sh
            src/bin/xfrout/xfrout.py
            src/bin/xfrout/xfrout.spec.pre
@@ -482,6 +540,12 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            src/bin/zonemgr/zonemgr.spec.pre
            src/bin/zonemgr/tests/zonemgr_test
            src/bin/zonemgr/run_b10-zonemgr.sh
+           src/bin/stats/stats.py
+           src/bin/stats/stats_stub.py
+           src/bin/stats/stats.spec.pre
+           src/bin/stats/run_b10-stats.sh
+           src/bin/stats/run_b10-stats_stub.sh
+           src/bin/stats/tests/stats_test
            src/bin/bind10/bind10.py
            src/bin/bind10/tests/bind10_test
            src/bin/bind10/run_bind10.sh
@@ -501,6 +565,7 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            src/bin/auth/spec_config.h.pre
            src/bin/recurse/recurse.spec.pre
            src/bin/recurse/spec_config.h.pre
+           src/bin/tests/process_rename_test.py
            src/lib/config/tests/data_def_unittests_config.h
            src/lib/python/isc/config/tests/config_test
            src/lib/python/isc/cc/tests/cc_test
@@ -516,6 +581,9 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            chmod +x src/bin/xfrin/run_b10-xfrin.sh
            chmod +x src/bin/xfrout/run_b10-xfrout.sh
            chmod +x src/bin/zonemgr/run_b10-zonemgr.sh
+           chmod +x src/bin/stats/tests/stats_test
+           chmod +x src/bin/stats/run_b10-stats.sh
+           chmod +x src/bin/stats/run_b10-stats_stub.sh
            chmod +x src/bin/bind10/run_bind10.sh
            chmod +x src/bin/cmdctl/tests/cmdctl_test
            chmod +x src/bin/xfrin/tests/xfrin_test
@@ -546,16 +614,18 @@ Package:
   Name:          $PACKAGE_NAME
   Version:       $PACKAGE_VERSION
 
+C++ Compiler:    $CXX
+
 Flags:
   DEFS:          $DEFS
   CPPFLAGS:      $CPPFLAGS
-  CFLAGS:        $CFLAGS
   CXXFLAGS:      $CXXFLAGS
   B10_CXXFLAGS:  $B10_CXXFLAGS
 dnl includes too
   Python:        ${PYTHON_INCLUDES}
                  ${PYTHON_LDFLAGS}
                  ${PYTHON_LIB}
+  Boost:         ${BOOST_INCLUDES}
   SQLite:        $SQLITE_CFLAGS
                  $SQLITE_LIBS
 
@@ -575,4 +645,3 @@ cat <<EOF
   Now you can type "make" to build BIND 10
 
 EOF
-

doc/Doxyfile

@@ -568,7 +568,7 @@ WARN_LOGFILE           =
 # directories like "/usr/src/myproject". Separate the files or directories
 # with spaces.
 
-INPUT                  = ../src/lib/cc ../src/lib/config ../src/lib/dns ../src/lib/exceptions ../src/lib/datasrc ../src/bin/auth ../src/lib/bench
+INPUT                  = ../src/lib/cc ../src/lib/config ../src/lib/dns ../src/lib/exceptions ../src/lib/datasrc ../src/bin/auth ../src/lib/bench ../src/lib/log ../src/lib/asiolink/
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is

doc/guide/bind10-guide.xml

@@ -42,7 +42,7 @@
 
     <note>
       <para>
-        BIND 10, at this time, does not provide an recursive
+        BIND 10, at this time, does not provide a recursive
         DNS server. It does provide a EDNS0- and DNSSEC-capable
         authoritative DNS server.
       </para>
@@ -74,9 +74,9 @@
 	For this development prototype release, the only supported
 	data source backend is SQLite3. The authoritative server
 	requires SQLite 3.3.9 or newer.
-        The <command>b10-xfrin</command> and <command>b10-xfrout</command>
+	The <command>b10-xfrin</command>, <command>b10-xfrout</command>,
-        modules require the libpython3 library and the Python
+	and <command>b10-zonemgr</command> modules require the
-        _sqlite3.so module.
+	libpython3 library and the Python _sqlite3.so module.
       </para></note>
 <!-- TODO: this will change ... -->
 
@@ -165,6 +165,15 @@
             </simpara>
           </listitem>
 
+          <listitem>
+            <simpara>
+              <command>b10-zonemgr</command> &mdash;
+              Secondary manager.
+	      This process keeps track of timers and other
+              necessary information for BIND 10 to act as a slave server.
+            </simpara>
+          </listitem>
+
         </itemizedlist>
       </para>
 
@@ -650,8 +659,9 @@ var/
       The <command>bind10</command> master process will also start up
       <command>b10-cmdctl</command> for admins to communicate with the
       system, <command>b10-auth</command> for Authoritative DNS service,
-      <command>b10-xfrin</command> for inbound DNS zone transfers.
+      <command>b10-xfrin</command> for inbound DNS zone transfers,
-      and <command>b10-xfrout</command> for outbound DNS zone transfers.
+      <command>b10-xfrout</command> for outbound DNS zone transfers,
+      and <command>b10-zonemgr</command> for secondary service.
     </para>
 
     <section id="start">
@@ -1173,14 +1183,14 @@ TODO
       transfer. When received, it is stored in the BIND 10
       data store, and its records can be served by
       <command>b10-auth</command>.
-      This allows the BIND 10 server to provide
+      In combination with <command>b10-zonemgr</command> (for
-      <quote>secondary</quote> service.
+      automated SOA checks), this allows the BIND 10 server to
+      provide <quote>secondary</quote> service.
     </para>
 
     <note><simpara>
      The current development release of BIND 10 only supports
      AXFR. (IXFR is not supported.) 
-     It also does not yet support automated SOA checks.
     </simpara></note>
 
     <para>
@@ -1204,12 +1214,13 @@ TODO
       sends the zone.
       This is used to provide master DNS service to share zones
       to secondary name servers.
+      The <command>b10-xfrout</command> is also used to send
+      NOTIFY messages to slaves.
     </para>
 
     <note><simpara>
      The current development release of BIND 10 only supports
      AXFR. (IXFR is not supported.) 
-     It also does not yet support NOTIFY.
      Access control is not yet provided.
     </simpara></note>
 
@@ -1226,6 +1237,31 @@ what is XfroutClient xfr_client??
 
   </chapter>
 
+  <chapter id="zonemgr">
+    <title>Secondary Manager</title>
+
+    <para>
+      The <command>b10-zonemgr</command> process is started by
+      <command>bind10</command>.
+      It keeps track of SOA refresh, retry, and expire timers
+      and other details for BIND 10 to perform as a slave.
+      When the <command>b10-auth</command> authoritative DNS server
+      receives a NOTIFY message, <command>b10-zonemgr</command>
+      may tell <command>b10-xfrin</command> to do a refresh
+      to start an inbound zone transfer.
+      The secondary manager resets its counters when a new zone is
+      transferred in.
+    </para>
+
+    <note><simpara>
+     Access control (such as allowing notifies) is not yet provided.
+     The primary/secondary service is not yet complete.
+    </simpara></note>
+
+<!-- TODO: lots to describe for zonemgr -->
+
+  </chapter>
+
 <!-- TODO: how to help: run unit tests, join lists, review trac tickets -->
 
   <!-- <index>    <title>Index</title> </index> -->

src/bin/Makefile.am

@@ -1 +1,4 @@
-SUBDIRS = bind10 bindctl cfgmgr loadzone msgq host cmdctl auth recurse xfrin xfrout usermgr zonemgr
+SUBDIRS = bind10 bindctl cfgmgr loadzone msgq host cmdctl auth recurse xfrin \
+	xfrout usermgr zonemgr stats tests
+
+check-recursive: all-recursive

src/bin/auth/Makefile.am

@@ -1,4 +1,4 @@
-SUBDIRS = . tests
+SUBDIRS = . tests benchmarks
 
 AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
 AM_CPPFLAGS += -I$(top_srcdir)/src/bin -I$(top_builddir)/src/bin
@@ -6,6 +6,7 @@ AM_CPPFLAGS += -I$(top_srcdir)/src/lib/dns -I$(top_builddir)/src/lib/dns
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/cc -I$(top_builddir)/src/lib/cc
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/asiolink
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/asiolink
+AM_CPPFLAGS += $(BOOST_INCLUDES)
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 

src/bin/auth/auth_srv.cc

@@ -32,10 +32,13 @@
 #include <exceptions/exceptions.h>
 
 #include <dns/buffer.h>
+#include <dns/edns.h>
 #include <dns/exceptions.h>
 #include <dns/messagerenderer.h>
 #include <dns/name.h>
 #include <dns/question.h>
+#include <dns/opcode.h>
+#include <dns/rcode.h>
 #include <dns/rrset.h>
 #include <dns/rrttl.h>
 #include <dns/message.h>
@@ -76,7 +79,7 @@ public:
                             OutputBufferPtr buffer);
     bool processAxfrQuery(const IOMessage& io_message, MessagePtr message,
                           OutputBufferPtr buffer);
-    bool processNotify(const IOMessage& io_message, MessagePtr message, 
+    bool processNotify(const IOMessage& io_message, MessagePtr message,
                        OutputBufferPtr buffer);
 
     /// Currently non-configurable, but will be.
@@ -87,6 +90,8 @@ public:
     bool verbose_mode_;
     AbstractSession* xfrin_session_;
 
+    /// Hot spot cache
+    isc::datasrc::HotCache cache_;
 private:
     std::string db_file_;
 
@@ -98,9 +103,6 @@ private:
 
     bool xfrout_connected_;
     AbstractXfroutClient& xfrout_client_;
-
-    /// Hot spot cache
-    isc::datasrc::HotCache cache_;
 };
 
 AuthSrvImpl::AuthSrvImpl(const bool use_cache,
@@ -155,13 +157,15 @@ public:
     {
         MessageRenderer renderer(*buffer);
         if (io_message.getSocket().getProtocol() == IPPROTO_UDP) {
-            renderer.setLengthLimit(message->getUDPSize());
+            ConstEDNSPtr edns(message->getEDNS());
+            renderer.setLengthLimit(edns ? edns->getUDPSize() :
+                Message::DEFAULT_MAX_UDPSIZE);
         } else {
             renderer.setLengthLimit(65535);
         }
         message->toWire(renderer);
         if (server_->getVerbose()) {
-            cerr << "[b10-recurse] sending a response (" << renderer.getLength()
+            cerr << "[b10-auth] sending a response (" << renderer.getLength()
                  << " bytes):\n" << message->toText() << endl;
         }
     }
@@ -176,7 +180,7 @@ private:
 class ConfigChecker : public SimpleCallback {
 public:
     ConfigChecker(AuthSrv* srv) : server_(srv) {}
-    virtual void operator()(const IOMessage& io_message UNUSED_PARAM) const {
+    virtual void operator()(const IOMessage&) const {
         if (server_->getConfigSession()->hasQueuedMsgs()) {
             server_->getConfigSession()->checkCommand();
         }
@@ -217,8 +221,8 @@ makeErrorMessage(MessagePtr message, OutputBufferPtr buffer,
     // XXX: with the current implementation, it's not easy to set EDNS0
     // depending on whether the query had it.  So we'll simply omit it.
     const qid_t qid = message->getQid();
-    const bool rd = message->getHeaderFlag(MessageFlag::RD());
+    const bool rd = message->getHeaderFlag(Message::HEADERFLAG_RD);
-    const bool cd = message->getHeaderFlag(MessageFlag::CD());
+    const bool cd = message->getHeaderFlag(Message::HEADERFLAG_CD);
     const Opcode& opcode = message->getOpcode();
     vector<QuestionPtr> questions;
 
@@ -231,13 +235,12 @@ makeErrorMessage(MessagePtr message, OutputBufferPtr buffer,
     message->clear(Message::RENDER);
     message->setQid(qid);
     message->setOpcode(opcode);
-    message->setHeaderFlag(MessageFlag::QR());
+    message->setHeaderFlag(Message::HEADERFLAG_QR);
-    message->setUDPSize(AuthSrvImpl::DEFAULT_LOCAL_UDPSIZE);
     if (rd) {
-        message->setHeaderFlag(MessageFlag::RD());
+        message->setHeaderFlag(Message::HEADERFLAG_RD);
     }
     if (cd) {
-        message->setHeaderFlag(MessageFlag::CD());
+        message->setHeaderFlag(Message::HEADERFLAG_CD);
     }
     for_each(questions.begin(), questions.end(), QuestionInserter(message));
     message->setRcode(rcode);
@@ -263,6 +266,16 @@ AuthSrv::getVerbose() const {
 }
 
 void
+AuthSrv::setCacheSlots(const size_t slots) {
+    impl_->cache_.setSlots(slots);
+}
+
+size_t
+AuthSrv::getCacheSlots() const {
+    return (impl_->cache_.getSlots());
+}
+
+void
 AuthSrv::setXfrinSession(AbstractSession* xfrin_session) {
     impl_->xfrin_session_ = xfrin_session;
 }
@@ -289,7 +302,7 @@ AuthSrv::processMessage(const IOMessage& io_message, MessagePtr message,
         message->parseHeader(request_buffer);
 
         // Ignore all responses.
-        if (message->getHeaderFlag(MessageFlag::QR())) {
+        if (message->getHeaderFlag(Message::HEADERFLAG_QR)) {
             if (impl_->verbose_mode_) {
                 cerr << "[b10-auth] received unexpected response, ignoring"
                      << endl;
@@ -342,7 +355,7 @@ AuthSrv::processMessage(const IOMessage& io_message, MessagePtr message,
         }
         makeErrorMessage(message, buffer, Rcode::NOTIMP(),
                          impl_->verbose_mode_);
-    } else if (message->getRRCount(Section::QUESTION()) != 1) {
+    } else if (message->getRRCount(Message::SECTION_QUESTION) != 1) {
         makeErrorMessage(message, buffer, Rcode::FORMERR(),
                          impl_->verbose_mode_);
     } else {
@@ -365,14 +378,21 @@ bool
 AuthSrvImpl::processNormalQuery(const IOMessage& io_message, MessagePtr message,
                                 OutputBufferPtr buffer)
 {
-    const bool dnssec_ok = message->isDNSSECSupported();
+    ConstEDNSPtr remote_edns = message->getEDNS();
-    const uint16_t remote_bufsize = message->getUDPSize();
+    const bool dnssec_ok = remote_edns && remote_edns->getDNSSECAwareness();
+    const uint16_t remote_bufsize = remote_edns ? remote_edns->getUDPSize() :
+        Message::DEFAULT_MAX_UDPSIZE;
 
     message->makeResponse();
-    message->setHeaderFlag(MessageFlag::AA());
+    message->setHeaderFlag(Message::HEADERFLAG_AA);
     message->setRcode(Rcode::NOERROR());
-    message->setDNSSECSupported(dnssec_ok);
+
-    message->setUDPSize(AuthSrvImpl::DEFAULT_LOCAL_UDPSIZE);
+    if (remote_edns) {
+        EDNSPtr local_edns = EDNSPtr(new EDNS());
+        local_edns->setDNSSECAwareness(dnssec_ok);
+        local_edns->setUDPSize(AuthSrvImpl::DEFAULT_LOCAL_UDPSIZE);
+        message->setEDNS(local_edns);
+    }
 
     try {
         Query query(*message, cache_, dnssec_ok);
@@ -415,8 +435,10 @@ AuthSrvImpl::processAxfrQuery(const IOMessage& io_message, MessagePtr message,
     }
 
     try {
-        xfrout_client_.connect();
+        if (!xfrout_connected_) {
-        xfrout_connected_ = true;
+            xfrout_client_.connect();
+            xfrout_connected_ = true;
+        }
         xfrout_client_.sendXfroutRequestInfo(
             io_message.getSocket().getNative(),
             io_message.getData(),
@@ -430,7 +452,7 @@ AuthSrvImpl::processAxfrQuery(const IOMessage& io_message, MessagePtr message,
             xfrout_client_.disconnect();
             xfrout_connected_ = false;
         }
-        
+
         if (verbose_mode_) {
             cerr << "[b10-auth] Error in handling XFR request: " << err.what()
                  << endl;
@@ -439,9 +461,6 @@ AuthSrvImpl::processAxfrQuery(const IOMessage& io_message, MessagePtr message,
         return (true);
     }
 
-    xfrout_client_.disconnect();
-    xfrout_connected_ = false;
-
     return (false);
 }
 
@@ -451,10 +470,10 @@ AuthSrvImpl::processNotify(const IOMessage& io_message, MessagePtr message,
 {
     // The incoming notify must contain exactly one question for SOA of the
     // zone name.
-    if (message->getRRCount(Section::QUESTION()) != 1) {
+    if (message->getRRCount(Message::SECTION_QUESTION) != 1) {
         if (verbose_mode_) {
                 cerr << "[b10-auth] invalid number of questions in notify: "
-                     << message->getRRCount(Section::QUESTION()) << endl;
+                     << message->getRRCount(Message::SECTION_QUESTION) << endl;
         }
         makeErrorMessage(message, buffer, Rcode::FORMERR(), verbose_mode_);
         return (true);
@@ -487,7 +506,7 @@ AuthSrvImpl::processNotify(const IOMessage& io_message, MessagePtr message,
         }
         return (false);
     }
-    
+
     const string remote_ip_address =
         io_message.getRemoteEndpoint().getAddress().toText();
     static const string command_template_start =
@@ -498,7 +517,7 @@ AuthSrvImpl::processNotify(const IOMessage& io_message, MessagePtr message,
 
     try {
         ConstElementPtr notify_command = Element::fromJSON(
-                command_template_start + question->getName().toText() + 
+                command_template_start + question->getName().toText() +
                 command_template_master + remote_ip_address +
                 command_template_rrclass + question->getClass().toText() +
                 command_template_end);
@@ -512,7 +531,7 @@ AuthSrvImpl::processNotify(const IOMessage& io_message, MessagePtr message,
         if (rcode != 0) {
             if (verbose_mode_) {
                 cerr << "[b10-auth] failed to notify Zonemgr: "
-                     << parsed_answer->str() << endl; 
+                     << parsed_answer->str() << endl;
             }
             return (false);
         }
@@ -524,7 +543,7 @@ AuthSrvImpl::processNotify(const IOMessage& io_message, MessagePtr message,
     }
 
     message->makeResponse();
-    message->setHeaderFlag(MessageFlag::AA());
+    message->setHeaderFlag(Message::HEADERFLAG_AA);
     message->setRcode(Rcode::NOERROR());
 
     MessageRenderer renderer(*buffer);

src/bin/auth/auth_srv.h

@@ -30,8 +30,30 @@ class AbstractXfroutClient;
 }
 }
 
+/// \brief The implementation class for the \c AuthSrv class using the pimpl
+/// idiom.
 class AuthSrvImpl;
 
+/// \brief The authoritative nameserver class.
+///
+/// \c AuthSrv is a concrete class that implements authoritative DNS server
+/// protocol processing.
+/// An \c AuthSrv object is primarily responsible for handling incoming DNS
+/// requests: It parses the request and dispatches subsequent processing to
+/// the corresponding module (which may be an internal library or a separate
+/// process) depending on the request type.  For normal queries, the
+/// \c AuthSrv object searches configured data sources for the answer to the
+/// query, and builds a response containing the answer.
+///
+/// This class uses the "pimpl" idiom, and hides detailed implementation
+/// through the \c impl_ pointer (which points to an instance of the
+/// \c AuthSrvImpl class).  An \c AuthSrv object is supposed to exist for quite
+/// a long period, and only a few \c AuthSrv objects will be created (in fact,
+/// in this current implementation there will only be one object), so the
+/// construction overhead of this approach should be acceptable.
+///
+/// The design of this class is still in flux.  It's quite likely to change
+/// in future versions.
 class AuthSrv {
     ///
     /// \name Constructors, Assignment Operator and Destructor.
@@ -74,17 +96,80 @@ public:
     /// \brief Set verbose flag
     ///
     /// \param on The new value of the verbose flag
-    void setVerbose(bool on);
+
+    /// \brief Enable or disable verbose logging.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \param on \c true to enable verbose logging; \c false to disable
+    /// verbose logging.
+    void setVerbose(const bool on);
+
+    /// \brief Returns the logging verbosity of the \c AuthSrv object.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return \c true if verbose logging is enabled; otherwise \c false.
 
     /// \brief Get the current value of the verbose flag
     bool getVerbose() const;
 
-    /// \brief Set and get the config session
+    /// \brief Updates the data source for the \c AuthSrv object.
-    void setConfigSession(isc::config::ModuleCCSession* config_session);
+    ///
+    /// This method installs or replaces the data source that the \c AuthSrv
+    /// object refers to for query processing.
+    /// Although the method name is generic, the only thing it does is to
+    /// update the data source information.
+    /// If there is a data source installed, it will be replaced with the
+    /// new one.
+    ///
+    /// In the current implementation, the SQLite data source is assumed.
+    /// The \c config parameter will simply be passed to the initialization
+    /// routine of the \c Sqlite3DataSrc class.
+    ///
+    /// On success this method returns a data \c Element (in the form of a
+    /// pointer like object) indicating the successful result,
+    /// i.e., {"result": [0]}.
+    /// Otherwise, it returns a data \c Element explaining the error:
+    /// {"result": [1, <error-description>]}.
+    ///
+    /// This method is mostly exception free (error conditions are represented
+    /// via the return value).  But it may still throw a standard exception
+    /// if memory allocation fails inside the method.
+    /// When a standard exception is thrown or an implementation specific
+    /// exception is triggered and caught internally, this function provides
+    /// the strong exception guarantee: Unless everything succeeds, currently
+    /// installed data source (if any) won't be replaced.
+    ///
+    /// \param config An immutable pointer-like object to a data \c Element,
+    /// possibly containing the data source information to be used.
+    /// \return An immutable pointer-like object to a data \c Element
+    /// containing the result of the update operation.
+    isc::data::ConstElementPtr updateConfig(isc::data::ConstElementPtr config);
+
+    /// \param Returns the command and configuration session for the
+    /// \c AuthSrv.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return A pointer to \c ModuleCCSession object stored in the
+    /// \c AuthSrv object.  In this implementation it could be NULL.
     isc::config::ModuleCCSession* getConfigSession() const;
 
-    /// \brief Handle commands from the config session
+    /// \brief Set the command and configuration session for the \c AuthSrv.
-    isc::data::ConstElementPtr updateConfig(isc::data::ConstElementPtr config);
+    ///
+    /// Note: this interface is tentative.  We'll revisit the ASIO and session
+    /// frameworks, at which point the session will probably be passed on
+    /// construction of the server.
+    /// In the current implementation, this method is expected to be called
+    /// exactly once as part of initialization.  If this method is called
+    /// multiple times, previously specified session is silently overridden.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \param config_session A pointer to \c ModuleCCSession object to receive
+    /// control commands and configuration updates.
+    void setConfigSession(isc::config::ModuleCCSession* config_session);
 
     /// \brief Assign an ASIO IO Service queue to this Recursor object
     void setIOService(asiolink::IOService& ios) { io_service_ = &ios; }
@@ -101,6 +186,28 @@ public:
     /// \brief Return pointer to the Checkin callback function
     asiolink::SimpleCallback* getCheckinProvider() const { return (checkin_); }
 
+    /// \brief Set or update the size (number of slots) of hot spot cache.
+    ///
+    /// If the specified size is 0, it means the size will be unlimited.
+    /// The specified size is recorded even if the cache is disabled; the
+    /// new size will be effective when the cache is enabled.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \param slots The number of cache slots.
+    void setCacheSlots(const size_t slots);
+
+    /// \brief Get the current size (number of slots) of hot spot cache.
+    ///
+    /// It always returns the recorded size regardless of the cache is enabled.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return The current number of cache slots.
+    size_t getCacheSlots() const;
+
+    /// \brief Set the communication session with a separate process for
+    /// outgoing zone transfers.
     ///
     /// Note: this interface is tentative.  We'll revisit the ASIO and session
     /// frameworks, at which point the session will probably be passed on
@@ -113,6 +220,7 @@ public:
     /// Ownership isn't transferred: the caller is responsible for keeping
     /// this object to be valid while the server object is working and for
     /// disconnecting the session and destroying the object when the server
+    /// is shutdown.
     ///
     void setXfrinSession(isc::cc::AbstractSession* xfrin_session);
 private:

src/bin/auth/b10-auth.8

@@ -137,6 +137,7 @@ configuration is not defined\&.
 \fBb10-cmdctl\fR(8),
 \fBb10-loadzone\fR(8),
 \fBb10-msgq\fR(8),
+\fBb10-zonemgr\fR(8),
 \fBbind10\fR(8),
 BIND 10 Guide\&.
 .SH "HISTORY"

src/bin/auth/b10-auth.xml

@@ -201,6 +201,9 @@
         <refentrytitle>b10-msgq</refentrytitle><manvolnum>8</manvolnum>
       </citerefentry>,
       <citerefentry>
+        <refentrytitle>b10-zonemgr</refentrytitle><manvolnum>8</manvolnum>
+      </citerefentry>,
+      <citerefentry>
         <refentrytitle>bind10</refentrytitle><manvolnum>8</manvolnum>
       </citerefentry>,
       <citetitle>BIND 10 Guide</citetitle>.

src/bin/auth/benchmarks/Makefile.am

@@ -0,0 +1,21 @@
+AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
+AM_CPPFLAGS += -I$(top_srcdir)/src/bin -I$(top_builddir)/src/bin
+AM_CPPFLAGS += $(BOOST_INCLUDES)
+
+AM_CXXFLAGS = $(B10_CXXFLAGS)
+
+CLEANFILES = *.gcno *.gcda
+
+noinst_PROGRAMS = query_bench
+query_bench_SOURCES = query_bench.cc
+query_bench_SOURCES += ../auth_srv.h ../auth_srv.cc
+
+query_bench_LDADD = $(top_builddir)/src/lib/dns/libdns++.la
+query_bench_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la
+query_bench_LDADD += $(top_builddir)/src/lib/bench/libbench.la
+query_bench_LDADD += $(top_builddir)/src/lib/datasrc/libdatasrc.la
+query_bench_LDADD += $(top_builddir)/src/lib/config/libcfgclient.la
+query_bench_LDADD += $(top_builddir)/src/lib/cc/libcc.la
+query_bench_LDADD += $(top_builddir)/src/lib/xfr/libxfr.la
+query_bench_LDADD += $(top_builddir)/src/lib/asiolink/libasiolink.la
+query_bench_LDADD += $(SQLITE_LIBS)

src/bin/auth/benchmarks/query_bench.cc

@@ -0,0 +1,190 @@
+// Copyright (C) 2010  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+// $Id$
+
+#include <stdlib.h>
+
+#include <iostream>
+#include <vector>
+
+#include <boost/shared_ptr.hpp>
+
+#include <bench/benchmark.h>
+#include <bench/benchmark_util.h>
+
+#include <dns/buffer.h>
+#include <dns/message.h>
+#include <dns/name.h>
+#include <dns/question.h>
+#include <dns/rrclass.h>
+
+#include <xfr/xfrout_client.h>
+
+#include <auth/auth_srv.h>
+#include <asiolink/asiolink.h>
+
+using namespace std;
+using namespace isc;
+using namespace isc::data;
+using namespace isc::dns;
+using namespace isc::xfr;
+using namespace isc::bench;
+using namespace asiolink;
+
+namespace {
+// Commonly used constant:
+XfroutClient xfrout_client("dummy_path"); // path doesn't matter
+
+// Just something to pass as the server to resume
+class DummyServer : public DNSServer {
+    public:
+        virtual void operator()(asio::error_code, size_t) { }
+        virtual void resume(const bool) { }
+        virtual DNSServer* clone() {
+            return new DummyServer(*this);
+        }
+};
+
+class QueryBenchMark {
+private:
+    // Maintain dynamically generated objects via shared pointers because
+    // QueryBenchMark objects will be copied.
+    typedef boost::shared_ptr<AuthSrv> AuthSrvPtr;
+    typedef boost::shared_ptr<const IOEndpoint> IOEndpointPtr;
+public:
+    QueryBenchMark(const int cache_slots, const char* const datasrc_file,
+                   const BenchQueries& queries, MessagePtr query_message,
+                   OutputBufferPtr buffer) :
+        server_(new AuthSrv(cache_slots >= 0 ? true : false, xfrout_client)),
+        queries_(queries),
+        query_message_(query_message),
+        buffer_(buffer),
+        dummy_socket(IOSocket::getDummyUDPSocket()),
+        dummy_endpoint(IOEndpointPtr(IOEndpoint::create(IPPROTO_UDP,
+                                                        IOAddress("192.0.2.1"),
+                                                        5300)))
+    {
+        if (cache_slots >= 0) {
+            server_->setCacheSlots(cache_slots);
+        }
+        server_->updateConfig(Element::fromJSON("{\"database_file\": \"" +
+                                                string(datasrc_file) + "\"}"));
+    }
+    unsigned int run() {
+        BenchQueries::const_iterator query;
+        const BenchQueries::const_iterator query_end = queries_.end();
+        DummyServer server;
+        for (query = queries_.begin(); query != query_end; ++query) {
+            IOMessage io_message(&(*query)[0], (*query).size(), dummy_socket,
+                                 *dummy_endpoint);
+            query_message_->clear(Message::PARSE);
+            server_->processMessage(io_message, query_message_, buffer_,
+                &server);
+        }
+
+        return (queries_.size());
+    }
+private:
+    AuthSrvPtr server_;
+    const BenchQueries& queries_;
+    MessagePtr query_message_;
+    OutputBufferPtr buffer_;
+    IOSocket& dummy_socket;
+    IOEndpointPtr dummy_endpoint;
+};
+
+}
+
+namespace isc {
+namespace bench {
+template<>
+void
+BenchMark<QueryBenchMark>::printResult() const {
+    cout.precision(6);
+    cout << "Processed " << getIteration() << " queries in "
+         << fixed << getDuration() << "s";
+    cout.precision(2);
+    cout << " (" << fixed << getIterationPerSecond() << "qps)" << endl;
+}
+}
+}
+
+namespace {
+void
+usage() {
+    cerr << "Usage: query_bench [-n iterations] datasrc_file query_datafile"
+         << endl;
+    exit (1);
+}
+}
+
+int
+main(int argc, char* argv[]) {
+    int ch;
+    int iteration = 1;
+    while ((ch = getopt(argc, argv, "n:")) != -1) {
+        switch (ch) {
+        case 'n':
+            iteration = atoi(optarg);
+            break;
+        case '?':
+        default:
+            usage();
+        }
+    }
+    argc -= optind;
+    argv += optind;
+    if (argc < 2) {
+        usage();
+    }
+    const char* const datasrc_file = argv[0];
+    const char* const query_data_file = argv[1];
+
+    BenchQueries queries;
+    loadQueryData(query_data_file, queries, RRClass::IN());
+    OutputBufferPtr buffer(new OutputBuffer(4096));
+    MessagePtr message(new Message(Message::PARSE));
+
+    cout << "Parameters:" << endl;
+    cout << "  Iterations: " << iteration << endl;
+    cout << "  Data Source: " << datasrc_file << endl;
+    cout << "  Query data: file=" << query_data_file << " (" << queries.size()
+         << " queries)" << endl << endl;
+
+    cout << "Benchmark enabling Hot Spot Cache with unlimited slots "
+         << endl;
+    BenchMark<QueryBenchMark>(iteration,
+                              QueryBenchMark(0, datasrc_file, queries, message,
+                                             buffer));
+
+    cout << "Benchmark enabling Hot Spot Cache with 10*#queries slots "
+         << endl;
+    BenchMark<QueryBenchMark>(iteration,
+                              QueryBenchMark(10 * queries.size(), datasrc_file,
+                                             queries, message, buffer));
+
+    cout << "Benchmark enabling Hot Spot Cache with #queries/2 slots "
+         << endl;
+    BenchMark<QueryBenchMark>(iteration,
+                              QueryBenchMark(queries.size() / 2, datasrc_file,
+                                             queries, message, buffer));
+
+    cout << "Benchmark disabling Hot Spot Cache" << endl;
+    BenchMark<QueryBenchMark>(iteration,
+                              QueryBenchMark(-1, datasrc_file, queries,
+                                             message, buffer));    
+
+    return (0);
+}

src/bin/auth/main.cc

@@ -240,6 +240,8 @@ main(int argc, char* argv[]) {
 
         cout << "[b10-auth] Server started." << endl;
         io_service.run();
+
+        delete dns_service;
     } catch (const std::exception& ex) {
         cerr << "[b10-auth] Server failed: " << ex.what() << endl;
         ret = 1;

src/bin/auth/tests/Makefile.am

@@ -1,7 +1,9 @@
 AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/dns -I$(top_srcdir)/src/bin
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/cc
-AM_CPPFLAGS += -DTEST_DATA_DIR=\"$(srcdir)/testdata\"
+AM_CPPFLAGS += $(BOOST_INCLUDES)
+AM_CPPFLAGS += -DTEST_DATA_DIR=\"$(top_srcdir)/src/lib/testutils/testdata\"
+AM_CPPFLAGS += -DTEST_DATA_BUILDDIR=\"$(abs_top_builddir)/src/lib/testutils/testdata\"
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 
@@ -35,25 +37,3 @@ run_unittests_LDADD += $(top_builddir)/src/lib/xfr/libxfr.la
 endif
 
 noinst_PROGRAMS = $(TESTS)
-
-EXTRA_DIST =  testdata/badExampleQuery_fromWire
-EXTRA_DIST += testdata/badExampleQuery_fromWire.spec
-EXTRA_DIST += testdata/example.com
-EXTRA_DIST += testdata/examplequery_fromWire
-EXTRA_DIST += testdata/examplequery_fromWire.spec
-EXTRA_DIST += testdata/example.sqlite3
-EXTRA_DIST += testdata/iqueryresponse_fromWire
-EXTRA_DIST += testdata/iqueryresponse_fromWire.spec
-EXTRA_DIST += testdata/multiquestion_fromWire
-EXTRA_DIST += testdata/multiquestion_fromWire.spec
-EXTRA_DIST += testdata/queryBadEDNS_fromWire
-EXTRA_DIST += testdata/queryBadEDNS_fromWire.spec
-EXTRA_DIST += testdata/shortanswer_fromWire
-EXTRA_DIST += testdata/shortanswer_fromWire.spec
-EXTRA_DIST += testdata/shortmessage_fromWire
-EXTRA_DIST += testdata/shortquestion_fromWire
-EXTRA_DIST += testdata/shortresponse_fromWire
-EXTRA_DIST += testdata/simplequery_fromWire
-EXTRA_DIST += testdata/simplequery_fromWire.spec
-EXTRA_DIST += testdata/simpleresponse_fromWire
-EXTRA_DIST += testdata/simpleresponse_fromWire.spec

src/bin/auth/tests/auth_srv_unittest.cc

@@ -15,26 +15,8 @@
 // $Id$
 
 #include <config.h>
-
-#include <gtest/gtest.h>
-
-#include <dns/buffer.h>
-#include <dns/name.h>
-#include <dns/message.h>
-#include <dns/messagerenderer.h>
-#include <dns/rrclass.h>
-#include <dns/rrtype.h>
-
-#include <cc/data.h>
-#include <cc/session.h>
-
-#include <xfr/xfrout_client.h>
-
 #include <auth/auth_srv.h>
-#include <asiolink/asiolink.h>
+#include <testutils/srv_unittest.h>
-
-#include <dns/tests/unittest_util.h>
-#include <auth/tests/mockups.h>
 
 using namespace std;
 using namespace isc::cc;
@@ -51,262 +33,60 @@ const char* const CONFIG_TESTDB =
 // the sqlite3 test).
 const char* const BADCONFIG_TESTDB =
     "{ \"database_file\": \"" TEST_DATA_DIR "/nodir/notexist\"}";
-const char* const DEFAULT_REMOTE_ADDRESS = "192.0.2.1";
-
-class DummySocket : public IOSocket {
-private:
-    DummySocket(const DummySocket& source);
-    DummySocket& operator=(const DummySocket& source);
-public:
-    DummySocket(const int protocol) : protocol_(protocol) {}
-    virtual int getNative() const { return (-1); }
-    virtual int getProtocol() const { return (protocol_); }
-private:
-    const int protocol_;
-};
 
-class AuthSrvTest : public ::testing::Test {
+class AuthSrvTest : public SrvTestBase {
 protected:
-    AuthSrvTest() : server(true, xfrout),
+    AuthSrvTest() : server(true, xfrout) {
-                    request_message(Message::RENDER),
-                    parse_message(new Message(Message::PARSE)),
-                    default_qid(0x1035), opcode(Opcode(Opcode::QUERY())),
-                    qname("www.example.com"), qclass(RRClass::IN()),
-                    qtype(RRType::A()), io_message(NULL), endpoint(NULL),
-                    request_obuffer(0), request_renderer(request_obuffer),
-                    response_obuffer(new OutputBuffer(0))
-    {
         server.setXfrinSession(&notify_session);
     }
-    ~AuthSrvTest() {
-        delete io_message;
-        delete endpoint;
-    }
-    MockSession notify_session;
     MockXfroutClient xfrout;
-    MockServer dnsserv;
     AuthSrv server;
-    Message request_message;
-    MessagePtr parse_message;
-    const qid_t default_qid;
-    const Opcode opcode;
-    const Name qname;
-    const RRClass qclass;
-    const RRType qtype;
-    IOSocket* io_sock;
-    IOMessage* io_message;
-    const IOEndpoint* endpoint;
-    OutputBuffer request_obuffer;
-    MessageRenderer request_renderer;
-    OutputBufferPtr response_obuffer;
-    vector<uint8_t> data;
-
-    void createDataFromFile(const char* const datafile, int protocol);
-    void createRequestPacket(Message& message, int protocol);
 };
 
-
-// These are flags to indicate whether the corresponding flag bit of the
-// DNS header is to be set in the test cases.  (Note that the flag values
-// is irrelevant to their wire-format values)
-const unsigned int QR_FLAG = 0x1;
-const unsigned int AA_FLAG = 0x2;
-const unsigned int TC_FLAG = 0x4;
-const unsigned int RD_FLAG = 0x8;
-const unsigned int RA_FLAG = 0x10;
-const unsigned int AD_FLAG = 0x20;
-const unsigned int CD_FLAG = 0x40;
-
-void
-AuthSrvTest::createDataFromFile(const char* const datafile,
-                                const int protocol = IPPROTO_UDP)
-{
-    delete io_message;
-    data.clear();
-
-    delete endpoint;
-
-    endpoint = IOEndpoint::create(protocol,
-                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 5300);
-    UnitTestUtil::readWireData(datafile, data);
-    io_sock = new DummySocket(protocol);
-    io_message = new IOMessage(&data[0], data.size(), *io_sock, *endpoint);
-}
-
-void
-AuthSrvTest::createRequestPacket(Message& message,
-                                 const int protocol = IPPROTO_UDP)
-{
-    message.toWire(request_renderer);
-
-    delete io_message;
-    delete io_sock;
-
-    endpoint = IOEndpoint::create(protocol,
-                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 5300);
-    io_sock = new DummySocket(protocol);
-    io_message = new IOMessage(request_renderer.getData(),
-                               request_renderer.getLength(),
-                               *io_sock, *endpoint);
-}
-
-void
-headerCheck(const Message& message, const qid_t qid, const Rcode& rcode,
-            const uint16_t opcodeval, const unsigned int flags,
-            const unsigned int qdcount,
-            const unsigned int ancount, const unsigned int nscount,
-            const unsigned int arcount)
-{
-    EXPECT_EQ(qid, message.getQid());
-    EXPECT_EQ(rcode, message.getRcode());
-    EXPECT_EQ(opcodeval, message.getOpcode().getCode());
-    EXPECT_EQ((flags & QR_FLAG) != 0, message.getHeaderFlag(MessageFlag::QR()));
-    EXPECT_EQ((flags & AA_FLAG) != 0, message.getHeaderFlag(MessageFlag::AA()));
-    EXPECT_EQ((flags & TC_FLAG) != 0, message.getHeaderFlag(MessageFlag::TC()));
-    EXPECT_EQ((flags & RA_FLAG) != 0, message.getHeaderFlag(MessageFlag::RA()));
-    EXPECT_EQ((flags & RD_FLAG) != 0, message.getHeaderFlag(MessageFlag::RD()));
-    EXPECT_EQ((flags & AD_FLAG) != 0, message.getHeaderFlag(MessageFlag::AD()));
-    EXPECT_EQ((flags & CD_FLAG) != 0, message.getHeaderFlag(MessageFlag::CD()));
-
-    EXPECT_EQ(qdcount, message.getRRCount(Section::QUESTION()));
-    EXPECT_EQ(ancount, message.getRRCount(Section::ANSWER()));
-    EXPECT_EQ(nscount, message.getRRCount(Section::AUTHORITY()));
-    EXPECT_EQ(arcount, message.getRRCount(Section::ADDITIONAL()));
-}
-
 // Unsupported requests.  Should result in NOTIMP.
 TEST_F(AuthSrvTest, unsupportedRequest) {
-    for (unsigned int i = 0; i < 16; ++i) {
+    UNSUPPORTED_REQUEST_TEST;
-        // set Opcode to 'i', which iterators over all possible codes except
-        // the standard query and notify
-        if (i == Opcode::QUERY().getCode() ||
-            i == Opcode::NOTIFY().getCode()) {
-            continue;
-        }
-        createDataFromFile("simplequery_fromWire");
-        data[2] = ((i << 3) & 0xff);
-
-        parse_message->clear(Message::PARSE);
-        server.processMessage(*io_message, parse_message, response_obuffer,
-                              &dnsserv);
-        EXPECT_TRUE(dnsserv.hasAnswer());
-        headerCheck(*parse_message, default_qid, Rcode::NOTIMP(), i, QR_FLAG,
-                    0, 0, 0, 0);
-    }
 }
 
 // Simple API check
 TEST_F(AuthSrvTest, verbose) {
-    EXPECT_FALSE(server.getVerbose());
+    VERBOSE_TEST;
-    server.setVerbose(true);
-    EXPECT_TRUE(server.getVerbose());
-    server.setVerbose(false);
-    EXPECT_FALSE(server.getVerbose());
 }
 
 // Multiple questions.  Should result in FORMERR.
 TEST_F(AuthSrvTest, multiQuestion) {
-    createDataFromFile("multiquestion_fromWire");
+    MULTI_QUESTION_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 2, 0, 0, 0);
-
-    QuestionIterator qit = parse_message->beginQuestion();
-    EXPECT_EQ(Name("example.com"), (*qit)->getName());
-    EXPECT_EQ(RRClass::IN(), (*qit)->getClass());
-    EXPECT_EQ(RRType::A(), (*qit)->getType());
-    ++qit;
-    EXPECT_EQ(Name("example.com"), (*qit)->getName());
-    EXPECT_EQ(RRClass::IN(), (*qit)->getClass());
-    EXPECT_EQ(RRType::AAAA(), (*qit)->getType());
-    ++qit;
-    EXPECT_TRUE(qit == parse_message->endQuestion());
 }
 
 // Incoming data doesn't even contain the complete header.  Must be silently
 // dropped.
 TEST_F(AuthSrvTest, shortMessage) {
-    createDataFromFile("shortmessage_fromWire");
+    SHORT_MESSAGE_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
 }
 
 // Response messages.  Must be silently dropped, whether it's a valid response
 // or malformed or could otherwise cause a protocol error.
 TEST_F(AuthSrvTest, response) {
-    // A valid (although unusual) response
+    RESPONSE_TEST;
-    createDataFromFile("simpleresponse_fromWire");
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
-
-    // A response with a broken question section.  must be dropped rather than
-    // returning FORMERR.
-    createDataFromFile("shortresponse_fromWire");
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
-
-    // A response to iquery.  must be dropped rather than returning NOTIMP.
-    createDataFromFile("iqueryresponse_fromWire");
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
 }
 
 // Query with a broken question
 TEST_F(AuthSrvTest, shortQuestion) {
-    createDataFromFile("shortquestion_fromWire");
+    SHORT_QUESTION_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-    // Since the query's question is broken, the question section of the
-    // response should be empty.
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 0, 0, 0, 0);
 }
 
 // Query with a broken answer section
 TEST_F(AuthSrvTest, shortAnswer) {
-    createDataFromFile("shortanswer_fromWire");
+    SHORT_ANSWER_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-
-    // This is a bogus query, but question section is valid.  So the response
-    // should copy the question section.
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 0);
-
-    QuestionIterator qit = parse_message->beginQuestion();
-    EXPECT_EQ(Name("example.com"), (*qit)->getName());
-    EXPECT_EQ(RRClass::IN(), (*qit)->getClass());
-    EXPECT_EQ(RRType::A(), (*qit)->getType());
-    ++qit;
-    EXPECT_TRUE(qit == parse_message->endQuestion());
 }
 
 // Query with unsupported version of EDNS.
 TEST_F(AuthSrvTest, ednsBadVers) {
-    createDataFromFile("queryBadEDNS_fromWire");
+    EDNS_BADVERS_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-
-    // The response must have an EDNS OPT RR in the additional section.
-    // Note that the DNSSEC DO bit is cleared even if this bit in the query
-    // is set.  This is a limitation of the current implementation.
-    headerCheck(*parse_message, default_qid, Rcode::BADVERS(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 1);
-    EXPECT_EQ(4096, parse_message->getUDPSize());
-    EXPECT_FALSE(parse_message->isDNSSECSupported());
 }
 
 TEST_F(AuthSrvTest, AXFROverUDP) {
-    // AXFR over UDP is invalid and should result in FORMERR.
+    AXFR_OVER_UDP_TEST;
-    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("example.com"), RRClass::IN(),
-                         RRType::AXFR());
-    createRequestPacket(request_message, IPPROTO_UDP);
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 0);
 }
 
 TEST_F(AuthSrvTest, AXFRSuccess) {
@@ -318,7 +98,7 @@ TEST_F(AuthSrvTest, AXFRSuccess) {
     // so we shouldn't have to respond.
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_FALSE(dnsserv.hasAnswer());
-    EXPECT_FALSE(xfrout.isConnected());
+    EXPECT_TRUE(xfrout.isConnected());
 }
 
 TEST_F(AuthSrvTest, AXFRConnectFail) {
@@ -331,8 +111,6 @@ TEST_F(AuthSrvTest, AXFRConnectFail) {
     EXPECT_TRUE(dnsserv.hasAnswer());
     headerCheck(*parse_message, default_qid, Rcode::SERVFAIL(),
                 opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
-    // For a shot term workaround with xfrout we currently close the connection
-    // for each AXFR attempt
     EXPECT_FALSE(xfrout.isConnected());
 }
 
@@ -343,7 +121,7 @@ TEST_F(AuthSrvTest, AXFRSendFail) {
                          Name("example.com"), RRClass::IN(), RRType::AXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(xfrout.isConnected()); // see above
+    EXPECT_TRUE(xfrout.isConnected());
 
     xfrout.disableSend();
     parse_message->clear(Message::PARSE);
@@ -380,7 +158,7 @@ TEST_F(AuthSrvTest, AXFRDisconnectFail) {
 TEST_F(AuthSrvTest, notify) {
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -412,7 +190,7 @@ TEST_F(AuthSrvTest, notifyForCHClass) {
     // Same as the previous test, but for the CH RRClass.
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::CH(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -427,7 +205,8 @@ TEST_F(AuthSrvTest, notifyForCHClass) {
 TEST_F(AuthSrvTest, notifyEmptyQuestion) {
     request_message.clear(Message::RENDER);
     request_message.setOpcode(Opcode::NOTIFY());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setRcode(Rcode::NOERROR());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     request_message.setQid(default_qid);
     request_message.toWire(request_renderer);
     createRequestPacket(request_message, IPPROTO_UDP);
@@ -443,7 +222,7 @@ TEST_F(AuthSrvTest, notifyMultiQuestions) {
     // add one more SOA question
     request_message.addQuestion(Question(Name("example.com"), RRClass::IN(),
                                          RRType::SOA()));
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -454,7 +233,7 @@ TEST_F(AuthSrvTest, notifyMultiQuestions) {
 TEST_F(AuthSrvTest, notifyNonSOAQuestion) {
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::NS());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -476,7 +255,7 @@ TEST_F(AuthSrvTest, notifyWithoutAA) {
 TEST_F(AuthSrvTest, notifyWithErrorRcode) {
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     request_message.setRcode(Rcode::SERVFAIL());
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -490,7 +269,7 @@ TEST_F(AuthSrvTest, notifyWithoutSession) {
 
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
 
     // we simply ignore the notify and let it be resent if an internal error
@@ -504,7 +283,7 @@ TEST_F(AuthSrvTest, notifySendFail) {
 
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
 
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -516,7 +295,7 @@ TEST_F(AuthSrvTest, notifyReceiveFail) {
 
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_FALSE(dnsserv.hasAnswer());
@@ -527,7 +306,7 @@ TEST_F(AuthSrvTest, notifyWithBogusSessionMessage) {
 
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_FALSE(dnsserv.hasAnswer());
@@ -539,7 +318,7 @@ TEST_F(AuthSrvTest, notifyWithSessionMessageError) {
 
     UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
                          Name("example.com"), RRClass::IN(), RRType::SOA());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_FALSE(dnsserv.hasAnswer());
@@ -566,7 +345,7 @@ TEST_F(AuthSrvTest, updateConfig) {
     // query for existent data in the installed data source.  The resulting
     // response should have the AA flag on, and have an RR in each answer
     // and authority section.
-    createDataFromFile("examplequery_fromWire");
+    createDataFromFile("examplequery_fromWire.wire");
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
     headerCheck(*parse_message, default_qid, Rcode::NOERROR(), opcode.getCode(),
@@ -580,7 +359,7 @@ TEST_F(AuthSrvTest, datasourceFail) {
     // tool and the data source itself naively accept it).  This will result
     // in a SERVFAIL response, and the answer and authority sections should
     // be empty.
-    createDataFromFile("badExampleQuery_fromWire");
+    createDataFromFile("badExampleQuery_fromWire.wire");
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
     headerCheck(*parse_message, default_qid, Rcode::SERVFAIL(),
@@ -595,10 +374,20 @@ TEST_F(AuthSrvTest, updateConfigFail) {
     updateConfig(&server, BADCONFIG_TESTDB, false);
 
     // The original data source should still exist.
-    createDataFromFile("examplequery_fromWire");
+    createDataFromFile("examplequery_fromWire.wire");
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
     headerCheck(*parse_message, default_qid, Rcode::NOERROR(), opcode.getCode(),
                 QR_FLAG | AA_FLAG, 1, 1, 1, 0);
 }
+
+TEST_F(AuthSrvTest, cacheSlots) {
+    // simple check for the get/set operations
+    server.setCacheSlots(10);    // 10 = arbitrary choice
+    EXPECT_EQ(10, server.getCacheSlots());
+
+    // 0 is a valid size
+    server.setCacheSlots(0);
+    EXPECT_EQ(00, server.getCacheSlots());
+}
 }

src/bin/auth/tests/run_unittests.cc

@@ -19,10 +19,10 @@
 #include <dns/tests/unittest_util.h>
 
 int
-main(int argc, char* argv[])
+main(int argc, char* argv[]) {
-{
     ::testing::InitGoogleTest(&argc, argv);
     isc::UnitTestUtil::addDataPath(TEST_DATA_DIR);
+    isc::UnitTestUtil::addDataPath(TEST_DATA_BUILDDIR);
 
     return (RUN_ALL_TESTS());
 }

src/bin/auth/tests/testdata/badExampleQuery_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from badExampleQuery_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=broken.example.com QTYPE=AAAA(28) QCLASS=IN(1)
-0662726f6b656e076578616d706c6503636f6d00 001c 0001

src/bin/auth/tests/testdata/examplequery_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from examplequery_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=ns.example.com QTYPE=A(1) QCLASS=IN(1)
-026e73076578616d706c6503636f6d00 0001 0001

src/bin/auth/tests/testdata/iqueryresponse_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from iqueryresponse_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Response Opcode=IQUERY(1) Rcode=NOERROR(0)
-1035 c000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/auth/tests/testdata/multiquestion_fromWire

@@ -1,17 +0,0 @@
-###
-### This data file was auto-generated from multiquestion_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=2, ANCNT=0, NSCNT=0, ARCNT=0
-0002 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001
-
-# Question Section
-# QNAME=example.com. QTYPE=AAAA(28) QCLASS=IN(1)
-076578616d706c6503636f6d00 001c 0001

src/bin/auth/tests/testdata/queryBadEDNS_fromWire

@@ -1,19 +0,0 @@
-###
-### This data file was auto-generated from queryBadEDNS_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=1
-0001 0000 0000 0001
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001
-
-# EDNS OPT RR
-# NAME=. TYPE=OPT(41) UDPSize=4096 ExtRcode=0 Version=1 DO=1
-00 0029 1000 0001 8000
-# RDLEN=0
-0000

src/bin/auth/tests/testdata/shortanswer_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from shortanswer_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=1, NSCNT=0, ARCNT=0
-0001 0001 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/auth/tests/testdata/simplequery_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from simplequery_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/auth/tests/testdata/simpleresponse_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from simpleresponse_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Response Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 8000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/bind10/Makefile.am

@@ -1,4 +1,4 @@
-SUBDIRS = tests
+SUBDIRS = . tests
 
 sbin_SCRIPTS = bind10
 CLEANFILES = bind10 bind10.pyc

src/bin/bind10/bind10.8

@@ -1,7 +1,7 @@
 '\" t
 .\"     Title: bind10
 .\"    Author: [see the "AUTHORS" section]
-.\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
+.\" Generator: DocBook XSL Stylesheets v1.76.0 <http://docbook.sf.net/>
 .\"      Date: July 29, 2010
 .\"    Manual: BIND10
 .\"    Source: BIND10
@@ -9,6 +9,15 @@
 .\"
 .TH "BIND10" "8" "July 29, 2010" "BIND10" "BIND10"
 .\" -----------------------------------------------------------------
+.\" * Define some portability stuff
+.\" -----------------------------------------------------------------
+.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.\" http://bugs.debian.org/507673
+.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
+.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.\" -----------------------------------------------------------------
 .\" * set default formatting
 .\" -----------------------------------------------------------------
 .\" disable hyphenation
@@ -22,7 +31,7 @@
 bind10 \- BIND 10 boss process
 .SH "SYNOPSIS"
 .HP \w'\fBbind10\fR\ 'u
-\fBbind10\fR [\fB\-a\ \fR\fB\fIaddress\fR\fR] [\fB\-m\ \fR\fB\fIfile\fR\fR] [\fB\-n\fR] [\fB\-p\ \fR\fB\fInumber\fR\fR] [\fB\-u\ \fR\fB\fIuser\fR\fR] [\fB\-v\fR] [\fB\-\-address\ \fR\fB\fIaddress\fR\fR] [\fB\-\-msgq\-socket\-file\ \fR\fB\fIfile\fR\fR] [\fB\-\-no\-cache\fR] [\fB\-\-port\ \fR\fB\fInumber\fR\fR] [\fB\-\-user\ \fR\fB\fIuser\fR\fR] [\fB\-\-verbose\fR]
+\fBbind10\fR [\fB\-a\ \fR\fB\fIaddress\fR\fR] [\fB\-m\ \fR\fB\fIfile\fR\fR] [\fB\-n\fR] [\fB\-p\ \fR\fB\fInumber\fR\fR] [\fB\-u\ \fR\fB\fIuser\fR\fR] [\fB\-v\fR] [\fB\-\-address\ \fR\fB\fIaddress\fR\fR] [\fB\-\-msgq\-socket\-file\ \fR\fB\fIfile\fR\fR] [\fB\-\-no\-cache\fR] [\fB\-\-port\ \fR\fB\fInumber\fR\fR] [\fB\-\-user\ \fR\fB\fIuser\fR\fR] [\fB\-\-pretty\-name\ \fR\fB\fIname\fR\fR] [\fB\-\-verbose\fR]
 .SH "DESCRIPTION"
 .PP
 The
@@ -86,6 +95,15 @@ to run as\&.
 must be initially ran as the root user to use this option\&. The default is to run as the current user\&.
 .RE
 .PP
+\fB\-\-pretty\-name \fR\fB\fIname\fR\fR
+.RS 4
+The name this process should have in tools like
+\fBps\fR
+or
+\fBtop\fR\&. This is handy if you have multiple versions/installations of
+\fBbind10\fR\&.
+.RE
+.PP
 \fB\-v\fR, \fB\-\-verbose\fR
 .RS 4
 Display more about what is going on for
@@ -101,6 +119,8 @@ and its child processes\&.
 \fBb10-cmdctl\fR(8),
 \fBb10-msgq\fR(8),
 \fBb10-xfrin\fR(8),
+\fBb10-xfrout\fR(8),
+\fBb10-zonemgr\fR(8),
 BIND 10 Guide\&.
 .SH "HISTORY"
 .PP

src/bin/bind10/bind10.py.in

@@ -63,9 +63,19 @@ import pwd
 import posix
 
 import isc.cc
+import isc.util.process
+import isc.net.parse
+
+# Assign this process some longer name
+isc.util.process.rename(sys.argv[0])
 
 # This is the version that gets displayed to the user.
-__version__ = "v20100531"
+# The VERSION string consists of the module name, the module version
+# number, and the overall BIND 10 version number (set in configure.ac).
+VERSION = "bind10 20100916 (BIND 10 @PACKAGE_VERSION@)"
+
+# This is for bind10.boottime of stats module
+_BASETIME = time.gmtime()
 
 class RestartSchedule:
     """
@@ -131,9 +141,14 @@ class ProcessInfo:
         self.username = username
         self._spawn()
 
-    def _setuid(self):
+    def _preexec_work(self):
         """Function used before running a program that needs to run as a
         different user."""
+        # First, put us into a separate process group so we don't get
+        # SIGINT signals on Ctrl-C (the boss will shut everthing down by
+        # other means).
+        os.setpgrp()
+        # Second, set the user ID if one has been specified
         if self.uid is not None:
             try:
                 posix.setuid(self.uid)
@@ -167,42 +182,17 @@ class ProcessInfo:
                                         stderr=spawn_stderr,
                                         close_fds=True,
                                         env=spawn_env,
-                                        preexec_fn=self._setuid)
+                                        preexec_fn=self._preexec_work)
         self.pid = self.process.pid
         self.restart_schedule.set_run_start_time()
 
     def respawn(self):
         self._spawn()
 
-class IPAddr:
-    """Stores an IPv4 or IPv6 address."""
-    family = None
-    addr = None
-
-    def __init__(self, addr):
-        try:
-            a = socket.inet_pton(socket.AF_INET, addr)
-            self.family = socket.AF_INET
-            self.addr = a
-            return
-        except:
-            pass
-
-        try:
-            a = socket.inet_pton(socket.AF_INET6, addr)
-            self.family = socket.AF_INET6
-            self.addr = a
-            return
-        except Exception as e:
-            raise e
-    
-    def __str__(self):
-        return socket.inet_ntop(self.family, self.addr)
-
 class BoB:
     """Boss of BIND class."""
     
-    def __init__(self, msgq_socket_file=None, dns_port=5300, address='',
+    def __init__(self, msgq_socket_file=None, dns_port=5300, address=None,
                  forward=None, nocache=False, verbose=False, setuid=None,
                  username=None):
         """Initialize the Boss of BIND. This is a singleton (only one
@@ -218,11 +208,11 @@ class BoB:
         self.address = None
         self.nocache = nocache
         if address:
-            self.address = IPAddr(address)
+            self.address = address
         self.forward = None
         self.recursive = False
         if forward:
-            self.forward = IPAddr(forward)
+            self.forward = forward
             self.recursive = True
             self.nocache = False
         self.cc_session = None
@@ -363,13 +353,14 @@ class BoB:
             dns_prog = 'b10-recurse'
         else:
             dns_prog = 'b10-auth'
-        dnsargs = [dns_prog, '-p', str(self.dns_port)]
+        dnsargs = [dns_prog]
-        if self.forward:
+        if not self.recursive:
-            dnsargs += ['-f', str(self.forward)]
+            # The recursive uses configuration manager for these
-        if self.address:
+            dnsargs += ['-p', str(self.dns_port)]
-            dnsargs += ['-a', str(self.address)]
+            if self.address:
-        if self.nocache:
+                dnsargs += ['-a', str(self.address)]
-            dnsargs += ['-n']
+            if self.nocache:
+                dnsargs += ['-n']
         if self.uid:
             dnsargs += ['-u', str(self.uid)]
         if self.verbose:
@@ -442,6 +433,27 @@ class BoB:
                 sys.stdout.write("[bind10] Started b10-zonemgr(PID %d)\n" % 
                                  zonemgr.pid)
 
+        # start b10-stats
+        stats_args = ['b10-stats']
+        if self.verbose:
+            sys.stdout.write("[bind10] Starting b10-stats\n")
+            stats_args += ['-v']
+        try:
+            statsd = ProcessInfo("b10-stats", stats_args,
+                                 c_channel_env)
+        except Exception as e:
+            c_channel.process.kill()
+            bind_cfgd.process.kill()
+            xfrout.process.kill()
+            auth.process.kill()
+            xfrind.process.kill()
+            zonemgr.process.kill()
+            return "Unable to start b10-stats; " + str(e)
+
+        self.processes[statsd.pid] = statsd
+        if self.verbose:
+            sys.stdout.write("[bind10] Started b10-stats (PID %d)\n" % statsd.pid)
+
         # start the b10-cmdctl
         # XXX: we hardcode port 8080
         cmdctl_args = ['b10-cmdctl']
@@ -461,6 +473,7 @@ class BoB:
                 xfrind.process.kill()
             if zonemgr:
                 zonemgr.process.kill()
+            statsd.process.kill()
             return "Unable to start b10-cmdctl; " + str(e)
         self.processes[cmd_ctrld.pid] = cmd_ctrld
         if self.verbose:
@@ -474,13 +487,15 @@ class BoB:
     def stop_all_processes(self):
         """Stop all processes."""
         cmd = { "command": ['shutdown']}
-        self.cc_session.group_sendmsg(cmd, 'Boss', 'Cmdctl')
+
-        self.cc_session.group_sendmsg(cmd, "Boss", "ConfigManager")
+        self.cc_session.group_sendmsg(cmd, 'Cmdctl', 'Cmdctl')
-        self.cc_session.group_sendmsg(cmd, "Boss", "Auth")
+        self.cc_session.group_sendmsg(cmd, "ConfigManager", "ConfigManager")
-        self.cc_session.group_sendmsg(cmd, "Boss", "Recurse")
+        self.cc_session.group_sendmsg(cmd, "Auth", "Auth")
-        self.cc_session.group_sendmsg(cmd, "Boss", "Xfrout")
+        self.cc_session.group_sendmsg(cmd, "Recurse", "Recurse")
-        self.cc_session.group_sendmsg(cmd, "Boss", "Xfrin")
+        self.cc_session.group_sendmsg(cmd, "Xfrout", "Xfrout")
-        self.cc_session.group_sendmsg(cmd, "Boss", "Zonemgr")
+        self.cc_session.group_sendmsg(cmd, "Xfrin", "Xfrin")
+        self.cc_session.group_sendmsg(cmd, "Zonemgr", "Zonemgr")
+        self.cc_session.group_sendmsg(cmd, "Stats", "Stats")
 
     def stop_process(self, process):
         """Stop the given process, friendly-like."""
@@ -497,7 +512,9 @@ class BoB:
         except:
             pass
         # XXX: some delay probably useful... how much is uncertain
-        time.sleep(0.5)  
+        # I have changed the delay from 0.5 to 1, but sometime it's 
+        # still not enough.
+        time.sleep(1)  
         self.reap_children()
         # next try sending a SIGTERM
         processes_to_stop = list(self.processes.values())
@@ -604,7 +621,7 @@ def reaper(signal_number, stack_frame):
     # the Python signal handler has been set up to write
     # down a pipe, waking up our select() bit
     pass
-                   
+
 def get_signame(signal_number):
     """Return the symbolic name for a signal."""
     for sig in dir(signal):
@@ -626,29 +643,31 @@ def fatal_signal(signal_number, stack_frame):
 def check_port(option, opt_str, value, parser):
     """Function to insure that the port we are passed is actually 
     a valid port number. Used by OptionParser() on startup."""
-    if not re.match('^(6553[0-5]|655[0-2]\d|65[0-4]\d\d|6[0-4]\d{3}|[1-5]\d{4}|[1-9]\d{0,3}|0)$', value):
+    try:
-        raise OptionValueError("%s requires a port number (0-65535)" % opt_str)
+        if opt_str in ['-p', '--port']:
-    if (opt_str == '-m' or opt_str == '--msgq-port'):
+            parser.values.auth_port = isc.net.parse.port_parse(value)
-        parser.values.msgq_port = value
+        else:
-    elif (opt_str == '-p' or opt_str == '--port'):
+            raise OptionValueError("Unknown option " + opt_str)
-        parser.values.dns_port = value
+    except ValueError as e:
-    else:
+        raise OptionValueError(str(e))
-        raise OptionValueError("Unknown option " + opt_str)
+
-  
 def check_addr(option, opt_str, value, parser):
     """Function to insure that the address we are passed is actually 
     a valid address. Used by OptionParser() on startup."""
     try:
-        IPAddr(value)
+        if opt_str in ['-a', '--address']:
-    except:
+            parser.values.address = isc.net.parse.addr_parse(value)
+        if opt_str in ['-f', '--forward']:
+            parser.values.forward = isc.net.parse.addr_parse(value)
+        else:
+            raise OptionValueError("Unknown option " + opt_str)
+    except ValueError:
         raise OptionValueError("%s requires a valid IPv4 or IPv6 address" % opt_str)
-    if (opt_str == '-a' or opt_str == '--address'):
+
-        parser.values.address = value
+def process_rename(option, opt_str, value, parser):
-    elif (opt_str == '-f' or opt_str == '--forward'):
+    """Function that renames the process if it is requested by a option."""
-        parser.values.forward = value
+    isc.util.process.rename(value)
-    else:
+
-        raise OptionValueError("Unknown option " + opt_str)
-  
 def main():
     global options
     global boss_of_bind
@@ -656,7 +675,7 @@ def main():
     sys.stdout = io.TextIOWrapper(sys.stdout.detach(), line_buffering=True)
 
     # Parse any command-line options.
-    parser = OptionParser(version=__version__)
+    parser = OptionParser(version=VERSION)
     parser.add_option("-a", "--address", dest="address", type="string",
                       action="callback", callback=check_addr, default='',
                       help="address the DNS server will use (default: listen on all addresses)")
@@ -668,13 +687,16 @@ def main():
                       help="UNIX domain socket file the b10-msgq daemon will use")
     parser.add_option("-n", "--no-cache", action="store_true", dest="nocache",
                       default=False, help="disable hot-spot cache in authoritative DNS server")
-    parser.add_option("-p", "--port", dest="dns_port", type="string",
+    parser.add_option("-p", "--port", dest="dns_port", type="int",
-                      action="callback", callback=check_port, default="5300",
+                      action="callback", callback=check_port, default=5300,
                       help="port the DNS server will use (default 5300)")
     parser.add_option("-u", "--user", dest="user", type="string", default=None,
                       help="Change user after startup (must run as root)")
     parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
                       help="display more about what is going on")
+    parser.add_option("--pretty-name", type="string", action="callback",
+                      callback=process_rename,
+                      help="Set the process name (displayed in ps, top, ...)")
     (options, args) = parser.parse_args()
     if args:
         parser.print_help()
@@ -711,11 +733,7 @@ def main():
 
     # Announce startup.
     if options.verbose:
-        sys.stdout.write("BIND 10 %s\n" % __version__)
+        sys.stdout.write("%s\n" % VERSION)
-
-    # TODO: set process name, perhaps by:
-    #       http://code.google.com/p/procname/
-    #       http://github.com/lericson/procname/
 
     # Create wakeup pipe for signal handlers
     wakeup_pipe = os.pipe()
@@ -728,8 +746,11 @@ def main():
     signal.signal(signal.SIGINT, fatal_signal)
     signal.signal(signal.SIGTERM, fatal_signal)
 
+    # Block SIGPIPE, as we don't want it to end this process
+    signal.signal(signal.SIGPIPE, signal.SIG_IGN)
+
     # Go bob!
-    boss_of_bind = BoB(options.msgq_socket_file, int(options.dns_port),
+    boss_of_bind = BoB(options.msgq_socket_file, options.dns_port,
                        options.address, options.forward, options.nocache,
                        options.verbose, setuid, username)
     startup_result = boss_of_bind.startup()
@@ -738,6 +759,17 @@ def main():
         sys.exit(1)
     sys.stdout.write("[bind10] BIND 10 started\n")
 
+    # send "bind10.boot_time" to b10-stats
+    time.sleep(1) # wait a second
+    if options.verbose:
+        sys.stdout.write("[bind10] send \"bind10.boot_time\" to b10-stats\n")
+    cmd = isc.config.ccsession.create_command('set', 
+            { "stats_data": {
+              'bind10.boot_time': time.strftime('%Y-%m-%dT%H:%M:%SZ', _BASETIME)
+              }
+            })
+    boss_of_bind.cc_session.group_sendmsg(cmd, 'Stats')
+
     # In our main loop, we check for dead processes or messages 
     # on the c-channel.
     wakeup_fd = wakeup_pipe[0]
@@ -766,7 +798,12 @@ def main():
 
         for fd in rlist + xlist:
             if fd == ccs_fd:
-                boss_of_bind.ccs.check_command()
+                try:
+                    boss_of_bind.ccs.check_command()
+                except isc.cc.session.ProtocolError:
+                    if options.verbose:
+                        sys.stderr.write("[bind10] msgq channel disappeared.\n")
+                    break
             elif fd == wakeup_fd:
                 os.read(wakeup_fd, 32)
 

src/bin/bind10/bind10.xml

@@ -56,6 +56,7 @@
       <arg><option>--no-cache</option></arg>
       <arg><option>--port <replaceable>number</replaceable></option></arg>
       <arg><option>--user <replaceable>user</replaceable></option></arg>
+      <arg><option>--pretty-name <replaceable>name</replaceable></option></arg>
       <arg><option>--verbose</option></arg>
     </cmdsynopsis>
   </refsynopsisdiv>
@@ -149,6 +150,17 @@
       </varlistentry>
 
       <varlistentry>
+        <term><option>--pretty-name <replaceable>name</replaceable></option></term>
+
+        <listitem>
+          <para>The name this process should have in tools like
+          <command>ps</command> or <command>top</command>. This
+          is handy if you have multiple versions/installations
+          of <command>bind10</command>.</para>
+        </listitem>
+      </varlistentry>
+
+      <varlistentry>
         <term><option>-v</option>, <option>--verbose</option></term>
         <listitem>
 	  <para>Display more about what is going on for
@@ -189,6 +201,12 @@
       <citerefentry>
         <refentrytitle>b10-xfrin</refentrytitle><manvolnum>8</manvolnum>
       </citerefentry>,
+      <citerefentry>
+        <refentrytitle>b10-xfrout</refentrytitle><manvolnum>8</manvolnum>
+      </citerefentry>,
+      <citerefentry>
+        <refentrytitle>b10-zonemgr</refentrytitle><manvolnum>8</manvolnum>
+      </citerefentry>,
       <citetitle>BIND 10 Guide</citetitle>.
     </para>
   </refsect1>

src/bin/bind10/run_bind10.sh.in

@@ -20,7 +20,7 @@ export PYTHON_EXEC
 
 BIND10_PATH=@abs_top_builddir@/src/bin/bind10
 
-PATH=@abs_top_builddir@/src/bin/msgq:@abs_top_builddir@/src/bin/auth:@abs_top_builddir@/src/bin/recurse:@abs_top_builddir@/src/bin/cfgmgr:@abs_top_builddir@/src/bin/cmdctl:@abs_top_builddir@/src/bin/xfrin:@abs_top_builddir@/src/bin/xfrout:@abs_top_builddir@/src/bin/zonemgr:$PATH
+PATH=@abs_top_builddir@/src/bin/msgq:@abs_top_builddir@/src/bin/auth:@abs_top_builddir@/src/bin/recurse:@abs_top_builddir@/src/bin/cfgmgr:@abs_top_builddir@/src/bin/cmdctl:@abs_top_builddir@/src/bin/stats:@abs_top_builddir@/src/bin/xfrin:@abs_top_builddir@/src/bin/xfrout:@abs_top_builddir@/src/bin/zonemgr:$PATH
 export PATH
 
 PYTHONPATH=@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/xfr/.libs

src/bin/bind10/tests/args_test.py

@@ -130,5 +130,28 @@ class TestBossArgs(unittest.TestCase):
         x = bob.wait()
         self.assertTrue(bob.wait() == 0)
 
+    def testPrettyName(self):
+        """Try the --pretty-name option."""
+        CMD_PRETTY_NAME = b'bob-name-test'
+        bob = subprocess.Popen(args=(BIND10_EXE, '--pretty-name',
+            CMD_PRETTY_NAME), stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        started_ok = self._waitForString(bob, '[bind10] BIND 10 started')
+        self.assertTrue(started_ok)
+        ps = subprocess.Popen(args=("ps", "axo", "pid,comm"),
+                              stdout=subprocess.PIPE)
+        s = ps.stdout.readline()
+        command = None
+        while True:
+            s = ps.stdout.readline()
+            if s == '': break
+            (pid,comm) = s.split(None, 1)
+            if int(pid) == bob.pid:
+                command = comm
+                break
+        self.assertEqual(command, CMD_PRETTY_NAME + b'\n')
+        time.sleep(0.1)
+        bob.terminate()
+        bob.wait()
+
 if __name__ == '__main__':
     unittest.main()

src/bin/bind10/tests/bind10_test.py

@@ -1,4 +1,4 @@
-from bind10 import ProcessInfo, BoB, IPAddr
+from bind10 import ProcessInfo, BoB
 
 # XXX: environment tests are currently disabled, due to the preprocessor
 #      setup that we have now complicating the environment
@@ -8,6 +8,7 @@ import sys
 import os
 import signal
 import socket
+from isc.net.addr import IPAddr
 
 class TestProcessInfo(unittest.TestCase):
     def setUp(self):
@@ -72,28 +73,6 @@ class TestProcessInfo(unittest.TestCase):
         self.assertTrue(type(pi.pid) is int)
         self.assertNotEqual(pi.pid, old_pid)
 
-class TestIPAddr(unittest.TestCase):
-    def test_v6ok(self):
-        addr = IPAddr('2001:4f8::1')
-        self.assertEqual(addr.family, socket.AF_INET6)
-        self.assertEqual(addr.addr, socket.inet_pton(socket.AF_INET6, '2001:4f8::1'))
-
-    def test_v4ok(self):
-        addr = IPAddr('127.127.127.127')
-        self.assertEqual(addr.family, socket.AF_INET)
-        self.assertEqual(addr.addr, socket.inet_aton('127.127.127.127'))
-
-    def test_badaddr(self):
-        self.assertRaises(socket.error, IPAddr, 'foobar')
-        self.assertRaises(socket.error, IPAddr, 'foo::bar')
-        self.assertRaises(socket.error, IPAddr, '123')
-        self.assertRaises(socket.error, IPAddr, '123.456.789.0')
-        self.assertRaises(socket.error, IPAddr, '127/8')
-        self.assertRaises(socket.error, IPAddr, '0/0')
-        self.assertRaises(socket.error, IPAddr, '1.2.3.4/32')
-        self.assertRaises(socket.error, IPAddr, '0')
-        self.assertRaises(socket.error, IPAddr, '')
-
 class TestBoB(unittest.TestCase):
     def test_init(self):
         bob = BoB()
@@ -127,7 +106,7 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.runnable, False)
 
     def test_init_alternate_address(self):
-        bob = BoB(None, 5300, '127.127.127.127')
+        bob = BoB(None, 5300, IPAddr('127.127.127.127'))
         self.assertEqual(bob.verbose, False)
         self.assertEqual(bob.dns_port, 5300)
         self.assertEqual(bob.msgq_socket_file, None)

src/bin/bindctl/Makefile.am

@@ -1,4 +1,4 @@
-SUBDIRS = tests
+SUBDIRS = . tests
 
 bin_SCRIPTS = bindctl
 man_MANS = bindctl.1

src/bin/bindctl/bindctl-source.py.in

@@ -24,6 +24,9 @@ from bindctl.moduleinfo import *
 from bindctl.bindcmd import *
 import pprint
 from optparse import OptionParser, OptionValueError
+import isc.util.process
+
+isc.util.process.rename()
 
 __version__ = 'Bindctl'
 

src/bin/cfgmgr/Makefile.am

@@ -1,4 +1,4 @@
-SUBDIRS = tests
+SUBDIRS = . tests
 
 pkglibexecdir = $(libexecdir)/@PACKAGE@
 

src/bin/cfgmgr/b10-cfgmgr.py.in

@@ -21,9 +21,12 @@ import sys; sys.path.append ('@@PYTHONPATH@@')
 
 from isc.config.cfgmgr import ConfigManager, ConfigManagerDataReadError
 from isc.cc import SessionError
+import isc.util.process
 import signal
 import os
 
+isc.util.process.rename()
+
 # If B10_FROM_SOURCE is set in the environment, we use data files
 # from a directory relative to that, otherwise we use the ones
 # installed on the system

src/bin/cmdctl/Makefile.am

@@ -1,4 +1,4 @@
-SUBDIRS = tests
+SUBDIRS = . tests
 
 pkglibexecdir = $(libexecdir)/@PACKAGE@
 
@@ -15,7 +15,7 @@ CMDCTL_CONFIGURATIONS += cmdctl-keyfile.pem cmdctl-certfile.pem
 
 b10_cmdctl_DATA = $(CMDCTL_CONFIGURATIONS)
 b10_cmdctl_DATA += cmdctl.spec
- 
+
 EXTRA_DIST = $(CMDCTL_CONFIGURATIONS)
 
 CLEANFILES=	b10-cmdctl cmdctl.pyc cmdctl.spec

src/bin/cmdctl/cmdctl.py.in

@@ -1,6 +1,7 @@
 #!@PYTHON@
 
 # Copyright (C) 2010  Internet Systems Consortium.
+# Copyright (C) 2010  CZ NIC
 #
 # Permission to use, copy, modify, and distribute this software for any
 # purpose with or without fee is hereby granted, provided that the above
@@ -42,13 +43,19 @@ import random
 import time
 import signal
 from isc.config import ccsession
+import isc.util.process
+import isc.net.parse
 from optparse import OptionParser, OptionValueError
 from hashlib import sha1
+from isc.util import socketserver_mixin
+
 try:
     import threading
 except ImportError:
     import dummy_threading as threading
 
+isc.util.process.rename()
+
 __version__ = 'BIND10'
 URL_PATTERN = re.compile('/([\w]+)(?:/([\w]+))?/?')
 CONFIG_DATA_URL = 'config_data'
@@ -320,8 +327,8 @@ class CommandControl():
     def _handle_msg_from_msgq(self):
         '''Process all the received commands with module session. '''
         while self._serving:
-            self._module_cc.check_command() 
+            self._module_cc.check_command(False)
- 
+
     def _parse_command_result(self, rcode, reply):
         '''Ignore the error reason when command rcode isn't 0, '''
         if rcode != 0:
@@ -380,6 +387,7 @@ class CommandControl():
     def send_command(self, module_name, command_name, params = None):
         '''Send the command from bindctl to proper module. '''
         errstr = 'unknown error'
+        answer = None
         if self._verbose:
             self.log_info("Begin send command '%s' to module '%s'" %(command_name, module_name))
 
@@ -390,7 +398,10 @@ class CommandControl():
             msg = ccsession.create_command(command_name, params)
             seq = self._cc.group_sendmsg(msg, module_name)
             #TODO, it may be blocked, msqg need to add a new interface waiting in timeout.
-            answer, env = self._cc.group_recvmsg(False, seq)
+            try:
+                answer, env = self._cc.group_recvmsg(False, seq)
+            except isc.cc.session.SessionTimeout:
+                errstr = "Module '%s' not responding" % module_name
 
         if self._verbose:
             self.log_info("Finish send command '%s' to module '%s'" % (command_name, module_name))
@@ -410,7 +421,6 @@ class CommandControl():
             except ccsession.ModuleCCSessionError as mcse:
                 errstr = str("Error in ccsession answer:") + str(mcse)
                 self.log_info(errstr)
-        
         return 1, {'error': errstr}
     
     def log_info(self, msg):
@@ -433,7 +443,9 @@ class CommandControl():
 
         return (keyfile, certfile, accountsfile)
 
-class SecureHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
+class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
+                       socketserver.ThreadingMixIn,
+                       http.server.HTTPServer):
     '''Make the server address can be reused.'''
     allow_reuse_address = True
 
@@ -441,6 +453,7 @@ class SecureHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
                  CommandControlClass,
                  idle_timeout = 1200, verbose = False):
         '''idle_timeout: the max idle time for login'''
+        socketserver_mixin.NoPollMixIn.__init__(self)
         try:
             http.server.HTTPServer.__init__(self, server_address, RequestHandlerClass)
         except socket.error as err:
@@ -558,22 +571,17 @@ def run(addr = 'localhost', port = 8080, idle_timeout = 1200, verbose = False):
     httpd.serve_forever()
 
 def check_port(option, opt_str, value, parser):
-    if (value < 0) or (value > 65535):
+    try:
-        raise OptionValueError('%s requires a port number (0-65535)' % opt_str)
+        parser.values.port = isc.net.parse.port_parse(value)
-    parser.values.port = value
+    except ValueError as e:
+        raise OptionValueError(str(e))
 
 def check_addr(option, opt_str, value, parser):
-    ipstr = value
-    ip_family = socket.AF_INET
-    if (ipstr.find(':') != -1):
-        ip_family = socket.AF_INET6
-
     try:
-        socket.inet_pton(ip_family, ipstr)
+        isc.net.parse.addr_parse(value)
-    except:
+        parser.values.addr = value
-        raise OptionValueError("%s invalid ip address" % ipstr)
+    except ValueError as e:
-
+        raise OptionValueError(str(e))
-    parser.values.addr = value
 
 def set_cmd_options(parser):
     parser.add_option('-p', '--port', dest = 'port', type = 'int',
@@ -602,6 +610,9 @@ if __name__ == '__main__':
     except isc.cc.SessionError as err:
         sys.stderr.write("[b10-cmdctl] Error creating b10-cmdctl, "
                          "is the command channel daemon running?\n")        
+    except isc.cc.SessionTimeout:
+        sys.stderr.write("[b10-cmdctl] Error creating b10-cmdctl, "
+                         "is the configuration manager running?\n")        
     except KeyboardInterrupt:
         sys.stderr.write("[b10-cmdctl] exit from Cmdctl\n")
     except CmdctlException as err:

src/bin/host/Makefile.am

@@ -1,5 +1,6 @@
 AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/dns -I$(top_builddir)/src/lib/dns
+AM_CPPFLAGS += $(BOOST_INCLUDES)
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 

src/bin/host/host.cc

@@ -28,6 +28,8 @@
 #include <dns/name.h>
 #include <dns/message.h>
 #include <dns/messagerenderer.h>
+#include <dns/opcode.h>
+#include <dns/rcode.h>
 #include <dns/rrclass.h>
 #include <dns/rrtype.h>
 #include <dns/rrset.h>
@@ -56,7 +58,7 @@ host_lookup(const char* const name, const char* const type) {
     msg.setOpcode(Opcode::QUERY());
     msg.setRcode(Rcode::NOERROR());
     if (recursive_bit) {
-        msg.setHeaderFlag(MessageFlag::RD());    // set recursive bit
+        msg.setHeaderFlag(Message::HEADERFLAG_RD); // set recursive bit
     }
 
     msg.addQuestion(Question(Name(name),
@@ -120,9 +122,10 @@ host_lookup(const char* const name, const char* const type) {
 
             rmsg.fromWire(ibuffer);
             if (!verbose) {
-                  for (RRsetIterator it = rmsg.beginSection(Section::ANSWER());
+                for (RRsetIterator it =
-                       it != rmsg.endSection(Section::ANSWER());
+                         rmsg.beginSection(Message::SECTION_ANSWER);
-                       ++it) {
+                     it != rmsg.endSection(Message::SECTION_ANSWER);
+                     ++it) {
                       if ((*it)->getType() != RRType::A()) {
                           continue;
                       }

src/bin/loadzone/Makefile.am

@@ -1,5 +1,4 @@
-SUBDIRS = tests/correct
+SUBDIRS = . tests/correct tests/error
-SUBDIRS += tests/error
 bin_SCRIPTS = b10-loadzone
 
 CLEANFILES = b10-loadzone

src/bin/loadzone/b10-loadzone.py.in

@@ -18,9 +18,13 @@
 import sys; sys.path.append ('@@PYTHONPATH@@')
 import re, getopt
 import isc.datasrc
+import isc.util.process
 from isc.datasrc.master import MasterFile
 import time
 import os
+
+isc.util.process.rename()
+
 #########################################################################
 # usage: print usage note and exit
 #########################################################################

src/bin/loadzone/tests/error/error.known

@@ -1,5 +1,5 @@
 Error reading zone file: Cannot parse RR, No $ORIGIN: @ IN SOA ns hostmaster 1 3600 1800 1814400 3600
-Error reading zone file: $ORIGIN is not absolute in record:$ORIGIN com
+Error reading zone file: $ORIGIN is not absolute in record: $ORIGIN com
 Error reading zone file: Cannot parse RR: $TL 300
 Error reading zone file: Cannot parse RR: $OIGIN com.
 Error loading database: Error while loading com.: Cannot parse RR: $INLUDE file.txt

src/bin/msgq/Makefile.am

@@ -1,4 +1,4 @@
-SUBDIRS = tests
+SUBDIRS = . tests
 
 pkglibexecdir = $(libexecdir)/@PACKAGE@
  

src/bin/msgq/msgq.py.in

@@ -31,9 +31,12 @@ import select
 import pprint
 import random
 from optparse import OptionParser, OptionValueError
+import isc.util.process
 
 import isc.cc
 
+isc.util.process.rename()
+
 # This is the version that gets displayed to the user.
 __version__ = "v20091030 (Paving the DNS Parking Lot)"
 
@@ -139,6 +142,10 @@ class MsgQ:
 
     def setup_listener(self):
         """Set up the listener socket.  Internal function."""
+        if self.verbose:
+            sys.stdout.write("[b10-msgq] Setting up socket at %s\n" %
+                             self.socket_file)
+
         self.listen_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
         
         if os.path.exists(self.socket_file):

src/bin/msgq/tests/Makefile.am

@@ -8,6 +8,7 @@ check-local:
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
 	env PYTHONPATH=$(abs_top_builddir)/src/bin/msgq:$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python \
+	BIND10_TEST_SOCKET_FILE=$(builddir)/test_msgq_socket.sock \
 	$(PYCOVERAGE) $(abs_srcdir)/$$pytest || exit ; \
 	done
 

src/bin/msgq/tests/msgq_test.py

@@ -79,9 +79,14 @@ class TestSubscriptionManager(unittest.TestCase):
 
     def test_open_socket_default(self):
         env_var = None
+        orig_socket_file = None
         if "BIND10_MSGQ_SOCKET_FILE" in os.environ:
             env_var = os.environ["BIND10_MSGQ_SOCKET_FILE"]
             del os.environ["BIND10_MSGQ_SOCKET_FILE"]
+        # temporarily replace the class "default" not to be disrupted by
+        # any running BIND 10 instance.
+        if "BIND10_TEST_SOCKET_FILE" in os.environ:
+            MsgQ.SOCKET_FILE = os.environ["BIND10_TEST_SOCKET_FILE"]
         socket_file = MsgQ.SOCKET_FILE
         self.assertFalse(os.path.exists(socket_file))
         msgq = MsgQ();
@@ -96,6 +101,8 @@ class TestSubscriptionManager(unittest.TestCase):
             pass
         if env_var is not None:
             os.environ["BIND10_MSGQ_SOCKET_FILE"] = env_var
+        if orig_socket_file is not None:
+            MsgQ.SOCKET_FILE = orig_socket_file
 
     def test_open_socket_bad(self):
         msgq = MsgQ("/does/not/exist")

src/bin/recurse/Makefile.am

@@ -6,6 +6,7 @@ AM_CPPFLAGS += -I$(top_srcdir)/src/lib/dns -I$(top_builddir)/src/lib/dns
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/cc -I$(top_builddir)/src/lib/cc
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/asiolink
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/asiolink
+AM_CPPFLAGS += $(BOOST_INCLUDES)
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 

src/bin/recurse/main.cc

@@ -22,7 +22,7 @@
 #include <stdlib.h>
 #include <errno.h>
 
-#include <cassert>
+#include <string>
 #include <iostream>
 
 #include <boost/foreach.hpp>
@@ -47,21 +47,19 @@
 #include <recurse/spec_config.h>
 #include <recurse/recursor.h>
 
+#include <log/dummylog.h>
+
 using namespace std;
-using namespace isc::data;
 using namespace isc::cc;
 using namespace isc::config;
-using namespace isc::dns;
+using namespace isc::data;
-using namespace isc::xfr;
+using isc::log::dlog;
 using namespace asiolink;
 
 namespace {
 
-static bool verbose_mode = false;
-
 // Default port current 5300 for testing purposes
 static const string PROGRAM = "Recurse";
-static const char* DNSPORT = "5300";
 
 IOService io_service;
 static Recursor *recursor;
@@ -82,21 +80,13 @@ my_command_handler(const string& command, ConstElementPtr args) {
     } else if (command == "shutdown") {
         io_service.stop();
     }
-    
+
     return (answer);
 }
 
 void
 usage() {
-    cerr << "Usage:  b10-recurse -f nameserver [-a address] [-p port] [-u user]"
+    cerr << "Usage:  b10-recurse [-u user] [-v]" << endl;
-                     "[-4|-6] [-v]" << endl;
-    cerr << "\t-f: specify the nameserver to which queries should be forwarded"
-         << endl;
-    cerr << "\t-a: specify the address to listen on (default: all)" << endl;
-    cerr << "\t-p: specify the port to listen on (default: " << DNSPORT << ")"
-         << endl;
-    cerr << "\t-4: listen on all IPv4 addresses (incompatible with -a)" << endl;
-    cerr << "\t-6: listen on all IPv6 addresses (incompatible with -a)" << endl;
     cerr << "\t-u: change process UID to the specified user" << endl;
     cerr << "\t-v: verbose output" << endl;
     exit(1);
@@ -105,40 +95,17 @@ usage() {
 
 int
 main(int argc, char* argv[]) {
+    isc::log::dprefix = "b10-recurse";
     int ch;
-    const char* port = DNSPORT;
-    const char* address = NULL;
-    const char* forward = NULL;
     const char* uid = NULL;
-    bool use_ipv4 = true, use_ipv6 = true;
 
-    while ((ch = getopt(argc, argv, "46a:f:p:u:v")) != -1) {
+    while ((ch = getopt(argc, argv, "u:v")) != -1) {
         switch (ch) {
-        case '4':
-            // Note that -4 means "ipv4 only", we need to set "use_ipv6" here,
-            // not "use_ipv4".  We could use something like "ipv4_only", but
-            // we found the negatively named variable could confuse the code
-            // logic.
-            use_ipv6 = false;
-            break;
-        case '6':
-            // The same note as -4 applies.
-            use_ipv4 = false;
-            break;
-        case 'a':
-            address = optarg;
-            break;
-        case 'f':
-            forward = optarg;
-            break;
-        case 'p':
-            port = optarg;
-            break;
         case 'u':
             uid = optarg;
             break;
         case 'v':
-            verbose_mode = true;
+            isc::log::denabled = true;
             break;
         case '?':
         default:
@@ -150,26 +117,16 @@ main(int argc, char* argv[]) {
         usage();
     }
 
-    if (!use_ipv4 && !use_ipv6) {
+    if (isc::log::denabled) { // Show the command line
-        cerr << "[b10-auth] Error: Cannot specify both -4 and -6 "
+        string cmdline("Command line:");
-             << "at the same time" << endl;
+        for (int i = 0; i < argc; ++ i) {
-        usage();
+            cmdline = cmdline + " " + argv[i];
-    }
+        }
-
+        dlog(cmdline);
-    if ((!use_ipv4 || !use_ipv6) && address != NULL) {
-        cerr << "[b10-auth] Error: Cannot specify -4 or -6 "
-             << "at the same time as -a" << endl;
-        usage();
-    }
-
-    if (forward == NULL) {
-        cerr << "[b10-recurse] No forward name server specified" << endl;
-        usage();
     }
 
     int ret = 0;
 
-    // XXX: we should eventually pass io_service here.
     Session* cc_session = NULL;
     ModuleCCSession* config_session = NULL;
     try {
@@ -181,51 +138,39 @@ main(int argc, char* argv[]) {
             specfile = string(RECURSE_SPECFILE_LOCATION);
         }
 
-        recursor = new Recursor(*forward);
+        recursor = new Recursor();
-        recursor->setVerbose(verbose_mode);
+        dlog("Server created.");
-        cout << "[b10-recurse] Server created." << endl;
 
         SimpleCallback* checkin = recursor->getCheckinProvider();
         DNSLookup* lookup = recursor->getDNSLookupProvider();
         DNSAnswer* answer = recursor->getDNSAnswerProvider();
 
-        DNSService* dns_service;
+        DNSService dns_service(io_service, checkin, lookup, answer);
-
+
-        if (address != NULL) {
+        recursor->setDNSService(dns_service);
-            // XXX: we can only specify at most one explicit address.
+        dlog("IOService created.");
-            // This also means the server cannot run in the dual address
-            // family mode if explicit addresses need to be specified.
-            // We don't bother to fix this problem, however.  The -a option
-            // is a short term workaround until we support dynamic listening
-            // port allocation.
-            dns_service = new DNSService(io_service, *port, *address,
-                                         checkin, lookup, answer);
-        } else {
-            dns_service = new DNSService(io_service, *port, use_ipv4, use_ipv6,
-                                         checkin, lookup, answer);
-        }
-        recursor->setDNSService(*dns_service);
-        cout << "[b10-recurse] IOService created." << endl;
 
         cc_session = new Session(io_service.get_io_service());
-        cout << "[b10-recurse] Configuration session channel created." << endl;
+        dlog("Configuration session channel created.");
 
         config_session = new ModuleCCSession(specfile, *cc_session,
                                              my_config_handler,
                                              my_command_handler);
-        cout << "[b10-recurse] Configuration channel established." << endl;
+        dlog("Configuration channel established.");
 
+        // FIXME: This does not belong here, but inside Boss
         if (uid != NULL) {
             changeUser(uid);
         }
 
         recursor->setConfigSession(config_session);
-        recursor->updateConfig(ElementPtr());
+        recursor->updateConfig(config_session->getFullConfig());
+        dlog("Config loaded");
 
-        cout << "[b10-recurse] Server started." << endl;
+        dlog("Server started.");
         io_service.run();
     } catch (const std::exception& ex) {
-        cerr << "[b10-recurse] Server failed: " << ex.what() << endl;
+        dlog(string("Server failed: ") + ex.what());
         ret = 1;
     }
 

src/bin/recurse/recurse.spec.pre.in

@@ -1,18 +1,74 @@
 {
   "module_spec": {
-    "module_name": "Auth",
+    "module_name": "Recurse",
-    "module_description": "Authoritative service",
+    "module_description": "Recursive service",
     "config_data": [
-      { "item_name": "database_file",
+      {
-        "item_type": "string",
+        "item_name": "forward_addresses",
-        "item_optional": true,
+        "item_type": "list",
-        "item_default": "@@LOCALSTATEDIR@@/@PACKAGE@/zone.sqlite3"
+        "item_optional": True,
+        "item_default": [],
+        "list_item_spec" : {
+          "item_name": "address",
+          "item_type": "map",
+          "item_optional": False,
+          "item_default": {},
+          "map_item_spec": [
+            {
+              "item_name": "address",
+              "item_type": "string",
+              "item_optional": False,
+              "item_default": "::1"
+            },
+            {
+              "item_name": "port",
+              "item_type": "integer",
+              "item_optional": False,
+              "item_default": 53
+            }
+          ]
+        }
+      },
+      {
+        "item_name": "listen_on",
+        "item_type": "list",
+        "item_optional": False,
+        "item_default": [
+          {
+            "address": "::1",
+            "port": 5300
+          },
+          {
+            "address": "127.0.0.1",
+            "port": 5300
+          },
+        ],
+        "list_item_spec": {
+          "item_name": "address",
+          "item_type": "map",
+          "item_optional": False,
+          "item_default": {},
+          "map_item_spec": [
+            {
+              "item_name": "address",
+              "item_type": "string",
+              "item_optional": False,
+              "item_default": "::1"
+            },
+            {
+              "item_name": "port",
+              "item_type": "integer",
+              "item_optional": False,
+              "item_default": 5300
+            }
+          ]
+        }
       }
     ],
     "commands": [
       {
         "command_name": "shutdown",
-        "command_description": "Shut down authoritative DNS server",
+        "command_description": "Shut down recursive DNS server",
         "command_args": []
       }
     ]

src/bin/recurse/recursor.cc

@@ -19,20 +19,20 @@
 #include <netinet/in.h>
 
 #include <algorithm>
-#include <cassert>
-#include <iostream>
 #include <vector>
 
 #include <asiolink/asiolink.h>
+#include <asiolink/ioaddress.h>
 
 #include <boost/foreach.hpp>
+#include <boost/lexical_cast.hpp>
 
 #include <config/ccsession.h>
 
-#include <cc/data.h>
-
 #include <exceptions/exceptions.h>
 
+#include <dns/opcode.h>
+#include <dns/rcode.h>
 #include <dns/buffer.h>
 #include <dns/exceptions.h>
 #include <dns/name.h>
@@ -42,30 +42,30 @@
 #include <dns/message.h>
 #include <dns/messagerenderer.h>
 
-#include <xfr/xfrout_client.h>
+#include <log/dummylog.h>
 
 #include <recurse/recursor.h>
 
 using namespace std;
 
 using namespace isc;
-using namespace isc::cc;
 using namespace isc::dns;
-using namespace isc::dns::rdata;
 using namespace isc::data;
 using namespace isc::config;
-using namespace isc::xfr;
+using isc::log::dlog;
 using namespace asiolink;
 
+typedef pair<string, uint16_t> addr_t;
+
 class RecursorImpl {
 private:
     // prohibit copy
     RecursorImpl(const RecursorImpl& source);
     RecursorImpl& operator=(const RecursorImpl& source);
 public:
-    RecursorImpl(const char& forward) :
+    RecursorImpl() :
-        config_session_(NULL), verbose_mode_(false),
+        config_session_(NULL),
-        forward_(forward), rec_query_()
+        rec_query_()
     {}
 
     ~RecursorImpl() {
@@ -73,12 +73,33 @@ public:
     }
 
     void querySetup(DNSService& dnss) {
-        rec_query_ = new RecursiveQuery(dnss, forward_);
+        dlog("Query setup");
+        rec_query_ = new RecursiveQuery(dnss, upstream_);
     }
 
     void queryShutdown() {
-        if (rec_query_) {
+        dlog("Query shutdown");
-            delete rec_query_;
+        delete rec_query_;
+        rec_query_ = NULL;
+    }
+
+    void setForwardAddresses(const vector<addr_t>& upstream,
+        DNSService *dnss)
+    {
+        queryShutdown();
+        upstream_ = upstream;
+        if (dnss) {
+            if (upstream_.empty()) {
+                dlog("Asked to do full recursive, but not implemented yet. "
+                    "I'll do nothing.");
+            } else {
+                dlog("Setting forward addresses:");
+                BOOST_FOREACH(const addr_t& address, upstream) {
+                    dlog(" " + address.first + ":" +
+                        boost::lexical_cast<string>(address.second));
+                }
+                querySetup(*dnss);
+            }
         }
     }
 
@@ -91,11 +112,12 @@ public:
 
     /// These members are public because Recursor accesses them directly.
     ModuleCCSession* config_session_;
-    bool verbose_mode_;
+    /// Addresses of the forward nameserver
+    vector<addr_t> upstream_;
+    /// Addresses we listen on
+    vector<addr_t> listen_;
 
 private:
-    /// Address of the forward nameserver
-    const char& forward_;
 
     /// Object to handle upstream queries
     RecursiveQuery* rec_query_;
@@ -105,6 +127,8 @@ class QuestionInserter {
 public:
     QuestionInserter(MessagePtr message) : message_(message) {}
     void operator()(const QuestionPtr question) {
+        dlog(string("Adding question ") + question->getName().toText() +
+            " to message");
         message_->addQuestion(question);
     }
     MessagePtr message_;
@@ -112,27 +136,30 @@ public:
 
 class SectionInserter {
 public:
-    SectionInserter(MessagePtr message, const Section& sect, bool sign) :
+    SectionInserter(MessagePtr message, const Message::Section sect,
+        bool sign) :
         message_(message), section_(sect), sign_(sign)
     {}
     void operator()(const RRsetPtr rrset) {
+        dlog("Adding RRSet to message section " +
+            boost::lexical_cast<string>(section_));
         message_->addRRset(section_, rrset, true);
     }
     MessagePtr message_;
-    const Section& section_;
+    const Message::Section section_;
     bool sign_;
 };
 
 void
 makeErrorMessage(MessagePtr message, OutputBufferPtr buffer,
-                 const Rcode& rcode, const bool verbose_mode)
+                 const Rcode& rcode)
 {
     // extract the parameters that should be kept.
     // XXX: with the current implementation, it's not easy to set EDNS0
     // depending on whether the query had it.  So we'll simply omit it.
     const qid_t qid = message->getQid();
-    const bool rd = message->getHeaderFlag(MessageFlag::RD());
+    const bool rd = message->getHeaderFlag(Message::HEADERFLAG_RD);
-    const bool cd = message->getHeaderFlag(MessageFlag::CD());
+    const bool cd = message->getHeaderFlag(Message::HEADERFLAG_CD);
     const Opcode& opcode = message->getOpcode();
     vector<QuestionPtr> questions;
 
@@ -145,23 +172,21 @@ makeErrorMessage(MessagePtr message, OutputBufferPtr buffer,
     message->clear(Message::RENDER);
     message->setQid(qid);
     message->setOpcode(opcode);
-    message->setHeaderFlag(MessageFlag::QR());
+    message->setHeaderFlag(Message::HEADERFLAG_QR);
-    message->setUDPSize(RecursorImpl::DEFAULT_LOCAL_UDPSIZE);
     if (rd) {
-        message->setHeaderFlag(MessageFlag::RD());
+        message->setHeaderFlag(Message::HEADERFLAG_RD);
     }
     if (cd) {
-        message->setHeaderFlag(MessageFlag::CD());
+        message->setHeaderFlag(Message::HEADERFLAG_CD);
     }
     for_each(questions.begin(), questions.end(), QuestionInserter(message));
     message->setRcode(rcode);
     MessageRenderer renderer(*buffer);
     message->toWire(renderer);
 
-    if (verbose_mode) {
+    dlog(string("Sending an error response (") +
-        cerr << "[b10-recurse] sending an error response (" <<
+        boost::lexical_cast<string>(renderer.getLength()) + " bytes):\n" +
-            renderer.getLength() << " bytes):\n" << message->toText() << endl;
+        message->toText());
-    }
 }
 
 // This is a derived class of \c DNSLookup, to serve as a
@@ -193,8 +218,8 @@ public:
                             OutputBufferPtr buffer) const
     {
         const qid_t qid = message->getQid();
-        const bool rd = message->getHeaderFlag(MessageFlag::RD());
+        const bool rd = message->getHeaderFlag(Message::HEADERFLAG_RD);
-        const bool cd = message->getHeaderFlag(MessageFlag::CD());
+        const bool cd = message->getHeaderFlag(Message::HEADERFLAG_CD);
         const Opcode& opcode = message->getOpcode();
         const Rcode& rcode = message->getRcode();
         vector<QuestionPtr> questions;
@@ -204,15 +229,14 @@ public:
         message->setQid(qid);
         message->setOpcode(opcode);
         message->setRcode(rcode);
-        message->setUDPSize(RecursorImpl::DEFAULT_LOCAL_UDPSIZE);
 
-        message->setHeaderFlag(MessageFlag::QR());
+        message->setHeaderFlag(Message::HEADERFLAG_QR);
-        message->setHeaderFlag(MessageFlag::RA());
+        message->setHeaderFlag(Message::HEADERFLAG_RA);
         if (rd) {
-            message->setHeaderFlag(MessageFlag::RD());
+            message->setHeaderFlag(Message::HEADERFLAG_RD);
         }
         if (cd) {
-            message->setHeaderFlag(MessageFlag::CD());
+            message->setHeaderFlag(Message::HEADERFLAG_CD);
         }
 
 
@@ -227,15 +251,18 @@ public:
                 Message incoming(Message::PARSE);
                 InputBuffer ibuf(buffer->getData(), buffer->getLength());
                 incoming.fromWire(ibuf);
-                for_each(incoming.beginSection(Section::ANSWER()), 
+                for_each(incoming.beginSection(Message::SECTION_ANSWER),
-                         incoming.endSection(Section::ANSWER()),
+                         incoming.endSection(Message::SECTION_ANSWER),
-                         SectionInserter(message, Section::ANSWER(), true));
+                         SectionInserter(message, Message::SECTION_ANSWER,
-                for_each(incoming.beginSection(Section::ADDITIONAL()), 
+                         true));
-                         incoming.endSection(Section::ADDITIONAL()),
+                for_each(incoming.beginSection(Message::SECTION_ADDITIONAL),
-                         SectionInserter(message, Section::ADDITIONAL(), true));
+                         incoming.endSection(Message::SECTION_ADDITIONAL),
-                for_each(incoming.beginSection(Section::AUTHORITY()), 
+                         SectionInserter(message, Message::SECTION_ADDITIONAL,
-                         incoming.endSection(Section::AUTHORITY()),
+                         true));
-                         SectionInserter(message, Section::AUTHORITY(), true));
+                for_each(incoming.beginSection(Message::SECTION_AUTHORITY),
+                         incoming.endSection(Message::SECTION_ADDITIONAL),
+                         SectionInserter(message, Message::SECTION_AUTHORITY,
+                         true));
             } catch (const Exception& ex) {
                 // Incoming message couldn't be read, we just SERVFAIL
                 message->setRcode(Rcode::SERVFAIL());
@@ -248,18 +275,18 @@ public:
         MessageRenderer renderer(*buffer);
 
         if (io_message.getSocket().getProtocol() == IPPROTO_UDP) {
-            renderer.setLengthLimit(message->getUDPSize());
+            ConstEDNSPtr edns(message->getEDNS());
+            renderer.setLengthLimit(edns ? edns->getUDPSize() :
+                Message::DEFAULT_MAX_UDPSIZE);
         } else {
             renderer.setLengthLimit(65535);
         }
 
         message->toWire(renderer);
 
-        if (server_->getVerbose()) {
+        dlog(string("sending a response (") +
-            cerr << "[b10-recurse] sending a response ("
+            boost::lexical_cast<string>(renderer.getLength()) + "bytes): \n" +
-                 << renderer.getLength() << " bytes):\n"
+            message->toText());
-                 << message->toText() << endl;
-        }
     }
 
 private:
@@ -272,7 +299,7 @@ private:
 class ConfigCheck : public SimpleCallback {
 public:
     ConfigCheck(Recursor* srv) : server_(srv) {}
-    virtual void operator()(const IOMessage& io_message UNUSED_PARAM) const {
+    virtual void operator()(const IOMessage&) const {
         if (server_->getConfigSession()->hasQueuedMsgs()) {
             server_->getConfigSession()->checkCommand();
         }
@@ -281,8 +308,8 @@ private:
     Recursor* server_;
 };
 
-Recursor::Recursor(const char& forward) :
+Recursor::Recursor() :
-    impl_(new RecursorImpl(forward)),
+    impl_(new RecursorImpl()),
     checkin_(new ConfigCheck(this)),
     dns_lookup_(new MessageLookup(this)),
     dns_answer_(new MessageAnswer(this))
@@ -293,6 +320,7 @@ Recursor::~Recursor() {
     delete checkin_;
     delete dns_lookup_;
     delete dns_answer_;
+    dlog("Deleting the Recursor");
 }
 
 void
@@ -303,16 +331,6 @@ Recursor::setDNSService(asiolink::DNSService& dnss) {
 }
 
 void
-Recursor::setVerbose(const bool on) {
-    impl_->verbose_mode_ = on;
-}
-
-bool
-Recursor::getVerbose() const {
-    return (impl_->verbose_mode_);
-}
-
-void
 Recursor::setConfigSession(ModuleCCSession* config_session) {
     impl_->config_session_ = config_session;
 }
@@ -326,6 +344,7 @@ void
 Recursor::processMessage(const IOMessage& io_message, MessagePtr message,
                         OutputBufferPtr buffer, DNSServer* server)
 {
+    dlog("Got a DNS message");
     InputBuffer request_buffer(io_message.getData(), io_message.getDataSize());
     // First, check the header part.  If we fail even for the base header,
     // just drop the message.
@@ -333,18 +352,13 @@ Recursor::processMessage(const IOMessage& io_message, MessagePtr message,
         message->parseHeader(request_buffer);
 
         // Ignore all responses.
-        if (message->getHeaderFlag(MessageFlag::QR())) {
+        if (message->getHeaderFlag(Message::HEADERFLAG_QR)) {
-            if (impl_->verbose_mode_) {
+            dlog("Received unexpected response, ignoring");
-                cerr << "[b10-recurse] received unexpected response, ignoring"
-                     << endl;
-            }
             server->resume(false);
             return;
         }
     } catch (const Exception& ex) {
-        if (impl_->verbose_mode_) {
+        dlog(string("DNS packet exception: ") + ex.what());
-            cerr << "[b10-recurse] DNS packet exception: " << ex.what() << endl;
-        }
         server->resume(false);
         return;
     }
@@ -353,57 +367,45 @@ Recursor::processMessage(const IOMessage& io_message, MessagePtr message,
     try {
         message->fromWire(request_buffer);
     } catch (const DNSProtocolError& error) {
-        if (impl_->verbose_mode_) {
+        dlog(string("returning ") + error.getRcode().toText() + ": " + 
-            cerr << "[b10-recurse] returning " <<  error.getRcode().toText()
+            error.what());
-                 << ": " << error.what() << endl;
+        makeErrorMessage(message, buffer, error.getRcode());
-        }
-        makeErrorMessage(message, buffer, error.getRcode(),
-                         impl_->verbose_mode_);
         server->resume(true);
         return;
     } catch (const Exception& ex) {
-        if (impl_->verbose_mode_) {
+        dlog(string("returning SERVFAIL: ") + ex.what());
-            cerr << "[b10-recurse] returning SERVFAIL: " << ex.what() << endl;
+        makeErrorMessage(message, buffer, Rcode::SERVFAIL());
-        }
-        makeErrorMessage(message, buffer, Rcode::SERVFAIL(),
-                         impl_->verbose_mode_);
         server->resume(true);
         return;
     } // other exceptions will be handled at a higher layer.
 
-    if (impl_->verbose_mode_) {
+    dlog("received a message:\n" + message->toText());
-        cerr << "[b10-recurse] received a message:\n"
-             << message->toText() << endl;
-    }
 
     // Perform further protocol-level validation.
     bool sendAnswer = true;
     if (message->getOpcode() == Opcode::NOTIFY()) {
-        makeErrorMessage(message, buffer, Rcode::NOTAUTH(),
+        makeErrorMessage(message, buffer, Rcode::NOTAUTH());
-                         impl_->verbose_mode_);
+        dlog("Notify arrived, but we are not authoritative");
     } else if (message->getOpcode() != Opcode::QUERY()) {
-        if (impl_->verbose_mode_) {
+        dlog("Unsupported opcode (got: " + message->getOpcode().toText() +
-            cerr << "[b10-recurse] unsupported opcode" << endl;
+            ", expected: " + Opcode::QUERY().toText());
-        }
+        makeErrorMessage(message, buffer, Rcode::NOTIMP());
-        makeErrorMessage(message, buffer, Rcode::NOTIMP(),
+    } else if (message->getRRCount(Message::SECTION_QUESTION) != 1) {
-                         impl_->verbose_mode_);
+        dlog("The query contained " +
-    } else if (message->getRRCount(Section::QUESTION()) != 1) {
+            boost::lexical_cast<string>(message->getRRCount(
-        makeErrorMessage(message, buffer, Rcode::FORMERR(),
+            Message::SECTION_QUESTION) + " questions, exactly one expected"));
-                         impl_->verbose_mode_);
+        makeErrorMessage(message, buffer, Rcode::FORMERR());
     } else {
         ConstQuestionPtr question = *message->beginQuestion();
         const RRType &qtype = question->getType();
         if (qtype == RRType::AXFR()) {
             if (io_message.getSocket().getProtocol() == IPPROTO_UDP) {
-                makeErrorMessage(message, buffer, Rcode::FORMERR(),
+                makeErrorMessage(message, buffer, Rcode::FORMERR());
-                                 impl_->verbose_mode_);
             } else {
-                makeErrorMessage(message, buffer, Rcode::NOTIMP(),
+                makeErrorMessage(message, buffer, Rcode::NOTIMP());
-                                 impl_->verbose_mode_);
             }
         } else if (qtype == RRType::IXFR()) {
-            makeErrorMessage(message, buffer, Rcode::NOTIMP(),
+            makeErrorMessage(message, buffer, Rcode::NOTIMP());
-                         impl_->verbose_mode_);
         } else {
             // The RecursiveQuery object will post the "resume" event to the
             // DNSServer when an answer arrives, so we don't have to do it now.
@@ -421,26 +423,146 @@ void
 RecursorImpl::processNormalQuery(const Question& question, MessagePtr message,
                                  OutputBufferPtr buffer, DNSServer* server)
 {
-    const bool dnssec_ok = message->isDNSSECSupported();
+    dlog("Processing normal query");
+    ConstEDNSPtr edns(message->getEDNS());
+    const bool dnssec_ok = edns && edns->getDNSSECAwareness();
 
     message->makeResponse();
-    message->setHeaderFlag(MessageFlag::RA());
+    message->setHeaderFlag(Message::HEADERFLAG_RA);
     message->setRcode(Rcode::NOERROR());
-    message->setDNSSECSupported(dnssec_ok);
+    if (edns) {
-    message->setUDPSize(RecursorImpl::DEFAULT_LOCAL_UDPSIZE);
+        EDNSPtr edns_response(new EDNS());
+        edns_response->setDNSSECAwareness(dnssec_ok);
+        edns_response->setUDPSize(RecursorImpl::DEFAULT_LOCAL_UDPSIZE);
+        message->setEDNS(edns_response);
+    }
     rec_query_->sendQuery(question, buffer, server);
 }
 
+namespace {
+
+vector<addr_t>
+parseAddresses(ConstElementPtr addresses) {
+    vector<addr_t> result;
+    if (addresses) {
+        if (addresses->getType() == Element::list) {
+            for (size_t i(0); i < addresses->size(); ++ i) {
+                ConstElementPtr addrPair(addresses->get(i));
+                ConstElementPtr addr(addrPair->get("address"));
+                ConstElementPtr port(addrPair->get("port"));
+                if (!addr || ! port) {
+                    isc_throw(BadValue, "Address must contain both the IP"
+                        "address and port");
+                }
+                try {
+                    IOAddress(addr->stringValue());
+                    if (port->intValue() < 0 ||
+                        port->intValue() > 0xffff) {
+                        isc_throw(BadValue, "Bad port value (" <<
+                            port->intValue() << ")");
+                    }
+                    result.push_back(addr_t(addr->stringValue(),
+                        port->intValue()));
+                }
+                catch (const TypeError &e) { // Better error message
+                    isc_throw(TypeError,
+                        "Address must be a string and port an integer");
+                }
+            }
+        } else if (addresses->getType() != Element::null) {
+            isc_throw(TypeError,
+                "forward_addresses config element must be a list");
+        }
+    }
+    return (result);
+}
+
+}
+
 ConstElementPtr
-Recursor::updateConfig(ConstElementPtr new_config UNUSED_PARAM) {
+Recursor::updateConfig(ConstElementPtr config) {
+    dlog("New config comes: " + config->toWire());
+
     try {
-        // We will do configuration updates here.  None are presently
+        // Parse forward_addresses
-        // defined, so we just return an empty answer.
+        ConstElementPtr forwardAddressesE(config->get("forward_addresses"));
+        vector<addr_t> forwardAddresses(parseAddresses(forwardAddressesE));
+        ConstElementPtr listenAddressesE(config->get("listen_on"));
+        vector<addr_t> listenAddresses(parseAddresses(listenAddressesE));
+        // Everything OK, so commit the changes
+        // listenAddresses can fail to bind, so try them first
+        if (listenAddressesE) {
+            setListenAddresses(listenAddresses);
+        }
+        if (forwardAddressesE) {
+            setForwardAddresses(forwardAddresses);
+        }
         return (isc::config::createAnswer());
     } catch (const isc::Exception& error) {
-        if (impl_->verbose_mode_) {
+        dlog(string("error in config: ") + error.what());
-            cerr << "[b10-recurse] error: " << error.what() << endl;
-        }
         return (isc::config::createAnswer(1, error.what()));
     }
 }
+
+void
+Recursor::setForwardAddresses(const vector<addr_t>& addresses)
+{
+    impl_->setForwardAddresses(addresses, dnss_);
+}
+
+bool
+Recursor::isForwarding() const {
+    return (!impl_->upstream_.empty());
+}
+
+vector<addr_t>
+Recursor::getForwardAddresses() const {
+    return (impl_->upstream_);
+}
+
+namespace {
+
+void
+setAddresses(DNSService *service, const vector<addr_t>& addresses) {
+    service->clearServers();
+    BOOST_FOREACH(const addr_t &address, addresses) {
+        service->addServer(address.second, address.first);
+    }
+}
+
+}
+
+void
+Recursor::setListenAddresses(const vector<addr_t>& addresses) {
+    try {
+        dlog("Setting listen addresses:");
+        BOOST_FOREACH(const addr_t& addr, addresses) {
+            dlog(" " + addr.first + boost::lexical_cast<string>(addr.second));
+        }
+        setAddresses(dnss_, addresses);
+        impl_->listen_ = addresses;
+    }
+    catch (const exception& e) {
+        /*
+         * We couldn't set it. So return it back. If that fails as well,
+         * we have a problem.
+         *
+         * If that fails, bad luck, but we are useless anyway, so just die
+         * and let boss start us again.
+         */
+        try {
+            setAddresses(dnss_, impl_->listen_);
+        }
+        catch (const exception& e2) {
+            dlog(string("Unable to recover from error: ") + e.what() +
+                " Rollback failed with: " + e2.what());
+            abort();
+        }
+        throw e; // Let it fly a little bit further
+    }
+}
+
+vector<addr_t>
+Recursor::getListenAddresses() const {
+    return (impl_->listen_);
+}

src/bin/recurse/recursor.h

@@ -18,6 +18,8 @@
 #define __RECURSOR_H 1
 
 #include <string>
+#include <vector>
+#include <utility>
 
 #include <cc/data.h>
 #include <config/ccsession.h>
@@ -38,12 +40,7 @@ private:
     Recursor& operator=(const Recursor& source);
 public:
     /// The constructor.
-    ///
+    Recursor();
-    /// \param forward The address of the name server to which requests
-    /// should be forwarded.  (In the future, when the server is running
-    /// in forwarding mode, the forward nameserver addresses will be set
-    /// via the config channel instaed.)
-    Recursor(const char& forward);
     ~Recursor();
     //@}
 
@@ -63,14 +60,6 @@ public:
                         isc::dns::OutputBufferPtr buffer,
                         asiolink::DNSServer* server);
 
-    /// \brief Set verbose flag
-    ///
-    /// \param on The new value of the verbose flag
-    void setVerbose(bool on);
-
-    /// \brief Get the current value of the verbose flag
-    bool getVerbose() const;
-
     /// \brief Set and get the config session
     isc::config::ModuleCCSession* getConfigSession() const;
     void setConfigSession(isc::config::ModuleCCSession* config_session);
@@ -93,6 +82,34 @@ public:
     /// \brief Return pointer to the Checkin callback function
     asiolink::SimpleCallback* getCheckinProvider() { return (checkin_); }
 
+    /**
+     * \brief Specify the list of upstream servers.
+     *
+     * Specify the list off addresses of upstream servers to forward queries
+     * to. If the list is empty, this server is set to full recursive mode.
+     * If it is non-empty, it switches to forwarder.
+     *
+     * @param addresses The list of addresses to use (each one is the address
+     * and port pair).
+     */
+    void setForwardAddresses(const std::vector<std::pair<std::string,
+        uint16_t> >& addresses);
+    /**
+     * \short Get list of upstream addresses.
+     *
+     * \see setForwardAddresses.
+     */
+    std::vector<std::pair<std::string, uint16_t> > getForwardAddresses() const;
+    /// Return if we are in forwarding mode (if not, we are in fully recursive)
+    bool isForwarding() const;
+
+    /**
+     * Set and get the addresses we listen on.
+     */
+    void setListenAddresses(const std::vector<std::pair<std::string,
+        uint16_t> >& addresses);
+    std::vector<std::pair<std::string, uint16_t> > getListenAddresses() const;
+
 private:
     RecursorImpl* impl_;
     asiolink::DNSService* dnss_;

src/bin/recurse/tests/Makefile.am

@@ -1,7 +1,10 @@
 AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/dns -I$(top_srcdir)/src/bin
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/cc
-AM_CPPFLAGS += -DTEST_DATA_DIR=\"$(srcdir)/testdata\"
+AM_CPPFLAGS += -DTEST_DATA_DIR=\"$(top_srcdir)/src/lib/testutils/testdata\"
+AM_CPPFLAGS += -DTEST_DATA_BUILDDIR=\"$(abs_top_builddir)/src/lib/testutils/testdata\"
+AM_CPPFLAGS += $(BOOST_INCLUDES)
+AM_CPPFLAGS += $(BOOST_INCLUDES)
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 
@@ -17,7 +20,6 @@ TESTS += run_unittests
 run_unittests_SOURCES = $(top_srcdir)/src/lib/dns/tests/unittest_util.h
 run_unittests_SOURCES += $(top_srcdir)/src/lib/dns/tests/unittest_util.cc
 run_unittests_SOURCES += ../recursor.h ../recursor.cc
-run_unittests_SOURCES += ../../auth/tests/mockups.h
 run_unittests_SOURCES += recursor_unittest.cc
 run_unittests_SOURCES += run_unittests.cc
 run_unittests_CPPFLAGS = $(AM_CPPFLAGS) $(GTEST_INCLUDES)
@@ -34,13 +36,3 @@ run_unittests_LDADD += $(top_builddir)/src/lib/xfr/libxfr.la
 endif
 
 noinst_PROGRAMS = $(TESTS)
-
-EXTRA_DIST = testdata/iqueryresponse_fromWire
-EXTRA_DIST += testdata/multiquestion_fromWire
-EXTRA_DIST += testdata/queryBadEDNS_fromWire
-EXTRA_DIST += testdata/shortanswer_fromWire
-EXTRA_DIST += testdata/shortmessage_fromWire
-EXTRA_DIST += testdata/shortquestion_fromWire
-EXTRA_DIST += testdata/shortresponse_fromWire
-EXTRA_DIST += testdata/simplequery_fromWire
-EXTRA_DIST += testdata/simpleresponse_fromWire

src/bin/recurse/tests/recursor_unittest.cc

@@ -15,28 +15,8 @@
 // $Id$
 
 #include <config.h>
-
-#include <gtest/gtest.h>
-
-#include <asiolink/asiolink.h>
-
-#include <dns/buffer.h>
-#include <dns/name.h>
-#include <dns/message.h>
-#include <dns/messagerenderer.h>
-#include <dns/rrclass.h>
-#include <dns/rrtype.h>
-
-#include <cc/data.h>
-#include <cc/session.h>
-
-#include <auth/common.h>
-
 #include <recurse/recursor.h>
-
+#include <testutils/srv_unittest.h>
-#include <dns/tests/unittest_util.h>
-
-#include <auth/tests/mockups.h>
 
 using isc::UnitTestUtil;
 using namespace std;
@@ -46,257 +26,53 @@ using namespace isc::data;
 using namespace asiolink;
 
 namespace {
-const char* const DEFAULT_REMOTE_ADDRESS = "192.0.2.1";
 const char* const TEST_PORT = "53535";
 
-class DummySocket : public IOSocket {
+class RecursorTest : public SrvTestBase{
-private:
-    DummySocket(const DummySocket& source);
-    DummySocket& operator=(const DummySocket& source);
-public:
-    DummySocket(const int protocol) : protocol_(protocol) {}
-    virtual int getNative() const { return (-1); }
-    virtual int getProtocol() const { return (protocol_); }
-private:
-    const int protocol_;
-};
-
-class RecursorTest : public ::testing::Test {
 protected:
-    RecursorTest() : server(*DEFAULT_REMOTE_ADDRESS),
+    RecursorTest() : server(){}
-                     request_message(Message::RENDER),
-                     parse_message(new Message(Message::PARSE)),
-                     default_qid(0x1035), opcode(Opcode(Opcode::QUERY())),
-                     qname("www.example.com"),
-                     qclass(RRClass::IN()), qtype(RRType::A()),
-                     io_message(NULL), endpoint(NULL), request_obuffer(0),
-                     request_renderer(request_obuffer),
-                     response_obuffer(new OutputBuffer(0))
-    {}
-    ~RecursorTest() {
-        delete io_message;
-        delete endpoint;
-    }
-    MockSession notify_session;
-    MockServer dnsserv;
     Recursor server;
-    Message request_message;
-    MessagePtr parse_message;
-    const qid_t default_qid;
-    const Opcode opcode;
-    const Name qname;
-    const RRClass qclass;
-    const RRType qtype;
-    IOMessage* io_message;
-    IOSocket* io_sock;
-    const IOEndpoint* endpoint;
-    OutputBuffer request_obuffer;
-    MessageRenderer request_renderer;
-    OutputBufferPtr response_obuffer;
-    vector<uint8_t> data;
-
-    void createDataFromFile(const char* const datafile, int protocol);
-    void createRequestPacket(Message& message, int protocol);
 };
 
-void
-RecursorTest::createDataFromFile(const char* const datafile,
-                                const int protocol = IPPROTO_UDP)
-{
-    delete io_message;
-    data.clear();
-
-    delete endpoint;
-    endpoint = IOEndpoint::create(protocol,
-                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 5300);
-    UnitTestUtil::readWireData(datafile, data);
-    io_sock = new DummySocket(protocol);
-    io_message = new IOMessage(&data[0], data.size(), *io_sock, *endpoint);
-}
-
-void
-RecursorTest::createRequestPacket(Message& message,
-                                  const int protocol = IPPROTO_UDP)
-{
-    message.toWire(request_renderer);
-
-    delete io_message;
-
-    endpoint = IOEndpoint::create(protocol,
-                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 5300);
-    io_sock = new DummySocket(protocol);
-    io_message = new IOMessage(request_renderer.getData(),
-                               request_renderer.getLength(),
-                               *io_sock, *endpoint);
-}
-
-// These are flags to indicate whether the corresponding flag bit of the
-// DNS header is to be set in the test cases.  (Note that the flag values
-// is irrelevant to their wire-format values)
-const unsigned int QR_FLAG = 0x1;
-const unsigned int AA_FLAG = 0x2;
-const unsigned int TC_FLAG = 0x4;
-const unsigned int RD_FLAG = 0x8;
-const unsigned int RA_FLAG = 0x10;
-const unsigned int AD_FLAG = 0x20;
-const unsigned int CD_FLAG = 0x40;
-
-void
-headerCheck(const Message& message, const qid_t qid, const Rcode& rcode,
-            const uint16_t opcodeval, const unsigned int flags,
-            const unsigned int qdcount,
-            const unsigned int ancount, const unsigned int nscount,
-            const unsigned int arcount)
-{
-    EXPECT_EQ(qid, message.getQid());
-    EXPECT_EQ(rcode, message.getRcode());
-    EXPECT_EQ(opcodeval, message.getOpcode().getCode());
-    EXPECT_EQ((flags & QR_FLAG) != 0, message.getHeaderFlag(MessageFlag::QR()));
-    EXPECT_EQ((flags & AA_FLAG) != 0, message.getHeaderFlag(MessageFlag::AA()));
-    EXPECT_EQ((flags & TC_FLAG) != 0, message.getHeaderFlag(MessageFlag::TC()));
-    EXPECT_EQ((flags & RA_FLAG) != 0, message.getHeaderFlag(MessageFlag::RA()));
-    EXPECT_EQ((flags & RD_FLAG) != 0, message.getHeaderFlag(MessageFlag::RD()));
-    EXPECT_EQ((flags & AD_FLAG) != 0, message.getHeaderFlag(MessageFlag::AD()));
-    EXPECT_EQ((flags & CD_FLAG) != 0, message.getHeaderFlag(MessageFlag::CD()));
-
-    EXPECT_EQ(qdcount, message.getRRCount(Section::QUESTION()));
-    EXPECT_EQ(ancount, message.getRRCount(Section::ANSWER()));
-    EXPECT_EQ(nscount, message.getRRCount(Section::AUTHORITY()));
-    EXPECT_EQ(arcount, message.getRRCount(Section::ADDITIONAL()));
-}
-
 // Unsupported requests.  Should result in NOTIMP.
 TEST_F(RecursorTest, unsupportedRequest) {
-    for (unsigned int i = 0; i < 16; ++i) {
+    UNSUPPORTED_REQUEST_TEST;
-        // set Opcode to 'i', which iterators over all possible codes except
-        // the standard query and notify
-        if (i == Opcode::QUERY().getCode() ||
-            i == Opcode::NOTIFY().getCode()) {
-            continue;
-        }
-        createDataFromFile("simplequery_fromWire");
-        data[2] = ((i << 3) & 0xff);
-
-        parse_message->clear(Message::PARSE);
-        server.processMessage(*io_message, parse_message,
-                              response_obuffer, &dnsserv);
-        EXPECT_TRUE(dnsserv.hasAnswer());
-        headerCheck(*parse_message, default_qid, Rcode::NOTIMP(), i, QR_FLAG,
-                    0, 0, 0, 0);
-    }
-}
-
-// Simple API check
-TEST_F(RecursorTest, verbose) {
-    EXPECT_FALSE(server.getVerbose());
-    server.setVerbose(true);
-    EXPECT_TRUE(server.getVerbose());
-    server.setVerbose(false);
-    EXPECT_FALSE(server.getVerbose());
 }
 
 // Multiple questions.  Should result in FORMERR.
 TEST_F(RecursorTest, multiQuestion) {
-    createDataFromFile("multiquestion_fromWire");
+    MULTI_QUESTION_TEST; 
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 2, 0, 0, 0);
-
-    QuestionIterator qit = parse_message->beginQuestion();
-    EXPECT_EQ(Name("example.com"), (*qit)->getName());
-    EXPECT_EQ(RRClass::IN(), (*qit)->getClass());
-    EXPECT_EQ(RRType::A(), (*qit)->getType());
-    ++qit;
-    EXPECT_EQ(Name("example.com"), (*qit)->getName());
-    EXPECT_EQ(RRClass::IN(), (*qit)->getClass());
-    EXPECT_EQ(RRType::AAAA(), (*qit)->getType());
-    ++qit;
-    EXPECT_TRUE(qit == parse_message->endQuestion());
 }
 
 // Incoming data doesn't even contain the complete header.  Must be silently
 // dropped.
 TEST_F(RecursorTest, shortMessage) {
-    createDataFromFile("shortmessage_fromWire");
+    SHORT_MESSAGE_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
 }
 
 // Response messages.  Must be silently dropped, whether it's a valid response
 // or malformed or could otherwise cause a protocol error.
 TEST_F(RecursorTest, response) {
-    // A valid (although unusual) response
+    RESPONSE_TEST;
-    createDataFromFile("simpleresponse_fromWire");
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
-
-    // A response with a broken question section.  must be dropped rather than
-    // returning FORMERR.
-    createDataFromFile("shortresponse_fromWire");
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
-
-    // A response to iquery.  must be dropped rather than returning NOTIMP.
-    createDataFromFile("iqueryresponse_fromWire");
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_FALSE(dnsserv.hasAnswer());
 }
 
 // Query with a broken question
 TEST_F(RecursorTest, shortQuestion) {
-    createDataFromFile("shortquestion_fromWire");
+    SHORT_QUESTION_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-    // Since the query's question is broken, the question section of the
-    // response should be empty.
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 0, 0, 0, 0);
 }
 
 // Query with a broken answer section
 TEST_F(RecursorTest, shortAnswer) {
-    createDataFromFile("shortanswer_fromWire");
+    SHORT_ANSWER_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-
-    // This is a bogus query, but question section is valid.  So the response
-    // should copy the question section.
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 0);
-
-    QuestionIterator qit = parse_message->beginQuestion();
-    EXPECT_EQ(Name("example.com"), (*qit)->getName());
-    EXPECT_EQ(RRClass::IN(), (*qit)->getClass());
-    EXPECT_EQ(RRType::A(), (*qit)->getType());
-    ++qit;
-    EXPECT_TRUE(qit == parse_message->endQuestion());
 }
 
 // Query with unsupported version of EDNS.
 TEST_F(RecursorTest, ednsBadVers) {
-    createDataFromFile("queryBadEDNS_fromWire");
+    EDNS_BADVERS_TEST;
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-
-    // The response must have an EDNS OPT RR in the additional section.
-    // Note that the DNSSEC DO bit is cleared even if this bit in the query
-    // is set.  This is a limitation of the current implementation.
-    headerCheck(*parse_message, default_qid, Rcode::BADVERS(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 1);
-    EXPECT_EQ(4096, parse_message->getUDPSize());
-    EXPECT_FALSE(parse_message->isDNSSECSupported());
 }
 
 TEST_F(RecursorTest, AXFROverUDP) {
-    // AXFR over UDP is invalid and should result in FORMERR.
+    AXFR_OVER_UDP_TEST;
-    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                        Name("example.com"), RRClass::IN(), RRType::AXFR());
-    createRequestPacket(request_message, IPPROTO_UDP);
-    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
-    EXPECT_TRUE(dnsserv.hasAnswer());
-    headerCheck(*parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 0);
 }
 
 TEST_F(RecursorTest, AXFRFail) {
@@ -315,14 +91,191 @@ TEST_F(RecursorTest, notifyFail) {
     // Notify should always return NOTAUTH
     request_message.clear(Message::RENDER);
     request_message.setOpcode(Opcode::NOTIFY());
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setRcode(Rcode::NOERROR());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     request_message.setQid(default_qid);
-    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
     headerCheck(*parse_message, default_qid, Rcode::NOTAUTH(),
                 Opcode::NOTIFY().getCode(), QR_FLAG, 0, 0, 0, 0);
 }
+class RecursorConfig : public ::testing::Test {
+    public:
+        IOService ios;
+        DNSService dnss;
+        Recursor server;
+        RecursorConfig() :
+            dnss(ios, NULL, NULL, NULL)
+        {
+            server.setDNSService(dnss);
+        }
+        void invalidTest(const string &JOSN);
+};
+
+TEST_F(RecursorConfig, forwardAddresses) {
+    // Default value should be fully recursive
+    EXPECT_TRUE(server.getForwardAddresses().empty());
+    EXPECT_FALSE(server.isForwarding());
+
+    // Try putting there some addresses
+    vector<pair<string, uint16_t> > addresses;
+    addresses.push_back(pair<string, uint16_t>(DEFAULT_REMOTE_ADDRESS, 53));
+    addresses.push_back(pair<string, uint16_t>("::1", 53));
+    server.setForwardAddresses(addresses);
+    EXPECT_EQ(2, server.getForwardAddresses().size());
+    EXPECT_EQ("::1", server.getForwardAddresses()[1].first);
+    EXPECT_TRUE(server.isForwarding());
+
+    // Is it independent from what we do with the vector later?
+    addresses.clear();
+    EXPECT_EQ(2, server.getForwardAddresses().size());
+
+    // Did it return to fully recursive?
+    server.setForwardAddresses(addresses);
+    EXPECT_TRUE(server.getForwardAddresses().empty());
+    EXPECT_FALSE(server.isForwarding());
+}
+
+TEST_F(RecursorConfig, forwardAddressConfig) {
+    // Try putting there some address
+    ElementPtr config(Element::fromJSON("{"
+        "\"forward_addresses\": ["
+        "   {"
+        "       \"address\": \"192.0.2.1\","
+        "       \"port\": 53"
+        "   }"
+        "]"
+        "}"));
+    ConstElementPtr result(server.updateConfig(config));
+    EXPECT_EQ(result->toWire(), isc::config::createAnswer()->toWire());
+    EXPECT_TRUE(server.isForwarding());
+    ASSERT_EQ(1, server.getForwardAddresses().size());
+    EXPECT_EQ("192.0.2.1", server.getForwardAddresses()[0].first);
+    EXPECT_EQ(53, server.getForwardAddresses()[0].second);
+
+    // And then remove all addresses
+    config = Element::fromJSON("{"
+        "\"forward_addresses\": null"
+        "}");
+    result = server.updateConfig(config);
+    EXPECT_EQ(result->toWire(), isc::config::createAnswer()->toWire());
+    EXPECT_FALSE(server.isForwarding());
+    EXPECT_EQ(0, server.getForwardAddresses().size());
+}
+
+void RecursorConfig::invalidTest(const string &JOSN) {
+    ElementPtr config(Element::fromJSON(JOSN));
+    EXPECT_FALSE(server.updateConfig(config)->equals(
+        *isc::config::createAnswer())) << "Accepted config " << JOSN << endl;
+}
+
+TEST_F(RecursorConfig, invalidForwardAddresses) {
+    // Try torturing it with some invalid inputs
+    invalidTest("{"
+        "\"forward_addresses\": \"error\""
+        "}");
+    invalidTest("{"
+        "\"forward_addresses\": [{}]"
+        "}");
+    invalidTest("{"
+        "\"forward_addresses\": [{"
+        "   \"port\": 1.5,"
+        "   \"address\": \"192.0.2.1\""
+        "}]}");
+    invalidTest("{"
+        "\"forward_addresses\": [{"
+        "   \"port\": -5,"
+        "   \"address\": \"192.0.2.1\""
+        "}]}");
+    invalidTest("{"
+        "\"forward_addresses\": [{"
+        "   \"port\": 53,"
+        "   \"address\": \"bad_address\""
+        "}]}");
+}
+
+TEST_F(RecursorConfig, listenAddresses) {
+    // Default value should be fully recursive
+    EXPECT_TRUE(server.getListenAddresses().empty());
+
+    // Try putting there some addresses
+    vector<pair<string, uint16_t> > addresses;
+    addresses.push_back(pair<string, uint16_t>("127.0.0.1", 5300));
+    addresses.push_back(pair<string, uint16_t>("::1", 5300));
+    server.setListenAddresses(addresses);
+    EXPECT_EQ(2, server.getListenAddresses().size());
+    EXPECT_EQ("::1", server.getListenAddresses()[1].first);
+
+    // Is it independent from what we do with the vector later?
+    addresses.clear();
+    EXPECT_EQ(2, server.getListenAddresses().size());
+
+    // Did it return to fully recursive?
+    server.setListenAddresses(addresses);
+    EXPECT_TRUE(server.getListenAddresses().empty());
+}
+
+TEST_F(RecursorConfig, DISABLED_listenAddressConfig) {
+    // Try putting there some address
+    ElementPtr config(Element::fromJSON("{"
+        "\"listen_on\": ["
+        "   {"
+        "       \"address\": \"127.0.0.1\","
+        "       \"port\": 5300"
+        "   }"
+        "]"
+        "}"));
+    ConstElementPtr result(server.updateConfig(config));
+    EXPECT_EQ(result->toWire(), isc::config::createAnswer()->toWire());
+    ASSERT_EQ(1, server.getListenAddresses().size());
+    EXPECT_EQ("127.0.0.1", server.getListenAddresses()[0].first);
+    EXPECT_EQ(5300, server.getListenAddresses()[0].second);
+
+    // As this is example address, the machine should not have it on
+    // any interface
+    // FIXME: This test aborts, because it tries to rollback and
+    //     it is impossible, since the sockets are not closed.
+    //     Once #388 is solved, enable this test.
+    config = Element::fromJSON("{"
+        "\"listen_on\": ["
+        "   {"
+        "       \"address\": \"192.0.2.0\","
+        "       \"port\": 5300"
+        "   }"
+        "]"
+        "}");
+    result = server.updateConfig(config);
+    EXPECT_FALSE(result->equals(*isc::config::createAnswer()));
+    ASSERT_EQ(1, server.getListenAddresses().size());
+    EXPECT_EQ("127.0.0.1", server.getListenAddresses()[0].first);
+    EXPECT_EQ(5300, server.getListenAddresses()[0].second);
+}
+
+TEST_F(RecursorConfig, invalidListenAddresses) {
+    // Try torturing it with some invalid inputs
+    invalidTest("{"
+        "\"listen_on\": \"error\""
+        "}");
+    invalidTest("{"
+        "\"listen_on\": [{}]"
+        "}");
+    invalidTest("{"
+        "\"listen_on\": [{"
+        "   \"port\": 1.5,"
+        "   \"address\": \"192.0.2.1\""
+        "}]}");
+    invalidTest("{"
+        "\"listen_on\": [{"
+        "   \"port\": -5,"
+        "   \"address\": \"192.0.2.1\""
+        "}]}");
+    invalidTest("{"
+        "\"listen_on\": [{"
+        "   \"port\": 53,"
+        "   \"address\": \"bad_address\""
+        "}]}");
+}
 
 }

src/bin/recurse/tests/testdata/iqueryresponse_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from iqueryresponse_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Response Opcode=IQUERY(1) Rcode=NOERROR(0)
-1035 c000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/recurse/tests/testdata/multiquestion_fromWire

@@ -1,17 +0,0 @@
-###
-### This data file was auto-generated from multiquestion_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=2, ANCNT=0, NSCNT=0, ARCNT=0
-0002 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001
-
-# Question Section
-# QNAME=example.com. QTYPE=AAAA(28) QCLASS=IN(1)
-076578616d706c6503636f6d00 001c 0001

src/bin/recurse/tests/testdata/queryBadEDNS_fromWire

@@ -1,19 +0,0 @@
-###
-### This data file was auto-generated from queryBadEDNS_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=1
-0001 0000 0000 0001
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001
-
-# EDNS OPT RR
-# NAME=. TYPE=OPT(41) UDPSize=4096 ExtRcode=0 Version=1 DO=1
-00 0029 1000 0001 8000
-# RDLEN=0
-0000

src/bin/recurse/tests/testdata/shortanswer_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from shortanswer_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=1, NSCNT=0, ARCNT=0
-0001 0001 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/recurse/tests/testdata/shortmessage_fromWire

@@ -1,9 +0,0 @@
-###
-### DNS message-like data but doesn't contain sufficient length of data.
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, (ARCNT is missing)
-0001 0000 0000

src/bin/recurse/tests/testdata/shortquestion_fromWire

@@ -1,13 +0,0 @@
-###
-### A query-like data, but missing QCLASS field in the Question section.
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) (QCLASS missing)
-076578616d706c6503636f6d00 0001

src/bin/recurse/tests/testdata/shortresponse_fromWire

@@ -1,13 +0,0 @@
-###
-### A response-like data, but missing QCLASS field in the Question section.
-###
-
-# Header Section
-# ID=4149 QR=Response Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 8000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) (QCLASS is missing)
-076578616d706c6503636f6d00 0001

src/bin/recurse/tests/testdata/simplequery_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from simplequery_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Query Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 0000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/recurse/tests/testdata/simpleresponse_fromWire

@@ -1,13 +0,0 @@
-###
-### This data file was auto-generated from simpleresponse_fromWire.spec
-###
-
-# Header Section
-# ID=4149 QR=Response Opcode=QUERY(0) Rcode=NOERROR(0)
-1035 8000
-# QDCNT=1, ANCNT=0, NSCNT=0, ARCNT=0
-0001 0000 0000 0000
-
-# Question Section
-# QNAME=example.com. QTYPE=A(1) QCLASS=IN(1)
-076578616d706c6503636f6d00 0001 0001

src/bin/stats/Makefile.am

@@ -0,0 +1,37 @@
+SUBDIRS = tests
+
+pkglibexecdir = $(libexecdir)/@PACKAGE@
+
+pkglibexec_SCRIPTS = b10-stats
+noinst_SCRIPTS = b10-stats_stub
+
+b10_statsdir = $(DESTDIR)$(pkgdatadir)
+b10_stats_DATA = stats.spec
+
+CLEANFILES = stats.spec b10-stats stats.pyc stats.pyo b10-stats_stub stats_stub.pyc stats_stub.pyo
+
+man_MANS = b10-stats.8
+EXTRA_DIST = $(man_MANS) b10-stats.xml
+
+if ENABLE_MAN
+
+b10-stats.8: b10-stats.xml
+	xsltproc --novalid --xinclude --nonet -o $@ http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl $(srcdir)/b10-stats.xml
+
+endif
+
+stats.spec: stats.spec.pre
+	$(SED) -e "s|@@LOCALSTATEDIR@@|$(localstatedir)|" stats.spec.pre >$@
+
+# TODO: does this need $$(DESTDIR) also?
+# this is done here since configure.ac AC_OUTPUT doesn't expand exec_prefix
+b10-stats: stats.py
+	$(SED) -e "s|@@PYTHONPATH@@|@pyexecdir@|" \
+	       -e "s|@@LOCALSTATEDIR@@|$(localstatedir)|" \
+	       -e "s|.*#@@REMOVED@@$$||"  stats.py >$@
+	chmod a+x $@
+
+b10-stats_stub: stats_stub.py stats.py
+	$(SED) -e "s|@@PYTHONPATH@@|@pyexecdir@|" \
+	       -e "s|@@LOCALSTATEDIR@@|$(localstatedir)|" stats_stub.py >$@
+	chmod a+x $@

src/bin/stats/b10-stats.8

@@ -0,0 +1,68 @@
+'\" t
+.\"     Title: b10-stats
+.\"    Author: [FIXME: author] [see http://docbook.sf.net/el/author]
+.\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
+.\"      Date: Oct 15, 2010
+.\"    Manual: BIND10
+.\"    Source: BIND10
+.\"  Language: English
+.\"
+.TH "B10\-STATS" "8" "Oct 15, 2010" "BIND10" "BIND10"
+.\" -----------------------------------------------------------------
+.\" * set default formatting
+.\" -----------------------------------------------------------------
+.\" disable hyphenation
+.nh
+.\" disable justification (adjust text to left margin only)
+.ad l
+.\" -----------------------------------------------------------------
+.\" * MAIN CONTENT STARTS HERE *
+.\" -----------------------------------------------------------------
+.SH "NAME"
+b10-stats \- BIND 10 statistics module
+.SH "SYNOPSIS"
+.HP \w'\fBb10\-stats\fR\ 'u
+\fBb10\-stats\fR [\fB\-v\fR] [\fB\-\-verbose\fR]
+.SH "DESCRIPTION"
+.PP
+The
+\fBb10\-stats\fR
+is a daemon forked by
+\fBbind10\fR\&. Stats module collects statistics data from each module and reports statistics information via
+\fBbindctl\fR\&. It communicates by using the Command Channel by
+\fBb10\-msgq\fR
+with other modules like
+\fBbind10\fR,
+\fBb10\-auth\fR
+and so on\&. It waits for coming data from other modules, then other modules send data to stats module periodically\&. Other modules send stats data to stats module independently from implementation of stats module, so the frequency of sending data may not be constant\&. Stats module collects data and aggregates it\&.
+.SH "OPTIONS"
+.PP
+The arguments are as follows:
+.PP
+\fB\-v\fR, \fB\-\-verbose\fR
+.RS 4
+This
+\fBb10\-stats\fR
+switches to verbose mode\&. It sends verbose messages to STDOUT\&.
+.RE
+.SH "FILES"
+.PP
+/usr/local/share/bind10\-devel/stats\&.spec
+\(em This is a spec file for
+\fBb10\-stats\fR\&. It contains definitions of statistics items of BIND 10 and commands received vi bindctl\&.
+.SH "SEE ALSO"
+.PP
+
+\fBbind10\fR(8),
+\fBbindctl\fR(1),
+\fBb10-auth\fR(8),
+BIND 10 Guide\&.
+.SH "HISTORY"
+.PP
+The
+\fBb10\-stats\fR
+daemon was initially designed and implemented by Naoki Kambe of JPRS in Oct 2010\&.
+.SH "COPYRIGHT"
+.br
+Copyright \(co 2010 Internet Systems Consortium, Inc. ("ISC")
+.br

src/bin/stats/b10-stats.xml

@@ -0,0 +1,124 @@
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+               "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+	       [<!ENTITY mdash "&#8212;">]>
+<!--
+ - Copyright (C) 2010  Internet Systems Consortium, Inc. ("ISC")
+ -
+ - Permission to use, copy, modify, and/or distribute this software for any
+ - purpose with or without fee is hereby granted, provided that the above
+ - copyright notice and this permission notice appear in all copies.
+ -
+ - THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+ - REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+ - AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+ - INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+ - LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+ - OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+ - PERFORMANCE OF THIS SOFTWARE.
+-->
+
+<!-- $Id$ -->
+<refentry>
+
+  <refentryinfo>
+    <date>Oct 15, 2010</date>
+  </refentryinfo>
+
+  <refmeta>
+    <refentrytitle>b10-stats</refentrytitle>
+    <manvolnum>8</manvolnum>
+    <refmiscinfo>BIND10</refmiscinfo>
+  </refmeta>
+
+  <refnamediv>
+    <refname>b10-stats</refname>
+    <refpurpose>BIND 10 statistics module</refpurpose>
+  </refnamediv>
+
+  <docinfo>
+    <copyright>
+      <year>2010</year>
+      <holder>Internet Systems Consortium, Inc. ("ISC")</holder>
+    </copyright>
+  </docinfo>
+
+  <refsynopsisdiv>
+    <cmdsynopsis>
+      <command>b10-stats</command>
+      <arg><option>-v</option></arg>
+      <arg><option>--verbose</option></arg>
+    </cmdsynopsis>
+  </refsynopsisdiv>
+
+  <refsect1>
+    <title>DESCRIPTION</title>
+    <para>
+      The <command>b10-stats</command> is a daemon forked by
+      <command>bind10</command>. Stats module collects statistics data
+      from each module and reports statistics information
+      via <command>bindctl</command>.  It communicates by using the
+      Command Channel by <command>b10-msgq</command> with other
+      modules
+      like <command>bind10</command>, <command>b10-auth</command> and
+      so on. It waits for coming data from other modules, then other
+      modules send data to stats module periodically. Other modules
+      send stats data to stats module independently from
+      implementation of stats module, so the frequency of sending data
+      may not be constant. Stats module collects data and aggregates
+      it.
+    </para>
+  </refsect1>
+
+  <refsect1>
+    <title>OPTIONS</title>
+    <para>The arguments are as follows:</para>
+    <variablelist>
+      <varlistentry>
+        <term><option>-v</option>, <option>--verbose</option></term>
+        <listitem>
+	  <para>
+          This <command>b10-stats</command> switches to verbose
+          mode. It sends verbose messages to STDOUT.
+	  </para>
+        </listitem>
+      </varlistentry>
+    </variablelist>
+  </refsect1>
+
+  <refsect1>
+    <title>FILES</title>
+    <para><filename>/usr/local/share/bind10-devel/stats.spec</filename>
+      &mdash; This is a spec file for <command>b10-stats</command>. It
+      contains definitions of statistics items of BIND 10 and commands
+      received vi bindctl.
+    </para>
+  </refsect1>
+
+  <refsect1>
+    <title>SEE ALSO</title>
+    <para>
+      <citerefentry>
+        <refentrytitle>bind10</refentrytitle><manvolnum>8</manvolnum>
+      </citerefentry>,
+      <citerefentry>
+        <refentrytitle>bindctl</refentrytitle><manvolnum>1</manvolnum>
+      </citerefentry>,
+      <citerefentry>
+        <refentrytitle>b10-auth</refentrytitle><manvolnum>8</manvolnum>
+      </citerefentry>,
+      <citetitle>BIND 10 Guide</citetitle>.
+    </para>
+  </refsect1>
+
+  <refsect1>
+    <title>HISTORY</title>
+    <para>
+      The <command>b10-stats</command> daemon was initially designed
+      and implemented by Naoki Kambe of JPRS in Oct 2010.
+    </para>
+  </refsect1>
+</refentry><!--
+ - Local variables:
+ - mode: sgml
+ - End:
+-->

src/bin/stats/run_b10-stats.sh.in

@@ -0,0 +1,30 @@
+#! /bin/sh
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
+export PYTHON_EXEC
+
+PYTHONPATH=@abs_top_builddir@/src/lib/python
+export PYTHONPATH
+
+B10_FROM_BUILD=@abs_top_builddir@
+export B10_FROM_BUILD
+
+STATS_PATH=@abs_top_builddir@/src/bin/stats
+
+cd ${STATS_PATH}
+exec ${PYTHON_EXEC} -O b10-stats $*

src/bin/stats/run_b10-stats_stub.sh.in

@@ -0,0 +1,30 @@
+#! /bin/sh
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
+export PYTHON_EXEC
+
+PYTHONPATH=@abs_top_builddir@/src/lib/python
+export PYTHONPATH
+
+B10_FROM_BUILD=@abs_top_srcdir@
+export B10_FROM_BUILD
+
+STATS_PATH=@abs_top_builddir@/src/bin/stats
+
+cd ${STATS_PATH}
+exec ${PYTHON_EXEC} -O b10-stats_stub $*

src/bin/stats/stats.py.in

@@ -0,0 +1,416 @@
+#!@PYTHON@
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+__version__ = "$Revision$"
+
+import sys; sys.path.append ('@@PYTHONPATH@@')
+import os
+import signal
+import select
+from time import time, strftime, gmtime
+from optparse import OptionParser, OptionValueError
+from collections import defaultdict
+from isc.config.ccsession import ModuleCCSession, create_answer
+from isc.cc import Session, SessionError
+# Note: Following lines are removed in b10-stats	#@@REMOVED@@
+if __name__ == 'stats':					#@@REMOVED@@
+    try:						#@@REMOVED@@
+        from fake_time import time, strftime, gmtime	#@@REMOVED@@
+    except ImportError:					#@@REMOVED@@
+        pass						#@@REMOVED@@
+
+# for setproctitle
+import isc.util.process
+isc.util.process.rename()
+
+# If B10_FROM_BUILD is set in the environment, we use data files
+# from a directory relative to that, otherwise we use the ones
+# installed on the system
+if "B10_FROM_BUILD" in os.environ:
+    SPECFILE_LOCATION = os.environ["B10_FROM_BUILD"] + "/src/bin/stats/stats.spec"
+else:
+    PREFIX = "@prefix@"
+    DATAROOTDIR = "@datarootdir@"
+    SPECFILE_LOCATION = "@datadir@/@PACKAGE@/stats.spec".replace("${datarootdir}", DATAROOTDIR).replace("${prefix}", PREFIX)
+
+class Singleton(type):
+    """
+    A abstract class of singleton pattern
+    """
+    # Because of singleton pattern: 
+    #   At the beginning of coding, one UNIX domain socket is needed
+    #  for config manager, another socket is needed for stats module,
+    #  then stats module might need two sockets. So I adopted the
+    #  singleton pattern because I avoid creating multiple sockets in
+    #  one stats module. But in the initial version stats module
+    #  reports only via bindctl, so just one socket is needed. To use
+    #  the singleton pattern is not important now. :(
+
+    def __init__(self, *args, **kwargs):
+        type.__init__(self, *args, **kwargs)
+        self._instances = {}
+
+    def __call__(self, *args, **kwargs):
+        if args not in self._instances:
+            self._instances[args]={}
+        kw = tuple(kwargs.items())
+        if  kw not in self._instances[args]:
+            self._instances[args][kw] = type.__call__(self, *args, **kwargs)
+        return self._instances[args][kw]
+
+class Callback():
+    """
+    A Callback handler class
+    """
+    def __init__(self, name=None, callback=None, args=(), kwargs={}):
+        self.name = name
+        self.callback = callback
+        self.args = args
+        self.kwargs = kwargs
+
+    def __call__(self, *args, **kwargs):
+        if not args:
+            args = self.args
+        if not kwargs:
+            kwargs = self.kwargs
+        if self.callback:
+            return self.callback(*args, **kwargs)
+
+class Subject():
+    """
+    A abstract subject class of observer pattern
+    """
+    # Because of observer pattern:
+    #   In the initial release, I'm also sure that observer pattern
+    #  isn't definitely needed because the interface between gathering
+    #  and reporting statistics data is single.  However in the future
+    #  release, the interfaces may be multiple, that is, multiple
+    #  listeners may be needed. For example, one interface, which
+    #  stats module has, is for between ''config manager'' and stats
+    #  module, another interface is for between ''HTTP server'' and
+    #  stats module, and one more interface is for between ''SNMP
+    #  server'' and stats module. So by considering that stats module
+    #  needs multiple interfaces in the future release, I adopted the
+    #  observer pattern in stats module. But I don't have concrete
+    #  ideas in case of multiple listener currently.
+
+    def __init__(self):
+        self._listeners = []
+
+    def attach(self, listener):
+        if not listener in self._listeners:
+            self._listeners.append(listener)
+
+    def detach(self, listener):
+        try:
+            self._listeners.remove(listener)
+        except ValueError:
+            pass
+
+    def notify(self, event, modifier=None):
+        for listener in self._listeners:
+            if modifier != listener:
+                listener.update(event)
+
+class Listener():
+    """
+    A abstract listener class of observer pattern
+    """
+    def __init__(self, subject):
+        self.subject = subject
+        self.subject.attach(self)
+        self.events = {}
+
+    def update(self, name):
+        if name in self.events:
+            callback = self.events[name]
+            return callback()
+
+    def add_event(self, event):
+        self.events[event.name]=event
+
+class SessionSubject(Subject, metaclass=Singleton):
+    """
+    A concrete subject class which creates CC session object
+    """
+    def __init__(self, session=None, verbose=False):
+        Subject.__init__(self)
+        self.verbose = verbose
+        self.session=session
+        self.running = False
+
+    def start(self):
+        self.running = True
+        self.notify('start')
+
+    def stop(self):
+        self.running = False
+        self.notify('stop')
+
+    def check(self):
+        self.notify('check')
+
+class CCSessionListener(Listener):
+    """
+    A concrete listener class which creates SessionSubject object and
+    ModuleCCSession object
+    """
+    def __init__(self, subject, verbose=False):
+        Listener.__init__(self, subject)
+        self.verbose = verbose
+        self.session = subject.session
+        self.boot_time = get_datetime()
+
+        # create ModuleCCSession object
+        self.cc_session = ModuleCCSession(SPECFILE_LOCATION,
+                                          self.config_handler,
+                                          self.command_handler,
+                                          self.session)
+
+        self.session = self.subject.session = self.cc_session._session
+
+        # initialize internal data
+        self.config_spec = self.cc_session.get_module_spec().get_config_spec()
+        self.stats_spec = self.config_spec
+        self.stats_data = self.initialize_data(self.stats_spec)
+
+        # add event handler invoked via SessionSubject object
+        self.add_event(Callback('start', self.start))
+        self.add_event(Callback('stop', self.stop))
+        self.add_event(Callback('check', self.check))
+        # don't add 'command_' suffix to the special commands in
+        # order to prevent executing internal command via bindctl
+
+        # get commands spec
+        self.commands_spec = self.cc_session.get_module_spec().get_commands_spec()
+
+        # add event handler related command_handler of ModuleCCSession
+        # invoked via bindctl
+        for cmd in self.commands_spec:
+            try:
+                # add prefix "command_"
+                name = "command_" + cmd["command_name"]
+                callback = getattr(self, name)
+                kwargs = self.initialize_data(cmd["command_args"])
+                self.add_event(Callback(name=name, callback=callback, args=(), kwargs=kwargs))
+            except AttributeError as ae:
+                sys.stderr.write("[b10-stats] Caught undefined command while parsing spec file: "
+                                 +str(cmd["command_name"])+"\n")
+
+    def start(self):
+        """
+        start the cc chanel
+        """
+        # set initial value
+        self.stats_data['stats.boot_time'] = self.boot_time
+        self.stats_data['stats.start_time'] = get_datetime()
+        self.stats_data['stats.last_update_time'] = get_datetime()
+        self.stats_data['stats.lname'] = self.session.lname
+        return self.cc_session.start()
+
+    def stop(self):
+        """
+        stop the cc chanel
+        """
+        return self.cc_session.close()
+
+    def check(self):
+        """
+        check the cc chanel
+        """
+        return self.cc_session.check_command(False)
+
+    def config_handler(self, new_config):
+        """
+        handle a configure from the cc channel
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] newconfig received: "+str(new_config)+"\n")
+
+        # do nothing currently
+        return create_answer(0)
+
+    def command_handler(self, command, *args, **kwargs):
+        """
+        handle commands from the cc channel
+        """
+        # add 'command_' suffix in order to executing command via bindctl
+        name = 'command_' + command
+        
+        if name in self.events:
+            event = self.events[name]
+            return event(*args, **kwargs)
+        else:
+            return self.command_unknown(command, args)
+
+    def command_shutdown(self, args):
+        """
+        handle shutdown command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] 'shutdown' command received\n")
+        self.subject.running = False
+        return create_answer(0)
+
+    def command_set(self, args, stats_data={}):
+        """
+        handle set command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] 'set' command received, args: "+str(args)+"\n")
+
+        # 'args' must be dictionary type
+        self.stats_data.update(args['stats_data'])
+
+        # overwrite "stats.LastUpdateTime"
+        self.stats_data['stats.last_update_time'] = get_datetime()
+
+        return create_answer(0)
+
+    def command_remove(self, args, stats_item_name=''):
+        """
+        handle remove command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] 'remove' command received, args: "+str(args)+"\n")
+
+        # 'args' must be dictionary type
+        if args and args['stats_item_name'] in self.stats_data:
+            stats_item_name = args['stats_item_name']
+
+        # just remove one item
+        self.stats_data.pop(stats_item_name)
+
+        return create_answer(0)
+
+    def command_show(self, args, stats_item_name=''):
+        """
+        handle show command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] 'show' command received, args: "+str(args)+"\n")
+
+        # always overwrite 'report_time' and 'stats.timestamp'
+        # if "show" command invoked
+        self.stats_data['report_time'] = get_datetime()
+        self.stats_data['stats.timestamp'] = get_timestamp()
+
+        # if with args
+        if args and args['stats_item_name'] in self.stats_data:
+            stats_item_name = args['stats_item_name']
+            return create_answer(0, {stats_item_name: self.stats_data[stats_item_name]})
+
+        return create_answer(0, self.stats_data)
+
+    def command_reset(self, args):
+        """
+        handle reset command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] 'reset' command received\n")
+
+        # re-initialize internal variables
+        self.stats_data = self.initialize_data(self.stats_spec)
+
+        # reset initial value
+        self.stats_data['stats.boot_time'] = self.boot_time
+        self.stats_data['stats.start_time'] = get_datetime()
+        self.stats_data['stats.last_update_time'] = get_datetime()
+        self.stats_data['stats.lname'] = self.session.lname
+
+        return create_answer(0)
+
+    def command_status(self, args):
+        """
+        handle status command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] 'status' command received\n")
+        # just return "I'm alive."
+        return create_answer(0, "I'm alive.")
+
+    def command_unknown(self, command, args):
+        """
+        handle an unknown command
+        """
+        if self.verbose:
+            sys.stdout.write("[b10-stats] Unknown command received: '"
+                             + str(command) + "'\n")
+        return create_answer(1, "Unknown command: '"+str(command)+"'")
+
+
+    def initialize_data(self, spec):
+        """
+        initialize stats data
+        """
+        def __get_init_val(spec):
+            if spec['item_type'] == 'null':
+                return None
+            elif spec['item_type'] == 'boolean':
+                return bool(spec.get('item_default', False))
+            elif spec['item_type'] == 'string':
+                return str(spec.get('item_default', ''))
+            elif spec['item_type'] in set(['number', 'integer']):
+                return int(spec.get('item_default', 0))
+            elif spec['item_type'] in set(['float', 'double', 'real']):
+                return float(spec.get('item_default', 0.0))
+            elif spec['item_type'] in set(['list', 'array']):
+                return spec.get('item_default',
+                                [ __get_init_val(s) for s in spec['list_item_spec'] ])
+            elif spec['item_type'] in set(['map', 'object']):
+                return spec.get('item_default',
+                                dict([ (s['item_name'], __get_init_val(s)) for s in spec['map_item_spec'] ]) )
+            else:
+                return spec.get('item_default')
+        return dict([ (s['item_name'], __get_init_val(s)) for s in spec ])
+
+def get_timestamp():
+    """
+    get current timestamp
+    """
+    return time()
+
+def get_datetime():
+    """
+    get current datetime
+    """
+    return strftime("%Y-%m-%dT%H:%M:%SZ", gmtime())
+
+def main(session=None):
+    try:
+        parser = OptionParser()
+        parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
+                      help="display more about what is going on")
+        (options, args) = parser.parse_args()
+        subject = SessionSubject(session=session, verbose=options.verbose)
+        listener = CCSessionListener(subject, verbose=options.verbose)
+        subject.start()
+        while subject.running:
+            subject.check()
+        subject.stop()
+
+    except OptionValueError:
+        sys.stderr.write("[b10-stats] Error parsing options\n")
+    except SessionError as se:
+        sys.stderr.write("[b10-stats] Error creating Stats module, "
+              + "is the command channel daemon running?\n")
+    except KeyboardInterrupt as kie:
+        sys.stderr.write("[b10-stats] Interrupted, exiting\n")
+
+if __name__ == "__main__":
+    main()

src/bin/stats/stats.spec.pre.in

@@ -0,0 +1,140 @@
+{
+  "module_spec": {
+    "module_name": "Stats",
+    "module_description": "Stats daemon",
+    "config_data": [
+      {
+        "item_name": "report_time",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "1970-01-01T00:00:00Z",
+        "item_title": "Report time",
+        "item_description": "A date time when stats module reports",
+        "item_format": "date-time"
+      },
+      {
+        "item_name": "bind10.boot_time",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "1970-01-01T00:00:00Z",
+        "item_title": "stats.BootTime",
+        "item_description": "A date time when bind10 process starts initially",
+        "item_format": "date-time"
+      },
+      {
+        "item_name": "stats.boot_time",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "1970-01-01T00:00:00Z",
+        "item_title": "stats.BootTime",
+        "item_description": "A date time when the stats module starts initially or when the stats module restarts",
+        "item_format": "date-time"
+      },
+      {
+        "item_name": "stats.start_time",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "1970-01-01T00:00:00Z",
+        "item_title": "stats.StartTime",
+        "item_description": "A date time when the stats module starts collecting data or resetting values last time",
+        "item_format": "date-time"
+      },
+      {
+        "item_name": "stats.last_update_time",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "1970-01-01T00:00:00Z",
+        "item_title": "stats.LastUpdateTime",
+        "item_description": "The latest date time when the stats module receives from other modules like auth server or boss process and so on",
+        "item_format": "date-time"
+      },
+      {
+        "item_name": "stats.timestamp",
+        "item_type": "real",
+        "item_optional": false,
+        "item_default": 0.0,
+        "item_title": "stats.Timestamp",
+        "item_description": "A current time stamp since epoch time (1970-01-01T00:00:00Z)",
+        "item_format": "second"
+      },
+      {
+        "item_name": "stats.lname",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "",
+        "item_title": "stats.LocalName",
+        "item_description": "A localname of stats module given via CC protocol"
+      },
+      {
+        "item_name": "auth.queries.tcp",
+        "item_type": "integer",
+        "item_optional": false,
+        "item_default": 0,
+        "item_title": "auth.queries.tcp",
+        "item_description": "A number of total query counts which all auth servers receive over TCP since they started initially"
+      },
+      {
+        "item_name": "auth.queries.udp",
+        "item_type": "integer",
+        "item_optional": false,
+        "item_default": 0,
+        "item_title": "auth.queries.udp",
+        "item_description": "A number of total query counts which all auth servers receive over UDP since they started initially"
+      }
+    ],
+    "commands": [
+      {
+        "command_name": "status",
+        "command_description": "identify whether stats module is alive or not",
+        "command_args": []
+      },
+      {
+        "command_name": "show",
+        "command_description": "show the specified/all statistics data",
+        "command_args": [
+          {
+            "item_name": "stats_item_name",
+            "item_type": "string",
+            "item_optional": true,
+            "item_default": ""
+          }
+        ]
+      },
+      {
+        "command_name": "set",
+        "command_description": "set the value of specified name in statistics data",
+        "command_args": [
+          {
+            "item_name": "stats_data",
+            "item_type": "map",
+            "item_optional": false,
+            "item_default": {},
+            "map_item_spec": []
+          }
+        ]
+      },
+      {
+        "command_name": "remove",
+        "command_description": "remove the specified name from statistics data",
+        "command_args": [
+          {
+            "item_name": "stats_item_name",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": ""
+          }
+        ]
+      },
+      {
+        "command_name": "reset",
+        "command_description": "reset all statistics data to default values except for several constant names",
+        "command_args": []
+      },
+      {
+        "command_name": "shutdown",
+        "command_description": "Shut down the stats module",
+        "command_args": []
+      }
+    ]
+  }
+}

src/bin/stats/stats_stub.py.in

@@ -0,0 +1,155 @@
+#!@PYTHON@
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+__version__ = "$Revision$"
+
+import sys; sys.path.append ('@@PYTHONPATH@@')
+import os
+import time
+from optparse import OptionParser, OptionValueError
+from isc.config.ccsession import ModuleCCSession, create_command, parse_answer, parse_command, create_answer
+from isc.cc import Session, SessionError
+from stats import get_datetime
+
+# for setproctitle
+import isc.util.process
+isc.util.process.rename()
+
+# If B10_FROM_BUILD is set in the environment, we use data files
+# from a directory relative to that, otherwise we use the ones
+# installed on the system
+if "B10_FROM_BUILD" in os.environ:
+    SPECFILE_LOCATION = os.environ["B10_FROM_BUILD"] + "/src/bin/stats/stats.spec"
+else:
+    PREFIX = "@prefix@"
+    DATAROOTDIR = "@datarootdir@"
+    SPECFILE_LOCATION = "@datadir@/@PACKAGE@/stats.spec".replace("${datarootdir}", DATAROOTDIR).replace("${prefix}", PREFIX)
+
+class CCSessionStub:
+    """
+    This class is intended to behaves as a sender to Stats module. It
+    creates MoudleCCSession object and send specified command.
+    """
+    def __init__(self, session=None, verbose=False):
+        # create ModuleCCSession object
+        self.verbose = verbose
+        self.cc_session = ModuleCCSession(SPECFILE_LOCATION,
+                                          self.__dummy, self.__dummy, session)
+        self.module_name = self.cc_session._module_name
+        self.session = self.cc_session._session
+
+    def __dummy(self, *args):
+        pass
+
+    def send_command(self, command, args):
+        """
+        send command to stats module with args
+        """
+        cmd = create_command(command, args)
+        if self.verbose:
+            sys.stdout.write("[b10-stats_stub] send command : " + str(cmd) + "\n")
+        seq = self.session.group_sendmsg(cmd, self.module_name)
+        msg, env = self.session.group_recvmsg(False, seq) # non-blocking is False
+        if self.verbose:
+            sys.stdout.write("[b10-stats_stub] received env : " + str(env) + "\n")
+            sys.stdout.write("[b10-stats_stub] received message : " + str(msg) + "\n")
+        (ret, arg) = (None, None)
+        if 'result' in msg:
+            ret, arg = parse_answer(msg)
+        elif 'command' in msg:
+            ret, arg = parse_command(msg)
+        self.session.group_reply(env, create_answer(0))
+        return ret, arg, env
+        
+class BossModuleStub:
+    """
+    This class is customized from CCSessionStub and is intended to behaves
+    as a virtual Boss module to send to Stats Module.
+    """
+    def __init__(self, session=None, verbose=False):
+        self.stub = CCSessionStub(session=session, verbose=verbose)
+    
+    def send_boottime(self):
+        return self.stub.send_command("set", {"stats_data": {"bind10.boot_time": get_datetime()}})
+
+class AuthModuleStub:
+    """
+    This class is customized CCSessionStub and is intended to behaves
+    as a virtual Auth module to send to Stats Module.
+    """
+    def __init__(self, session=None, verbose=False):
+        self.stub = CCSessionStub(session=session, verbose=verbose)
+        self.count = { "udp": 0, "tcp": 0 }
+    
+    def send_udp_query_count(self, cmd="set", cnt=0):
+        """
+        count up udp query count
+        """
+        prt = "udp"
+        self.count[prt] = 1
+        if cnt > 0:
+            self.count[prt] = cnt
+        return self.stub.send_command(cmd,
+                                      {"stats_data":
+                                           {"auth.queries."+prt: self.count[prt]}
+                                       })
+
+    def send_tcp_query_count(self, cmd="set", cnt=0):
+        """
+        set udp query count
+        """
+        prt = "tcp"
+        self.count[prt] = self.count[prt] + 1
+        if cnt > 0:
+            self.count[prt] = cnt
+        return self.stub.send_command(cmd,
+                                      {"stats_data":
+                                           {"auth.queries."+prt: self.count[prt]}
+                                       })
+
+def main(session=None):
+    try:
+        parser=OptionParser()
+        parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
+                      help="display more about what is going on")
+        (options, args) = parser.parse_args()
+        stub = CCSessionStub(session=session, verbose=options.verbose)
+        boss = BossModuleStub(session=stub.session, verbose=options.verbose)
+        auth = AuthModuleStub(session=stub.session, verbose=options.verbose)
+        stub.send_command("status", None)
+        boss.send_boottime()
+        t_cnt=0
+        u_cnt=81120
+        auth.send_udp_query_count(cnt=u_cnt) # This is an example.
+        while True:
+            u_cnt = u_cnt + 1
+            t_cnt = t_cnt + 1
+            auth.send_udp_query_count(cnt=u_cnt)
+            auth.send_tcp_query_count(cnt=t_cnt)
+            time.sleep(1)
+
+    except OptionValueError:
+        sys.stderr.write("[b10-stats_stub] Error parsing options\n")
+    except SessionError as se:
+        sys.stderr.write("[b10-stats_stub] Error creating Stats module, "
+              + "is the command channel daemon running?\n")
+    except KeyboardInterrupt as kie:
+        sys.stderr.write("[b10-stats_stub] Interrupted, exiting\n")
+
+if __name__ == "__main__":
+    main()

src/bin/stats/statsd.py

@@ -1,151 +0,0 @@
-#!/usr/bin/python
-#
-# This program collects 'counters' from 'statistics' channel.
-# It accepts one command: 'Boss' group 'shutdown'
-
-import isc.cc
-import time
-import select
-import os
-
-bossgroup = 'Boss'
-myname = 'statsd'
-debug = 0
-
-def total(s):
-    def totalsub(d,s):
-        for k in s.keys():
-            if (k == 'component' or k == 'version' 
-                or k == 'timestamp' or k == 'from'):
-                continue
-            if (k in d):
-                if (isinstance(s[k], dict)):
-                    totalsub(d[k], s[k])
-                else:
-                    d[k] = s[k] + d[k]
-            else:
-                d[k] = s[k]
-
-    if (len(s) == 0):
-        return {}
-    if (len(s) == 1):
-        for k in s.keys():
-            out = s[k]
-        out['components'] = 1
-        out['timestamp2'] = out['timestamp']
-        del out['from']
-        return out
-    _time1 = 0
-    _time2 = 0
-    out = {}
-    for i in s.values():
-        if (_time1 == 0 or _time1 < i['timestamp']):
-            _time1 = i['timestamp']
-        if (_time2 == 0 or _time2 > i['timestamp']):
-            _time2 = i['timestamp']
-        totalsub(out, i)
-    out['components'] = len(s)
-    out['timestamp'] = _time1;
-    out['timestamp2'] = _time2;
-    return out
-
-def dicttoxml(stats, level = 0):
-    def dicttoxmlsub(s, level):
-        output = ''
-        spaces = ' ' * level
-        for k in s.keys():
-            if (isinstance(s[k], dict)):
-                output += spaces + ('<%s>\n' %k) \
-                  + dicttoxmlsub(s[k], level+1) \
-                  + spaces + '</%s>\n' %k
-            else:
-                output += spaces + '<%s>%s</%s>\n' % (k, s[k], k)
-        return output
-
-    for k in stats.keys():
-        space = ' ' * level
-        output = space + '<component component="%s">\n' % k
-        s = stats[k]
-        if ('component' in s or 'components' in s):
-            output += dicttoxmlsub(s, level+1)
-        else:
-            for l in s.keys():
-                output +=  space + ' <from from="%s">\n' % l \
-                          + dicttoxmlsub(s[l], level+2) \
-                          + space + ' </from>\n'
-        output += space + '</component>\n'
-        return output
-
-def dump_stats(statpath, statcount, stat, statraw):
-    newfile = open(statpath + '.new', 'w')
-    newfile.write('<?xml version="1.0" encoding="UTF-8"?>\n')
-    newfile.write('<!-- created at '+time.strftime('%Y%m%d %H%M%S')+' -->\n')
-    newfile.write('<isc version="0.0">\n')
-    newfile.write(' <bind10>\n')
-    newfile.write('  <total>\n')
-    newfile.write(dicttoxml(stat, 3))
-    newfile.write('  </total>\n')
-    newfile.write('  <each>\n')
-    newfile.write(dicttoxml(statraw, 3))
-    newfile.write('  </each>\n')
-    newfile.write(' </bind10>\n')
-    newfile.write('</isc>\n')
-    newfile.close()
-    loop = statcount
-    while(loop > 0):
-        old = statpath + '.%d' % loop
-        loop -= 1
-        new = statpath + '.%d' % loop
-        if (os.access(new, os.F_OK)):
-            os.rename(new, old)
-    if (os.access(statpath, os.F_OK)):
-        os.rename(statpath, new)
-    os.rename(statpath + '.new', statpath)
-
-def collector(statgroup,step,statpath,statcount):
-    cc = isc.cc.Session()
-    if debug:
-        print ("cc.lname=",cc.lname)
-    cc.group_subscribe(statgroup)
-    cc.group_subscribe(bossgroup, myname)
-    wrote_time = -1
-    last_wrote_time = -1
-    last_recvd_time = -1
-    stats = {}
-    statstotal = {}
-    while 1:
-        wait = wrote_time + step - time.time()
-        if wait <= 0 and last_recvd_time > wrote_time:
-            if debug:
-                print ("dump stats")
-            dump_stats(statpath, statcount, statstotal, stats)
-            last_wrote_time = wrote_time;
-            wrote_time = time.time();
-            wait = last_wrote_time + step - time.time()
-            if wait < 0:
-                wait = step
-        r,w,e = select.select([cc._socket],[],[], wait)
-        for sock in r:
-            if sock == cc._socket:
-                data,envelope = cc.group_recvmsg(False)
-                if (envelope['group'] == bossgroup):
-                    if ('shutdown' in data):
-                        exit()
-                if (envelope['group'] == statgroup):
-                    # Check received data
-                    if (not('component' in data and 'version' in data
-                        and 'stats' in data)):
-                        continue
-                    component = data['component']
-                    _from = envelope['from']
-                    data['from'] = _from
-                    if debug:
-                        print ("received from ",_from)
-                    if (not (component in stats)):
-                        stats[component] = {}
-                    (stats[component])[_from] = data;
-                    statstotal[component] = total(stats[component])
-                    last_recvd_time = time.time()
-
-if __name__ == '__main__':
-    collector('statistics', 10, '/tmp/stats.xml', 100)

src/bin/stats/statsd.txt

@@ -1,55 +0,0 @@
-= Statistics overview =
-
-Result of 26 Jan 2010 evening discussion,
-statistics overview was almost fixed.
-
-Statsd listens msgq "statistics" channel, gathers statistics
-from each BIND 10 components and dump them into a XML file periodically.
-
-= Statsd current status =
-
-Statsd can run with msgq.
-Statsd is not controlled by BoB.
-Statsd does not read configuration from cfgd.
-File path, dump frequency, rotate generations are fixed.
-Statsd dumps to "/tmp/stats" every 10 seconds except no statistics received.
-"/tmp/stats" are preserved 100 generations.
-
-Current implementation is put on "bind10/branches/parkinglot/src/bin/stats/".
-
-= statistics channel Message format =
-
-The Statsd accepts python dictionary format data from msgq
-"statistics" channel.
-
-The data need to contain "components", "version", "timestamp", "stats" keys.
-
-The statistics data format is { "component" : "<component_name>",
-"version": "<version number>", "timestamp": "<unixtime>", "stats":
-<python dictionary format statistics>}.
-
-"stats" data may be nested.
-"stats" data is defined by each component.
-
-Each component sends statistics data to "statistics" group periodically
-without joining the group.
-
-See a example component: "stats/test/test-agent.py".
-
-= How to publish statistics from each component =
-
-For example, parkinglot auth server has one "counter".
-Then, parkinglot's statistics message may be
- { "component":"parkinglot", "version":1, "timestamp":unixtime,
-   stats: { "counter": counter } }.
-Send it to msgq "statistics" channel periodically
-(For example, every 10 second).
-
-Then "Statsd" will write it to the statistics file periodically.
-
-= TODO =
-
-- statsd.spec
-- read configuration from cfgd.
-- how to publish statistics data
-- controlled by BoB

src/bin/stats/test/shutdown.py

@@ -1,6 +0,0 @@
-#!/usr/bin/python
-
-import isc
-cc = isc.cc.Session()
-cc.group_subscribe("Boss")
-cc.group_sendmsg({ "command":"shutdown"},"Boss")

src/bin/stats/test/test_agent.py

@@ -1,178 +0,0 @@
-#!/usr/bin/python
-
-# This program acts statistics agent.
-# It has pseudo counters which is incremented each 10 second and
-# sends data to "statistics" channel periodically.
-# One command is available
-#   "Boss"       group: "shutdown"
-
-import isc
-import time
-import select
-import random
-
-step_time = 10
-statgroup = "statistics"
-
-cc = isc.cc.Session()
-print (cc.lname)
-#cc.group_subscribe(statgroup)
-cc.group_subscribe("Boss")
-
-# counters
-
-NSSTATDESC={}
-NSSTATDESC["counterid"] = 0
-NSSTATDESC["requestv4"] = 0
-NSSTATDESC["requestv6"] = 0
-NSSTATDESC["edns0in"] = 0
-NSSTATDESC["badednsver"] = 0
-NSSTATDESC["tsigin"] = 0
-NSSTATDESC["sig0in"] = 0
-NSSTATDESC["invalidsig"] = 0
-NSSTATDESC["tcp"] = 0
-NSSTATDESC["authrej"] = 0
-NSSTATDESC["recurserej"] = 0
-NSSTATDESC["xfrrej"] = 0
-NSSTATDESC["updaterej"] = 0
-NSSTATDESC["response"] = 0
-NSSTATDESC["truncatedresp"] = 0
-NSSTATDESC["edns0out"] = 0
-NSSTATDESC["tsigout"] = 0
-NSSTATDESC["sig0out"] = 0
-NSSTATDESC["success"] = 0
-NSSTATDESC["authans"] = 0
-NSSTATDESC["nonauthans"] = 0
-NSSTATDESC["referral"] = 0
-NSSTATDESC["nxrrset"] = 0
-NSSTATDESC["servfail"] = 0
-NSSTATDESC["formerr"] = 0
-NSSTATDESC["nxdomain"] = 0
-NSSTATDESC["recursion"] = 0
-NSSTATDESC["duplicate"] = 0
-NSSTATDESC["dropped"] = 0
-NSSTATDESC["failure"] = 0
-NSSTATDESC["xfrdone"] = 0
-NSSTATDESC["updatereqfwd"] = 0
-NSSTATDESC["updaterespfwd"] = 0
-NSSTATDESC["updatefwdfail"] = 0
-NSSTATDESC["updatedone"] = 0
-NSSTATDESC["updatefail"] = 0
-NSSTATDESC["updatebadprereq"] = 0
-RESSTATDESC={}
-RESSTATDESC["counterid"] = 0
-RESSTATDESC["queryv4"] = 0
-RESSTATDESC["queryv6"] = 0
-RESSTATDESC["responsev4"] = 0
-RESSTATDESC["responsev6"] = 0
-RESSTATDESC["nxdomain"] = 0
-RESSTATDESC["servfail"] = 0
-RESSTATDESC["formerr"] = 0
-RESSTATDESC["othererror"] = 0
-RESSTATDESC["edns0fail"] = 0
-RESSTATDESC["mismatch"] = 0
-RESSTATDESC["truncated"] = 0
-RESSTATDESC["lame"] = 0
-RESSTATDESC["retry"] = 0
-RESSTATDESC["dispabort"] = 0
-RESSTATDESC["dispsockfail"] = 0
-RESSTATDESC["querytimeout"] = 0
-RESSTATDESC["gluefetchv4"] = 0
-RESSTATDESC["gluefetchv6"] = 0
-RESSTATDESC["gluefetchv4fail"] = 0
-RESSTATDESC["gluefetchv6fail"] = 0
-RESSTATDESC["val"] = 0
-RESSTATDESC["valsuccess"] = 0
-RESSTATDESC["valnegsuccess"] = 0
-RESSTATDESC["valfail"] = 0
-RESSTATDESC["queryrtt0"] = 0
-RESSTATDESC["queryrtt1"] = 0
-RESSTATDESC["queryrtt2"] = 0
-RESSTATDESC["queryrtt3"] = 0
-RESSTATDESC["queryrtt4"] = 0
-RESSTATDESC["queryrtt5"] = 0
-SOCKSTATDESC={}
-SOCKSTATDESC["counterid"] = 0
-SOCKSTATDESC["udp4open"] = 0
-SOCKSTATDESC["udp6open"] = 0
-SOCKSTATDESC["tcp4open"] = 0
-SOCKSTATDESC["tcp6open"] = 0
-SOCKSTATDESC["unixopen"] = 0
-SOCKSTATDESC["udp4openfail"] = 0
-SOCKSTATDESC["udp6openfail"] = 0
-SOCKSTATDESC["tcp4openfail"] = 0
-SOCKSTATDESC["tcp6openfail"] = 0
-SOCKSTATDESC["unixopenfail"] = 0
-SOCKSTATDESC["udp4close"] = 0
-SOCKSTATDESC["udp6close"] = 0
-SOCKSTATDESC["tcp4close"] = 0
-SOCKSTATDESC["tcp6close"] = 0
-SOCKSTATDESC["unixclose"] = 0
-SOCKSTATDESC["fdwatchclose"] = 0
-SOCKSTATDESC["udp4bindfail"] = 0
-SOCKSTATDESC["udp6bindfail"] = 0
-SOCKSTATDESC["tcp4bindfail"] = 0
-SOCKSTATDESC["tcp6bindfail"] = 0
-SOCKSTATDESC["unixbindfail"] = 0
-SOCKSTATDESC["fdwatchbindfail"] = 0
-SOCKSTATDESC["udp4connectfail"] = 0
-SOCKSTATDESC["udp6connectfail"] = 0
-SOCKSTATDESC["tcp4connectfail"] = 0
-SOCKSTATDESC["tcp6connectfail"] = 0
-SOCKSTATDESC["unixconnectfail"] = 0
-SOCKSTATDESC["fdwatchconnectfail"] = 0
-SOCKSTATDESC["udp4connect"] = 0
-SOCKSTATDESC["udp6connect"] = 0
-SOCKSTATDESC["tcp4connect"] = 0
-SOCKSTATDESC["tcp6connect"] = 0
-SOCKSTATDESC["unixconnect"] = 0
-SOCKSTATDESC["fdwatchconnect"] = 0
-SOCKSTATDESC["tcp4acceptfail"] = 0
-SOCKSTATDESC["tcp6acceptfail"] = 0
-SOCKSTATDESC["unixacceptfail"] = 0
-SOCKSTATDESC["tcp4accept"] = 0
-SOCKSTATDESC["tcp6accept"] = 0
-SOCKSTATDESC["unixaccept"] = 0
-SOCKSTATDESC["udp4sendfail"] = 0
-SOCKSTATDESC["udp6sendfail"] = 0
-SOCKSTATDESC["tcp4sendfail"] = 0
-SOCKSTATDESC["tcp6sendfail"] = 0
-SOCKSTATDESC["unixsendfail"] = 0
-SOCKSTATDESC["fdwatchsendfail"] = 0
-SOCKSTATDESC["udp4recvfail"] = 0
-SOCKSTATDESC["udp6recvfail"] = 0
-SOCKSTATDESC["tcp4recvfail"] = 0
-SOCKSTATDESC["tcp6recvfail"] = 0
-SOCKSTATDESC["unixrecvfail"] = 0
-SOCKSTATDESC["fdwatchrecvfail"] = 0
-SYSSTATDESC={}
-SYSSTATDESC['sockets'] = 0
-SYSSTATDESC['memory'] = 0
-
-sent = -1
-last_sent = -1
-loop = 0
-
-while 1:
-    NSSTATDESC["requestv4"] += random.randint(1,1000)
-    wait = sent + step_time - time.time()
-    if wait <= 0:
-        last_sent = sent;
-        sent = time.time();
-        msg = {'component':'auth', 'version':1, 'timestamp':time.time(),'stats':{'NSSTATDESC':NSSTATDESC,'RESSTATDESC':RESSTATDESC,'SOCKSTATDESC':SOCKSTATDESC,'SYSSTATDESC':SYSSTATDESC}}
-        print (msg)
-        print (cc.group_sendmsg(msg, statgroup))
-        wait = last_sent + step_time - time.time()
-        if wait < 0:
-            wait = step_time
-        loop += 1
-    r,w,e = select.select([cc._socket],[],[], wait)
-    for sock in r:
-        if sock == cc._socket:
-            data,envelope = cc.group_recvmsg(False)
-            print (data)
-            if (envelope["group"] == "Boss"):
-                if ("shutdown" in data):
-                    exit()
-            else:
-                print ("Unknown data: ", envelope,data)

src/bin/stats/test_total.py

@@ -1,54 +0,0 @@
-import sys
-sys.path.insert(0, '.')
-from statsd import *
-
-def test_total():
-    stats = {
-              'auth': {
-                     'from1': {
-                               'component':'auth',
-                               'version':1,
-                               'from':'from1',
-                               'timestamp':20100125,
-                               'stats': {
-                                   'AUTH': {
-                                       'counterid': 1,
-                                       'requestv4': 2,
-                                       'requestv6': 4,
-                                   },
-                                   'SYS': {
-                                       'sockets': 8,
-                                       'memory': 16,
-                                   },
-                                },
-                     },
-                     'from2': {
-                               'component':'auth',
-                               'version':1,
-                               'from':'from1',
-                               'timestamp':20100126,
-                               'stats': {
-                                   'AUTH': {
-                                       'counterid': 256,
-                                       'requestv4': 512,
-                                       'requestv6': 1024,
-                                   },
-                                   'SYS': {
-                                       'sockets': 2048,
-                                       'memory': 4096,
-                                   },
-                                },
-                     },
-              },
-            };
-    t = {}
-    for key in stats:
-        t[key] = total(stats[key])
-    print (stats)
-    print (dicttoxml(stats))
-    print (t)
-    print (dicttoxml(t))
-
-
-if __name__ == "__main__":
-    test_total()

src/bin/stats/tests/Makefile.am

@@ -0,0 +1,15 @@
+SUBDIRS = isc testdata
+PYTESTS = b10-stats_test.py b10-stats_stub_test.py
+EXTRA_DIST = $(PYTESTS) fake_time.py
+CLEANFILES = fake_time.pyc
+
+# later will have configure option to choose this, like: coverage run --branch
+PYCOVERAGE = $(PYTHON)
+# test using command-line arguments, so use check-local target instead of TESTS
+check-local:
+	for pytest in $(PYTESTS) ; do \
+	echo Running test: $$pytest ; \
+	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/stats:$(abs_top_builddir)/src/bin/stats/tests \
+	B10_FROM_BUILD=$(abs_top_builddir) \
+	$(PYCOVERAGE) $(abs_srcdir)/$$pytest || exit ; \
+	done

src/bin/stats/tests/b10-stats_stub_test.py

@@ -0,0 +1,116 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+__version__ = "$Revision$"
+
+#
+# Tests for the stats stub module
+#
+import unittest
+import time
+import os
+import imp
+import stats_stub
+from isc.cc.session import Session
+from stats_stub import CCSessionStub, BossModuleStub, AuthModuleStub
+from stats import get_datetime
+
+class TestStats(unittest.TestCase):
+
+    def setUp(self):
+        self.session = Session()
+        self.stub = CCSessionStub(session=self.session, verbose=True)
+        self.boss = BossModuleStub(session=self.session, verbose=True)
+        self.auth = AuthModuleStub(session=self.session, verbose=True)
+        self.env = {'from': self.session.lname, 'group': 'Stats',
+                    'instance': '*', 'to':'*',
+                    'type':'send','seq':0}
+        self.result_ok = {'result': [0]}
+
+    def tearDown(self):
+        self.session.close()
+
+    def test_stub(self):
+        """
+        Test for send_command of CCSessionStub object
+        """
+        env = self.env
+        result_ok = self.result_ok
+        self.assertEqual(('status', None, env),
+                         self.stub.send_command('status', None))
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+        self.assertEqual(('shutdown', None, env),
+                         self.stub.send_command('shutdown', None))
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+        self.assertEqual(('show', None, env),
+                         self.stub.send_command('show', None))
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+        self.assertEqual(('set', {'atest': 100.0}, env),
+                         self.stub.send_command('set', {'atest': 100.0}))
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+
+    def test_boss_stub(self):
+        """
+        Test for send_command of BossModuleStub object
+        """
+        env = self.env
+        result_ok = self.result_ok
+        self.assertEqual(('set', {"stats_data":
+                                      {"bind10.boot_time": get_datetime()}
+                                  }, env), self.boss.send_boottime())
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+
+    def test_auth_stub(self):
+        """
+        Test for send_command of AuthModuleStub object
+        """
+        env = self.env
+        result_ok = self.result_ok
+        self.assertEqual(
+            ('set', {"stats_data": {"auth.queries.udp": 1}}, env),
+            self.auth.send_udp_query_count())
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+        self.assertEqual(
+            ('set', {"stats_data": {"auth.queries.tcp": 1}}, env),
+            self.auth.send_tcp_query_count())
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+        self.assertEqual(
+            ('set', {"stats_data": {"auth.queries.udp": 100}}, env),
+            self.auth.send_udp_query_count(cmd='set', cnt=100))
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+        self.assertEqual(
+            ('set', {"stats_data": {"auth.queries.tcp": 99}}, env),
+            self.auth.send_tcp_query_count(cmd='set', cnt=99))
+        self.assertEqual(result_ok, self.session.get_message("Stats", None))
+
+    def test_func_main(self):
+        # explicitly make failed
+        self.session.close()
+        stats_stub.main(session=self.session)
+
+    def test_osenv(self):
+        """
+        test for not having environ "B10_FROM_BUILD"
+        """
+        if "B10_FROM_BUILD" in os.environ:
+            path = os.environ["B10_FROM_BUILD"]
+            os.environ.pop("B10_FROM_BUILD")
+            imp.reload(stats_stub)
+            os.environ["B10_FROM_BUILD"] = path
+            imp.reload(stats_stub)
+
+if __name__ == "__main__":
+    unittest.main()

src/bin/stats/tests/b10-stats_test.py

@@ -0,0 +1,646 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+__version__ = "$Revision$"
+
+#
+# Tests for the stats module
+#
+import os
+import sys
+import time
+import unittest
+import imp
+from isc.cc.session import Session, SessionError
+from isc.config.ccsession import ModuleCCSession, ModuleCCSessionError
+import stats
+from stats import SessionSubject, CCSessionListener, get_timestamp, get_datetime
+from fake_time import _TEST_TIME_SECS, _TEST_TIME_STRF
+
+# setting Constant
+if sys.path[0] == '':
+    TEST_SPECFILE_LOCATION = "./testdata/stats_test.spec"
+else:
+    TEST_SPECFILE_LOCATION = sys.path[0] + "/testdata/stats_test.spec"
+
+class TestStats(unittest.TestCase):
+
+    def setUp(self):
+        self.session = Session()
+        self.subject = SessionSubject(session=self.session, verbose=True)
+        self.listener = CCSessionListener(self.subject, verbose=True)
+        self.stats_spec = self.listener.cc_session.get_module_spec().get_config_spec()
+        self.module_name = self.listener.cc_session.get_module_spec().get_module_name()
+        self.stats_data = {
+                'report_time' : get_datetime(),
+                'bind10.boot_time' : "1970-01-01T00:00:00Z",
+                'stats.timestamp' : get_timestamp(),
+                'stats.lname' : self.session.lname,
+                'auth.queries.tcp': 0,
+                'auth.queries.udp': 0,
+                "stats.boot_time": get_datetime(),
+                "stats.start_time": get_datetime(),
+                "stats.last_update_time": get_datetime()
+                }
+        # check starting
+        self.assertFalse(self.subject.running)
+        self.subject.start()
+        self.assertTrue(self.subject.running)
+        self.assertEqual(len(self.session.message_queue), 0)
+        self.assertEqual(self.module_name, 'Stats')
+
+    def tearDown(self):
+        # check closing
+        self.subject.stop()
+        self.assertFalse(self.subject.running)
+        self.subject.detach(self.listener)
+        self.listener.stop()
+        self.session.close()
+
+    def test_local_func(self):
+        """
+        Test for local function
+        
+        """
+        # test for result_ok
+        self.assertEqual(type(result_ok()), dict)
+        self.assertEqual(result_ok(), {'result': [0]})
+        self.assertEqual(result_ok(1), {'result': [1]})
+        self.assertEqual(result_ok(0,'OK'), {'result': [0, 'OK']})
+        self.assertEqual(result_ok(1,'Not good'), {'result': [1, 'Not good']})
+        self.assertEqual(result_ok(None,"It's None"), {'result': [None, "It's None"]})
+        self.assertNotEqual(result_ok(), {'RESULT': [0]})
+
+        # test for get_timestamp
+        self.assertEqual(get_timestamp(), _TEST_TIME_SECS)
+
+        # test for get_datetime
+        self.assertEqual(get_datetime(), _TEST_TIME_STRF)
+
+    def test_show_command(self):
+        """
+        Test for show command
+        
+        """
+        # test show command without arg
+        self.session.group_sendmsg({"command": [ "show", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        # ignore under 0.9 seconds
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test show command with arg
+        self.session.group_sendmsg({"command": [ "show", {"stats_item_name": "stats.lname"}]}, "Stats")
+        self.assertEqual(len(self.subject.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.subject.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'stats.lname': self.stats_data['stats.lname']}),
+                         result_data)
+        self.assertEqual(len(self.subject.session.message_queue), 0)
+
+        # test show command with arg which has wrong name
+        self.session.group_sendmsg({"command": [ "show", {"stats_item_name": "stats.dummy"}]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        # ignore under 0.9 seconds
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_set_command(self):
+        """
+        Test for set command
+        
+        """
+        # test set command
+        self.stats_data['auth.queries.udp'] = 54321
+        self.assertEqual(self.stats_data['auth.queries.udp'], 54321)
+        self.assertEqual(self.stats_data['auth.queries.tcp'], 0)
+        self.session.group_sendmsg({ "command": [
+                                      "set", {
+                                          'stats_data': {'auth.queries.udp': 54321 }
+                                      } ] },
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test show command
+        self.session.group_sendmsg({"command": [ "show", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set command 2
+        self.stats_data['auth.queries.udp'] = 0
+        self.assertEqual(self.stats_data['auth.queries.udp'], 0)
+        self.assertEqual(self.stats_data['auth.queries.tcp'], 0)
+        self.session.group_sendmsg({ "command": [ "set", {'stats_data': {'auth.queries.udp': 0}} ]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test show command 2
+        self.session.group_sendmsg({"command": [ "show", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set command 3
+        self.stats_data['auth.queries.tcp'] = 54322
+        self.assertEqual(self.stats_data['auth.queries.udp'], 0)
+        self.assertEqual(self.stats_data['auth.queries.tcp'], 54322)
+        self.session.group_sendmsg({ "command": [
+                                      "set", {
+                                          'stats_data': {'auth.queries.tcp': 54322 }
+                                      } ] },
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test show command 3
+        self.session.group_sendmsg({"command": [ "show", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_remove_command(self):
+        """
+        Test for remove command
+        
+        """
+        self.session.group_sendmsg({"command":
+                                   [ "remove", {"stats_item_name": 'bind10.boot_time' }]},
+                              "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+        self.assertEqual(self.stats_data.pop('bind10.boot_time'), "1970-01-01T00:00:00Z")
+        self.assertFalse('bind10.boot_time' in self.stats_data)
+
+        # test show command with arg
+        self.session.group_sendmsg({"command":
+                                    [ "show", {"stats_item_name": 'bind10.boot_time'}]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertFalse('bind10.boot_time' in result_data['result'][1])
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_reset_command(self):
+        """
+        Test for reset command
+        
+        """
+        self.session.group_sendmsg({"command": [ "reset" ] }, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test show command
+        self.session.group_sendmsg({"command": [ "show" ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_status_command(self):
+        """
+        Test for status command
+        
+        """
+        self.session.group_sendmsg({"command": [ "status" ] }, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(0, "I'm alive."),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_unknown_command(self):
+        """
+        Test for unknown command
+        
+        """
+        self.session.group_sendmsg({"command": [ "hoge", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(1, "Unknown command: 'hoge'"),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_shutdown_command(self):
+        """
+        Test for shutdown command
+        
+        """
+        self.session.group_sendmsg({"command": [ "shutdown", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.assertTrue(self.subject.running)
+        self.subject.check()
+        self.assertFalse(self.subject.running)
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+
+    def test_some_commands(self):
+        """
+        Test for some commands in a row
+        
+        """
+        # test set command
+        self.stats_data['bind10.boot_time'] = '2010-08-02T14:47:56Z'
+        self.assertEqual(self.stats_data['bind10.boot_time'], '2010-08-02T14:47:56Z')
+        self.session.group_sendmsg({ "command": [
+                                      "set", {
+                                          'stats_data': {'bind10.boot_time': '2010-08-02T14:47:56Z' }
+                                      }]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check its value
+        self.session.group_sendmsg({ "command": [
+                                      "show", { 'stats_item_name': 'bind10.boot_time' }
+                                     ] }, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'bind10.boot_time': '2010-08-02T14:47:56Z'}),
+                         result_data)
+        self.assertEqual(result_ok(0, {'bind10.boot_time': self.stats_data['bind10.boot_time']}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set command 2nd
+        self.stats_data['auth.queries.udp'] = 98765
+        self.assertEqual(self.stats_data['auth.queries.udp'], 98765)
+        self.session.group_sendmsg({ "command": [
+                                      "set", { 'stats_data': {
+                                            'auth.queries.udp':
+                                              self.stats_data['auth.queries.udp']
+                                            } } 
+                                     ] }, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check its value
+        self.session.group_sendmsg({"command": [
+				      "show", {'stats_item_name': 'auth.queries.udp'}
+                                    ] }, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'auth.queries.udp': 98765}),
+                         result_data)
+        self.assertEqual(result_ok(0, {'auth.queries.udp': self.stats_data['auth.queries.udp']}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set command 3
+        self.stats_data['auth.queries.tcp'] = 4321
+        self.session.group_sendmsg({"command": [
+                                      "set",
+                                      {'stats_data': {'auth.queries.tcp': 4321 }} ]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check value
+        self.session.group_sendmsg({"command": [ "show", {'stats_item_name': 'auth.queries.tcp'} ]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'auth.queries.tcp': 4321}),
+                         result_data)
+        self.assertEqual(result_ok(0, {'auth.queries.tcp': self.stats_data['auth.queries.tcp']}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        self.session.group_sendmsg({"command": [ "show", {'stats_item_name': 'auth.queries.udp'} ]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'auth.queries.udp': 98765}),
+                         result_data)
+        self.assertEqual(result_ok(0, {'auth.queries.udp': self.stats_data['auth.queries.udp']}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set command 4
+        self.stats_data['auth.queries.tcp'] = 67890
+        self.session.group_sendmsg({"command": [
+                                      "set", {'stats_data': {'auth.queries.tcp': 67890 }} ]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test show command for all values
+        self.session.group_sendmsg({"command": [ "show", None ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, self.stats_data), result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_some_commands2(self):
+        """
+        Test for some commands in a row using list-type value
+        
+        """
+        self.stats_data['listtype'] = [1, 2, 3]
+        self.assertEqual(self.stats_data['listtype'], [1, 2, 3])
+        self.session.group_sendmsg({ "command": [
+                                      "set", {'stats_data': {'listtype': [1, 2, 3] }}
+                                      ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check its value
+        self.session.group_sendmsg({ "command": [
+                                      "show", { 'stats_item_name': 'listtype'}
+                                     ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'listtype': [1, 2, 3]}),
+                         result_data)
+        self.assertEqual(result_ok(0, {'listtype': self.stats_data['listtype']}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set list-type value
+        self.assertEqual(self.stats_data['listtype'], [1, 2, 3])
+        self.session.group_sendmsg({"command": [
+                                      "set", {'stats_data': {'listtype': [3, 2, 1, 0] }}
+                                    ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check its value
+        self.session.group_sendmsg({ "command": [
+                                      "show", { 'stats_item_name': 'listtype' }
+                                     ] }, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'listtype': [3, 2, 1, 0]}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_some_commands3(self):
+        """
+        Test for some commands in a row using dictionary-type value
+        
+        """
+        self.stats_data['dicttype'] = {"a": 1, "b": 2, "c": 3}
+        self.assertEqual(self.stats_data['dicttype'], {"a": 1, "b": 2, "c": 3})
+        self.session.group_sendmsg({ "command": [
+                                      "set", {
+                                          'stats_data': {'dicttype': {"a": 1, "b": 2, "c": 3} }
+                                      }]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check its value
+        self.session.group_sendmsg({ "command": [ "show", { 'stats_item_name': 'dicttype' } ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'dicttype': {"a": 1, "b": 2, "c": 3}}),
+                         result_data)
+        self.assertEqual(result_ok(0, {'dicttype': self.stats_data['dicttype']}),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # test set list-type value
+        self.assertEqual(self.stats_data['dicttype'], {"a": 1, "b": 2, "c": 3})
+        self.session.group_sendmsg({"command": [
+                                      "set", {'stats_data': {'dicttype': {"a": 3, "b": 2, "c": 1, "d": 0} }} ]},
+                                   "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+        self.assertEqual(len(self.session.message_queue), 0)
+
+        # check its value
+        self.session.group_sendmsg({ "command": [ "show", { 'stats_item_name': 'dicttype' }]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        result_data = self.session.get_message("Stats", None)
+        self.assertEqual(result_ok(0, {'dicttype': {"a": 3, "b": 2, "c": 1, "d": 0} }),
+                         result_data)
+        self.assertEqual(len(self.session.message_queue), 0)
+
+    def test_config_update(self):
+        """
+        Test for config update
+        
+        """
+        # test show command without arg
+        self.session.group_sendmsg({"command": [ "config_update", {"x-version":999} ]}, "Stats")
+        self.assertEqual(len(self.session.message_queue), 1)
+        self.subject.check()
+        self.assertEqual(result_ok(),
+                         self.session.get_message("Stats", None))
+
+class TestStats2(unittest.TestCase):
+
+    def setUp(self):
+        self.session = Session(verbose=True)
+        self.subject = SessionSubject(session=self.session, verbose=True)
+        self.listener = CCSessionListener(self.subject, verbose=True)
+        self.module_name = self.listener.cc_session.get_module_spec().get_module_name()
+        # check starting
+        self.assertFalse(self.subject.running)
+        self.subject.start()
+        self.assertTrue(self.subject.running)
+        self.assertEqual(len(self.session.message_queue), 0)
+        self.assertEqual(self.module_name, 'Stats')
+
+    def tearDown(self):
+        # check closing
+        self.subject.stop()
+        self.assertFalse(self.subject.running)
+        self.subject.detach(self.listener)
+        self.listener.stop()
+
+    def test_specfile(self):
+        """
+        Test for specfile
+        
+        """
+        if "B10_FROM_BUILD" in os.environ:
+            self.assertEqual(stats.SPECFILE_LOCATION,
+                             os.environ["B10_FROM_BUILD"] + "/src/bin/stats/stats.spec")
+        imp.reload(stats)
+        # change path of SPECFILE_LOCATION
+        stats.SPECFILE_LOCATION = TEST_SPECFILE_LOCATION
+        self.assertEqual(stats.SPECFILE_LOCATION, TEST_SPECFILE_LOCATION)
+        self.subject = stats.SessionSubject(session=self.session, verbose=True)
+        self.session = self.subject.session
+        self.listener = stats.CCSessionListener(self.subject, verbose=True)
+
+        self.assertEqual(self.listener.stats_spec, [])
+        self.assertEqual(self.listener.stats_data, {})
+
+        self.assertEqual(self.listener.commands_spec, [
+                {
+                    "command_name": "status",
+                    "command_description": "identify whether stats module is alive or not",
+                    "command_args": []
+                },
+                {
+                    "command_name": "the_dummy",
+                    "command_description": "this is for testing",
+                    "command_args": []
+                }])
+
+    def test_func_initialize_data(self):
+        """
+        Test for initialize_data function 
+        
+        """
+        # prepare for sample data set
+        stats_spec = [
+            {
+                "item_name": "none_sample",
+                "item_type": "null",
+                "item_default": "None"
+            },
+            {
+                "item_name": "boolean_sample",
+                "item_type": "boolean",
+                "item_default": True
+            },
+            {
+                "item_name": "string_sample",
+                "item_type": "string",
+                "item_default": "A something"
+            },
+            {
+                "item_name": "int_sample",
+                "item_type": "integer",
+                "item_default": 9999999
+            },
+            {
+                "item_name": "real_sample",
+                "item_type": "real",
+                "item_default": 0.0009
+            },
+            {
+                "item_name": "list_sample",
+                "item_type": "list",
+                "item_default": [0, 1, 2, 3, 4],
+                "list_item_spec": []
+            },
+            {
+                "item_name": "map_sample",
+                "item_type": "map",
+                "item_default": {'name':'value'},
+                "map_item_spec": []
+            },
+            {
+                "item_name": "other_sample",
+                "item_type": "__unknown__",
+                "item_default": "__unknown__"
+            }
+        ]
+        # data for comparison
+        stats_data = {
+            'none_sample': None,
+            'boolean_sample': True,
+            'string_sample': 'A something',
+            'int_sample': 9999999,
+            'real_sample': 0.0009,
+            'list_sample': [0, 1, 2, 3, 4],
+            'map_sample': {'name':'value'},
+            'other_sample': '__unknown__'
+        }
+        self.assertEqual(self.listener.initialize_data(stats_spec), stats_data)
+
+    def test_func_main(self):
+        # explicitly make failed
+        self.session.close()
+        stats.main(session=self.session)
+
+    def test_osenv(self):
+        """
+        test for not having environ "B10_FROM_BUILD"
+        """
+        if "B10_FROM_BUILD" in os.environ:
+            path = os.environ["B10_FROM_BUILD"]
+            os.environ.pop("B10_FROM_BUILD")
+            imp.reload(stats)
+            os.environ["B10_FROM_BUILD"] = path
+            imp.reload(stats)
+
+def result_ok(*args):
+    if args:
+        return { 'result': list(args) }
+    else:
+        return { 'result': [ 0 ] }
+
+if __name__ == "__main__":
+    unittest.main()

src/bin/stats/tests/fake_time.py

@@ -0,0 +1,48 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+__version__ = "$Revision$"
+
+# This is a dummy time class against a Python standard time class.
+# It is just testing use only.
+# Other methods which time class has is not implemented.
+# (This class isn't orderloaded for time class.)
+
+# These variables are constant. These are example.
+_TEST_TIME_SECS = 1283364938.229088
+_TEST_TIME_STRF = '2010-09-01T18:15:38Z'
+
+def time():
+    """
+    This is a dummy time() method against time.time()
+    """
+    # return float constant value
+    return _TEST_TIME_SECS
+
+def gmtime():
+    """
+    This is a dummy gmtime() method against time.gmtime()
+    """
+    # always return nothing
+    return None
+
+def strftime(*arg):
+    """
+    This is a dummy gmtime() method against time.gmtime()
+    """
+    return _TEST_TIME_STRF
+
+

src/bin/stats/tests/isc/Makefile.am

@@ -0,0 +1,3 @@
+SUBDIRS = cc config util
+EXTRA_DIST = __init__.py
+CLEANFILES = __init__.pyc

src/bin/stats/tests/isc/cc/Makefile.am

@@ -0,0 +1,2 @@
+EXTRA_DIST = __init__.py session.py
+CLEANFILES = __init__.pyc session.pyc

src/bin/stats/tests/isc/cc/__init__.py

@@ -0,0 +1 @@
+from isc.cc.session import *

src/bin/stats/tests/isc/cc/session.py

@@ -0,0 +1,127 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+# This module is a mock-up class of isc.cc.session
+
+__version__ = "$Revision$"
+
+import sys
+
+# set a dummy lname
+_TEST_LNAME = '123abc@xxxx'
+
+class Queue():
+    def __init__(self, msg=None, env={}):
+        self.msg = msg
+        self.env = env
+
+    def dump(self):
+        return { 'msg': self.msg, 'env': self.env }
+               
+class SessionError(Exception):
+    pass
+
+class Session:
+    def __init__(self, socket_file=None, verbose=False):
+        self._lname = _TEST_LNAME
+        self.message_queue = []
+        self.old_message_queue = []
+        self._socket = True
+        self.verbose = verbose
+
+    @property
+    def lname(self):
+        return self._lname
+
+    def close(self):
+        self._socket = False
+
+    def _next_sequence(self, que=None):
+        return len(self.message_queue)
+
+    def enqueue(self, msg=None, env={}):
+        if not self._socket:
+            raise SessionError("Session has been closed.")
+        seq = self._next_sequence()
+        env.update({"seq": 0}) # fixed here
+        que = Queue(msg=msg, env=env)
+        self.message_queue.append(que)
+        if self.verbose:
+            sys.stdout.write("[Session] enqueue: " + str(que.dump()) + "\n")
+        return seq
+
+    def dequeue(self, seq=0):
+        if not self._socket:
+            raise SessionError("Session has been closed.")
+        que = None
+        try:
+            que = self.message_queue.pop(seq)
+            self.old_message_queue.append(que)
+        except IndexError:
+            que = Queue()
+        if self.verbose:
+            sys.stdout.write("[Session] dequeue: " + str(que.dump()) + "\n")
+        return que
+
+    def get_queue(self, seq=None):
+        if not self._socket:
+            raise SessionError("Session has been closed.")
+        if seq is None:
+            seq = len(self.message_queue) - 1
+        que = None
+        try:
+            que = self.message_queue[seq]
+        except IndexError:
+            raise IndexError
+            que = Queue()
+        if self.verbose:
+            sys.stdout.write("[Session] get_queue: " + str(que.dump()) + "\n")
+        return que
+
+    def group_sendmsg(self, msg, group, instance="*", to="*"):
+        return self.enqueue(msg=msg, env={
+                "type": "send",
+                "from": self._lname,
+                "to": to,
+                "group": group,
+                "instance": instance })
+
+    def group_recvmsg(self, nonblock=True, seq=0):
+        que = self.dequeue(seq)
+        return que.msg, que.env
+        
+    def group_reply(self, routing, msg):
+        return self.enqueue(msg=msg, env={
+                "type": "send",
+                "from": self._lname,
+                "to": routing["from"],
+                "group": routing["group"],
+                "instance": routing["instance"],
+                "reply": routing["seq"] })
+
+    def get_message(self, group, to='*'):
+        if not self._socket:
+            raise SessionError("Session has been closed.")
+        que = Queue()
+        for q in self.message_queue:
+            if q.env['group'] == group:
+                self.message_queue.remove(q)
+                self.old_message_queue.append(q)
+                que = q
+        if self.verbose:
+            sys.stdout.write("[Session] get_message: " + str(que.dump()) + "\n")
+        return q.msg
+

src/bin/stats/tests/isc/config/Makefile.am

@@ -0,0 +1,2 @@
+EXTRA_DIST = __init__.py ccsession.py
+CLEANFILES = __init__.pyc ccsession.pyc

src/bin/stats/tests/isc/config/__init__.py

@@ -0,0 +1 @@
+from isc.config.ccsession import *

src/bin/stats/tests/isc/config/ccsession.py

@@ -0,0 +1,114 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+# This module is a mock-up class of isc.cc.session
+
+__version__ = "$Revision$"
+
+import json
+from isc.cc.session import Session
+
+COMMAND_CONFIG_UPDATE = "config_update"
+
+def parse_answer(msg):
+    try:
+        return msg['result'][0], msg['result'][1]
+    except IndexError:
+        return msg['result'][0], None
+
+def create_answer(rcode, arg = None):
+    if arg is None:
+        return { 'result': [ rcode ] }
+    else:
+        return { 'result': [ rcode, arg ] }
+
+def parse_command(msg):
+    try:
+        return msg['command'][0], msg['command'][1]
+    except IndexError:
+        return msg['command'][0], None
+
+def create_command(command_name, params = None):
+    if params is None:
+        return {"command": [command_name]}
+    else:
+        return {"command": [command_name, params]}
+
+def module_spec_from_file(spec_file, check = True):
+    file = open(spec_file)
+    module_spec = json.loads(file.read())
+    return ModuleSpec(module_spec['module_spec'], check)
+
+class ModuleSpec:
+    def __init__(self, module_spec, check = True):
+        self._module_spec = module_spec
+
+    def get_config_spec(self):
+        return self._module_spec['config_data']
+
+    def get_commands_spec(self):
+        return self._module_spec['commands']
+
+    def get_module_name(self):
+        return self._module_spec['module_name']
+
+class ModuleCCSessionError(Exception):
+    pass
+
+class ConfigData:
+    def __init__(self, specification):
+        self.specification = specification
+
+class ModuleCCSession(ConfigData):
+    def __init__(self, spec_file_name, config_handler, command_handler, cc_session = None):
+        module_spec = module_spec_from_file(spec_file_name)
+        ConfigData.__init__(self, module_spec)
+        self._module_name = module_spec.get_module_name()
+        self.set_config_handler(config_handler)
+        self.set_command_handler(command_handler)
+        if not cc_session:
+            self._session = Session(verbose=True)
+        else:
+            self._session = cc_session
+
+    def start(self):
+        pass
+
+    def close(self):
+        self._session.close()
+
+    def check_command(self, nonblock=True):
+        msg, env = self._session.group_recvmsg(nonblock)
+        if not msg or 'result' in msg:
+            return
+        cmd, arg = parse_command(msg)
+        answer = None
+        if cmd == COMMAND_CONFIG_UPDATE and self._config_handler:
+            answer = self._config_handler(arg)
+        elif env['group'] == self._module_name and self._command_handler:
+            answer = self._command_handler(cmd, arg)
+        if answer:
+            self._session.group_reply(env, answer)
+
+    def set_config_handler(self, config_handler):
+        self._config_handler = config_handler
+        # should we run this right now since we've changed the handler?
+
+    def set_command_handler(self, command_handler):
+        self._command_handler = command_handler
+
+    def get_module_spec(self):
+        return self.specification

src/bin/stats/tests/isc/util/Makefile.am

@@ -0,0 +1,2 @@
+EXTRA_DIST = __init__.py process.py
+CLEANFILES = __init__.pyc process.pyc

src/bin/stats/tests/isc/util/process.py

@@ -0,0 +1,20 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+
+# A dummy function of isc.util.process.rename()
+def rename(name=None):
+    pass

src/bin/stats/tests/isc/utils/Makefile.am

@@ -0,0 +1,2 @@
+EXTRA_DIST = __init__.py process.py
+CLEANFILES = __init__.pyc process.pyc

src/bin/stats/tests/isc/utils/process.py

@@ -0,0 +1,20 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id$
+
+# A dummy function of isc.utils.process.rename()
+def rename(name=None):
+    pass

src/bin/stats/tests/stats_test.in

@@ -0,0 +1,31 @@
+#! /bin/sh
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
+export PYTHON_EXEC
+
+PYTHONPATH=@abs_top_builddir@/src/lib/python:@abs_top_srcdir@/src/bin/stats:@abs_top_srcdir@/src/bin/stats/tests
+export PYTHONPATH
+
+B10_FROM_BUILD=@abs_top_builddir@
+export B10_FROM_BUILD
+
+TEST_PATH=@abs_top_srcdir@/src/bin/stats/tests
+
+cd ${TEST_PATH}
+${PYTHON_EXEC} -O b10-stats_test.py $*
+${PYTHON_EXEC} -O b10-stats_stub_test.py $*

src/bin/stats/tests/testdata/Makefile.am

@@ -0,0 +1 @@
+EXTRA_DIST = stats_test.spec

src/bin/stats/tests/testdata/stats_test.spec

@@ -0,0 +1,19 @@
+{
+  "module_spec": {
+    "module_name": "Stats",
+    "module_description": "Stats daemon",
+    "config_data": [],
+    "commands": [
+      {
+        "command_name": "status",
+        "command_description": "identify whether stats module is alive or not",
+        "command_args": []
+      },
+      {
+        "command_name": "the_dummy",
+        "command_description": "this is for testing",
+        "command_args": []
+      }
+    ]
+  }
+}

src/bin/tests/Makefile.am

@@ -0,0 +1,13 @@
+PYTESTS = process_rename_test.py
+# .py will be generated by configure, so we don't have to include it
+# in EXTRA_DIST.
+
+# later will have configure option to choose this, like: coverage run --branch
+PYCOVERAGE = $(PYTHON)
+# test using command-line arguments, so use check-local target instead of TESTS
+check-local:
+	for pytest in $(PYTESTS) ; do \
+	echo Running test: $$pytest ; \
+	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/lib/dns/python/.libs \
+	$(PYCOVERAGE) $(abs_builddir)/$$pytest || exit ; \
+	done