Merge branch 'master' into trac2571_commit_log_fix

.gitignore

@@ -34,5 +34,6 @@ TAGS
 /all.info
 /coverage-cpp-html
 /dns++.pc
-/report.info
+/local.zone.sqlite3
 /logger_lockfile
+/report.info

ChangeLog

@@ -1,3 +1,139 @@
+551.    [bug]       shane
+	Kill msgq if we cannot connect to it on startup.
+	When the boss process was unable to connect to the msgq, it would
+	exit. However, it would leave the msgq process running. This has
+	been fixed, and the msgq is now stopped in this case.
+	(Trac #2608, git 016925ef2437e0396127e135c937d3a55539d224)
+
+550.	[func]		tomek
+	b10-dhcp4: The DHCPv4 server now generates a server identifier
+	the first time it is run. The identifier is preserved in a file
+	across server restarts.
+	b10-dhcp6: The server identifier is now preserved in a file across
+	server restarts.
+	(Trac #2597, git fa342a994de5dbefe32996be7eebe58f6304cff7)
+
+549.	[func]		tomek
+	b10-dhcp6: It is now possible to specify that a configured subnet
+	is reachable locally over specified interface (see "interface"
+	parameter in Subnet6 configuration).
+	(Trac #2596, git a70f6172194a976b514cd7d67ce097bbca3c2798)
+
+548.	[func]		vorner
+	The message queue daemon now appears on the bus. This has two
+	effects, one is it obeys logging configuration and logs to the
+	correct place like the rest of the modules. The other is it
+	appears in bindctl as module (but it doesn't have any commands or
+	configuration yet).
+	(Trac #2582, git ced31d8c5a0f2ca930b976d3caecfc24fc04634e)
+
+547.	[func]*		vorner
+	The b10-loadzone now performs more thorough sanity check on the
+	loaded data.  Some of the checks are now fatal and zone failing
+	them will be rejected.
+	(Trac #2436, git 48d999f1cb59f308f9f30ba2639521d2a5a85baa)
+
+546.	[func]		marcin
+	DHCP option definitions can be now created using the
+	Configuration Manager. The option definition specifies
+	the option code, name and the types of the data being
+	carried by the option.  The Configuration Manager
+	reports an error on attempt to override standard DHCP
+	option definition.
+	(Trac #2317, git 71e25eb81e58a695cf3bad465c4254b13a50696e)
+
+545.	[func]		jinmei
+	libdns++: the SOA Rdata class now uses the generic lexer in
+	constructors from text.  This means that the MNAME and RNAME of an
+	SOA RR in a zone file can now be non absolute (the origin name
+	in that context will be used), e.g., when loaded by b10-loadzone.
+	(Trac #2500, git 019ca218027a218921519f205139b96025df2bb5)
+
+544.	[func]		tomek
+	b10-dhcp4: Allocation engine support for IPv4 added. Currently
+	supported operations are server selection (Discover/Offer),
+	address assignment (Request/Ack), address renewal (Request/Ack),
+	and address release (Release). Expired leases can be reused.
+	Some options (e.g. Router Option) are still hardcoded, so the
+	DHCPv4 server is not yet usable, although its address allocation
+	is operational.
+	(Trac #2320, git 60606cabb1c9584700b1f642bf2af21a35c64573)
+
+543.	[func]*		jelte
+	When calling getFullConfig() as a module, , the configuration is now
+	returned as properly-structured JSON.  Previously, the structure had
+	been flattened, with all data being labelled by fully-qualified element
+	names.
+	(Trac #2619, git bed3c88c25ea8f7e951317775e99ebce3340ca22)
+
+542.	[func]		marcin
+	Created OptionSpace and OptionSpace6 classes to represent DHCP
+	option spaces. The option spaces are used to group instances
+	and definitions of options having uniqe codes. A special type
+	of option space is the so-called "vendor specific option space"
+	which groups sub-options sent within Vendor Encapsulated Options.
+	The new classes are not used yet but they will be used once
+	the creation of option spaces by configuration manager is
+	implemented.
+	(Trac #2313, git 37a27e19be874725ea3d560065e5591a845daa89)
+
+541.	[func]		marcin
+	Added routines to search for configured DHCP options and their
+	definitions using name of the option space they belong to.
+	New routines are called internally from the DHCPv4 and DHCPv6
+	servers code.
+	(Trac #2315, git 741fe7bc96c70df35d9a79016b0aa1488e9b3ac8)
+
+540.	[func]		marcin
+	DHCP Option values can be now specified using a string of
+	tokens separated with comma sign. Subsequent tokens are used
+	to set values for corresponding data fields in a particular
+	DHCP option. The format of the token matches the data type
+	of the corresponding option field: e.g. "192.168.2.1" for IPv4
+	address, "5" for integer value etc.
+	(Trac #2545, git 792c129a0785c73dd28fd96a8f1439fe6534a3f1)
+
+539.	[func]		stephen
+	Add logging to the DHCP server library.
+	(Trac #2524, git b55b8b6686cc80eed41793c53d1779f4de3e9e3c)
+
+538.	[bug]		muks
+	Added escaping of special characters (double-quotes, semicolon,
+	backslash, etc.) in text-like RRType's toText() implementation.
+	Without this change, some TXT and SPF RDATA were incorrectly
+	stored in SQLite3 datasource as they were not escaped.
+	(Trac #2535, git f516fc484544b7e08475947d6945bc87636d4115)
+
+537.	[func]		tomek
+	b10-dhcp6: Support for RELEASE message has been added. Clients
+	are now able to release their non-temporary IPv6 addresses.
+	(Trac #2326, git 0974318566abe08d0702ddd185156842c6642424)
+
+536.	[build]		jinmei
+	Detect a build issue on FreeBSD with g++ 4.2 and Boost installed via
+	FreeBSD ports at ./configure time.  This seems to be a bug of
+	FreeBSD	ports setup and has been reported to the maintainer:
+	http://www.freebsd.org/cgi/query-pr.cgi?pr=174753
+	Until it's fixed, you need to build BIND 10 for FreeBSD that has
+	this problem with specifying --without-werror, with clang++
+	(development version), or with manually extracted Boost header
+	files (no compiled Boost library is necessary).
+	(Trac #1991, git 6b045bcd1f9613e3835551cdebd2616ea8319a36)
+
+535.	[bug]		jelte
+	The log4cplus internal logging mechanism has been disabled, and no
+	output from the log4cplus library itself should be printed to
+	stderr anymore. This output can be enabled by using the
+	compile-time option --enable-debug.
+	(Trac #1081, git db55f102b30e76b72b134cbd77bd183cd01f95c0)
+
+534.	[func]*			vorner
+	The b10-msgq now uses the same logging format as the rest
+	of the system. However, it still doesn't obey the common
+	configuration, as due to technical issues it is not able
+	to read it yet.
+	(git 9e6e821c0a33aab0cd0e70e51059d9a2761f76bb)
+
 bind10-1.0.0-beta released on December 20, 2012
 
 533.	[build]*		jreed

README

@@ -7,16 +7,20 @@ DHCP. BIND 10 is written in C++ and Python and provides a modular
 environment for serving, maintaining, and developing DNS and DHCP.
 
 This release includes the bind10 master process, b10-msgq message
-bus, b10-auth authoritative DNS server (with SQLite3 and in-memory
-backends), b10-resolver recursive or forwarding DNS server, b10-cmdctl
-remote control daemon, b10-cfgmgr configuration manager, b10-xfrin
-AXFR inbound service, b10-xfrout outgoing AXFR service, b10-zonemgr
-secondary manager, b10-stats statistics collection and reporting
-daemon, b10-stats-httpd for HTTP access to XML-formatted stats,
-b10-host DNS lookup utility, and a new libdns++ library for C++
-with a python wrapper. BIND 10 also provides experimental DHCPv4
-and DHCPv6 servers, b10-dhcp4 and b10-dhcp6, a portable DHCP library,
-libdhcp++, and a DHCP benchmarking tool, perfdhcp.
+bus, b10-cmdctl remote control daemon, b10-cfgmgr configuration
+manager, b10-stats statistics collection and reporting daemon, and
+b10-stats-httpd for HTTP access to XML-formatted stats.
+
+For DNS services, it provides the b10-auth authoritative DNS server
+(with SQLite3 and in-memory backends), b10-resolver recursive or
+forwarding DNS server, b10-xfrin IXFR/AXFR inbound service, b10-xfrout
+outgoing IXFR/AXFR service, b10-zonemgr secondary manager, libdns++
+library for C++ with a python wrapper, and many tests and example
+programs.
+
+BIND 10 also provides experimental DHCPv4 and DHCPv6 servers,
+b10-dhcp4 and b10-dhcp6, a portable DHCP library, libdhcp++, and
+a DHCP benchmarking tool, perfdhcp.
 
 Documentation is included with the source. See doc/guide/bind10-guide.txt
 (or bind10-guide.html) for installation instructions.  The

configure.ac

@@ -166,8 +166,6 @@ fi
 
 fi				dnl GXX = yes
 
-AM_CONDITIONAL(GCC_WERROR_OK, test $werror_ok = 1)
-
 # allow building programs with static link.  we need to make it selective
 # because loadable modules cannot be statically linked.
 AC_ARG_ENABLE([static-link],
@@ -284,7 +282,10 @@ AC_SUBST(PYTHON_LOGMSGPKG_DIR)
 
 # This is python package paths commonly used in python tests.  See
 # README of log_messages for why it's included.
-COMMON_PYTHON_PATH="\$(abs_top_builddir)/src/lib/python/isc/log_messages:\$(abs_top_srcdir)/src/lib/python:\$(abs_top_builddir)/src/lib/python"
+# lib/dns/python/.libs is necessary because __init__.py of isc package
+# automatically imports isc.datasrc, which then requires the DNS loadable
+# module.  #2145 should eliminate the need for it.
+COMMON_PYTHON_PATH="\$(abs_top_builddir)/src/lib/python/isc/log_messages:\$(abs_top_srcdir)/src/lib/python:\$(abs_top_builddir)/src/lib/python:\$(abs_top_builddir)/src/lib/dns/python/.libs"
 AC_SUBST(COMMON_PYTHON_PATH)
 
 # Check for python development environments
@@ -838,57 +839,22 @@ LIBS=$LIBS_SAVED
 #
 # Configure Boost header path
 #
-# If explicitly specified, use it.
-AC_ARG_WITH([boost-include],
-  AC_HELP_STRING([--with-boost-include=PATH],
-    [specify exact directory for Boost headers]),
-    [boost_include_path="$withval"])
-# If not specified, try some common paths.
-if test -z "$with_boost_include"; then
-	boostdirs="/usr/local /usr/pkg /opt /opt/local"
-	for d in $boostdirs
-	do
-		if test -f $d/include/boost/shared_ptr.hpp; then
-			boost_include_path=$d/include
-			break
-		fi
-	done
+AX_BOOST_FOR_BIND10
+# Boost offset_ptr is required in one library and not optional right now, so
+# we unconditionally fail here if it doesn't work.
+if test "$BOOST_OFFSET_PTR_FAILURE" = "yes"; then
+    AC_MSG_ERROR([Failed to compile a required header file.  Try upgrading Boost to 1.44 or higher (when using clang++) or specifying --without-werror.  See the ChangeLog entry for Trac no. 2147 for more details.])
 fi
-CPPFLAGS_SAVES="$CPPFLAGS"
-if test "${boost_include_path}" ; then
-	BOOST_INCLUDES="-I${boost_include_path}"
-	CPPFLAGS="$CPPFLAGS $BOOST_INCLUDES"
+
+# There's a known bug in FreeBSD ports for Boost that would trigger a false
+# warning in build with g++ and -Werror (we exclude clang++ explicitly to
+# avoid unexpected false positives).
+if test "$BOOST_NUMERIC_CAST_WOULDFAIL" = "yes" -a X"$werror_ok" = X1 -a $CLANGPP = "no"; then
+    AC_MSG_ERROR([Failed to compile a required header file.  If you are using FreeBSD and Boost installed via ports, retry with specifying --without-werror.  See the ChangeLog entry for Trac no. 1991 for more details.])
 fi
-AC_CHECK_HEADERS([boost/shared_ptr.hpp boost/foreach.hpp boost/interprocess/sync/interprocess_upgradable_mutex.hpp boost/date_time/posix_time/posix_time_types.hpp boost/bind.hpp boost/function.hpp],,
-  AC_MSG_ERROR([Missing required header files.]))
-
-# Detect whether Boost tries to use threads by default, and, if not,
-# make it sure explicitly.  In some systems the automatic detection
-# may depend on preceding header files, and if inconsistency happens
-# it could lead to a critical disruption.
-AC_MSG_CHECKING([whether Boost tries to use threads])
-AC_TRY_COMPILE([
-#include <boost/config.hpp>
-#ifdef BOOST_HAS_THREADS
-#error "boost will use threads"
-#endif],,
-[AC_MSG_RESULT(no)
- CPPFLAGS_BOOST_THREADCONF="-DBOOST_DISABLE_THREADS=1"],
-[AC_MSG_RESULT(yes)])
-
-# Boost offset_ptr is required in one library (not optional right now), and
-# it's known it doesn't compile on some platforms, depending on boost version,
-# its local configuration, and compiler.
-AC_MSG_CHECKING([Boost offset_ptr compiles])
-AC_TRY_COMPILE([
-#include <boost/interprocess/offset_ptr.hpp>
-],,
-[AC_MSG_RESULT(yes)],
-[AC_MSG_RESULT(no)
- AC_MSG_ERROR([Failed to compile a required header file.  Try upgrading Boost to 1.44 or higher (when using clang++) or specifying --without-werror.  See the ChangeLog entry for Trac no. 2147 for more details.])])
 
-CPPFLAGS="$CPPFLAGS_SAVES $CPPFLAGS_BOOST_THREADCONF"
-AC_SUBST(BOOST_INCLUDES)
+# Add some default CPP flags needed for Boost, identified by the AX macro.
+CPPFLAGS="$CPPFLAGS $CPPFLAGS_BOOST_THREADCONF"
 
 # I can't get some of the #include <asio.hpp> right without this
 # TODO: find the real cause of asio/boost wanting pthreads
@@ -1251,6 +1217,8 @@ AC_CONFIG_FILES([Makefile
                  src/lib/python/isc/server_common/tests/Makefile
                  src/lib/python/isc/sysinfo/Makefile
                  src/lib/python/isc/sysinfo/tests/Makefile
+                 src/lib/python/isc/statistics/Makefile
+                 src/lib/python/isc/statistics/tests/Makefile
                  src/lib/config/Makefile
                  src/lib/config/tests/Makefile
                  src/lib/config/tests/testdata/Makefile
@@ -1354,10 +1322,12 @@ AC_OUTPUT([doc/version.ent
            src/bin/usermgr/run_b10-cmdctl-usermgr.sh
            src/bin/usermgr/b10-cmdctl-usermgr.py
            src/bin/msgq/msgq.py
-           src/bin/msgq/tests/msgq_test
            src/bin/msgq/run_msgq.sh
            src/bin/auth/auth.spec.pre
            src/bin/auth/spec_config.h.pre
+           src/bin/auth/tests/testdata/example.zone
+           src/bin/auth/tests/testdata/example-base.zone
+           src/bin/auth/tests/testdata/example-nsec3.zone
            src/bin/dhcp4/spec_config.h.pre
            src/bin/dhcp6/spec_config.h.pre
            src/bin/tests/process_rename_test.py
@@ -1420,7 +1390,6 @@ AC_OUTPUT([doc/version.ent
            chmod +x src/bin/sysinfo/run_sysinfo.sh
            chmod +x src/bin/usermgr/run_b10-cmdctl-usermgr.sh
            chmod +x src/bin/msgq/run_msgq.sh
-           chmod +x src/bin/msgq/tests/msgq_test
            chmod +x src/lib/dns/gen-rdatacode.py
            chmod +x src/lib/log/tests/console_test.sh
            chmod +x src/lib/log/tests/destination_test.sh

doc/guide/bind10-guide.xml

@@ -1872,7 +1872,7 @@ tsig_keys/keys[0]   "example.key.:c2VjcmV0" string  (modified)
         Each rule is a name-value mapping (a dictionary, in the JSON
         terminology). Each rule must contain exactly one mapping called
         "action", which describes what should happen if the rule applies.
-        There may be more mappings, calld matches, which describe the
+        There may be more mappings, called matches, which describe the
         conditions under which the rule applies.
       </para>
 
@@ -3343,23 +3343,21 @@ then change those defaults with config set Resolver/forward_addresses[0]/address
 
     <note>
       <para>
-        As of November 2012, the DHCPv4 component is a
-        skeleton server. That means that while it is capable of
-        performing DHCP configuration, it is not fully functional.
-        In particular, it does not have a functional lease
-        database. This means that they will assign the same, fixed,
-        hardcoded addresses to any client that will ask. See <xref
-        linkend="dhcp4-limit"/> for a
-        detailed description.
+        As of January 2013, the DHCPv4 component is a work in progress.
+        That means that while it is capable of performing DHCP configuration,
+        it is not fully functional.  The server is able to offer,
+        assign, renew, release and reuse expired leases, but some of the
+        options are not configurable yet. In particular Router option is hardcoded.
+        This means that the server is not really usable in actual deployments
+        yet. See <xref linkend="dhcp4-limit"/> for a detailed description.
       </para>
     </note>
 
     <section id="dhcp4-usage">
       <title>DHCPv4 Server Usage</title>
       <para>BIND 10 has provided the DHCPv4 server component since December
-      2011. It is a skeleton server and can be described as an early
-      prototype that is not fully functional yet. It is mature enough
-      to conduct first tests in lab environment, but it has
+      2011. It is current experimental implementation and is not fully functional
+      yet. It is mature enough to conduct tests in lab environment, but it has
       significant limitations. See <xref linkend="dhcp4-limit"/> for
       details.
       </para>
@@ -3480,24 +3478,41 @@ Dhcp4/subnet4	         []     list    (default)</screen>
       </para>
 
       <para>
-        Note: Although configuration is now accepted, it is not internally used
-        by they server yet.  At this stage of development, the only way to alter
-        server configuration is to modify its source code. To do so, please edit
+        Note: Although configuration is now accepted, some parts of it is not internally used
+        by they server yet. Address pools are used, but option definitons are not.
+        The only way to alter some options (e.g. Router Option or DNS servers and Domain name)
+        is to modify source code. To do so, please edit
         src/bin/dhcp6/dhcp4_srv.cc file, modify the following parameters and
         recompile:
         <screen>
-const std::string HARDCODED_LEASE = "192.0.2.222"; // assigned lease
-const std::string HARDCODED_NETMASK = "255.255.255.0";
-const uint32_t    HARDCODED_LEASE_TIME = 60; // in seconds
 const std::string HARDCODED_GATEWAY = "192.0.2.1";
 const std::string HARDCODED_DNS_SERVER = "192.0.2.2";
-const std::string HARDCODED_DOMAIN_NAME = "isc.example.com";
-const std::string HARDCODED_SERVER_ID = "192.0.2.1";</screen>
+const std::string HARDCODED_DOMAIN_NAME = "isc.example.com";</screen>
 
         Lease database and configuration support is planned for end of 2012.
       </para>
     </section>
 
+    <section id="dhcp4-serverid">
+      <title>Server Identifier in DHCPv4</title>
+      <para>The DHCPv4 protocol uses a "server identifier" for clients to be able
+      to discriminate between several servers present on the same link: this
+      value is an IPv4 address of the server. When started for the first time,
+      the DHCPv4 server will choose one of its IPv4 addresses as its server-id,
+      and store the chosen value to a file. (The file is named b10-dhcp4-serverid and is
+      stored in the "local state directory".  This is set during installation
+      when "configure" is run, and can be changed by using "--localstatedir"
+      on the "configure" command line.)  That file will be read by the server
+      and the contained value used whenever the server is subsequently started.
+      </para>
+      <para>
+        It is unlikely that this parameter needs to be changed. If such a need
+        arises, please stop the server, edit the file and restart the server.
+        It is a text file that should contain an IPv4 address. Spaces are
+        ignored.  No extra characters are allowed in this file.
+      </para>
+    </section>
+
     <section id="dhcp4-std">
       <title>Supported standards</title>
       <para>The following standards and draft standards are currently
@@ -3505,7 +3520,7 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";</screen>
       <itemizedlist>
           <listitem>
             <simpara>RFC2131: Supported messages are DISCOVER, OFFER,
-            REQUEST, and ACK.</simpara>
+            REQUEST, ACK, NAK, RELEASE.</simpara>
           </listitem>
           <listitem>
             <simpara>RFC2132: Supported options are: PAD (0),
@@ -3513,6 +3528,10 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";</screen>
             Domain Name (15), DNS Servers (6), IP Address Lease Time
             (51), Subnet mask (1), and Routers (3).</simpara>
           </listitem>
+          <listitem>
+            <simpara>RFC6842: Server responses include client-id option
+            if client sent it in its message.</simpara>
+          </listitem>
       </itemizedlist>
     </section>
 
@@ -3533,20 +3552,6 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";</screen>
             communication).</simpara>
           </listitem>
           <listitem>
-            <simpara><command>b10-dhcp4</command> provides a single,
-            fixed, hardcoded lease to any client that asks.  There is
-            no lease manager implemented. If two clients request
-            addresses, they will both get the same fixed
-            address.</simpara>
-          </listitem>
-          <listitem>
-            <simpara><command>b10-dhcp4</command> does not support any
-            configuration mechanisms yet. The whole configuration is
-            currently hardcoded. The only way to tweak configuration
-            is to directly modify source code. See see <xref
-            linkend="dhcp4-config"/> for details.</simpara>
-          </listitem>
-          <listitem>
             <simpara>Upon start, the server will open sockets on all
             interfaces that are not loopback, are up and running and
             have IPv4 address.</simpara>
@@ -3574,9 +3579,9 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";</screen>
             sending ICMP echo request.</simpara>
           </listitem>
           <listitem>
-            <simpara>Address renewal (RENEW), rebinding (REBIND),
-            confirmation (CONFIRM), duplication report (DECLINE) and
-            release (RELEASE) are not supported yet.</simpara>
+            <simpara>Address rebinding (REQUEST/Rebinding), confirmation
+            (CONFIRM) and duplication report (DECLINE) are not supported
+            yet.</simpara>
           </listitem>
           <listitem>
             <simpara>DNS Update is not supported yet.</simpara>
@@ -3852,6 +3857,31 @@ Dhcp6/subnet6	         []     list    (default)</screen>
       </note>
     </section>
 
+    <section id="dhcp6-serverid">
+      <title>Server Identifier in DHCPv6</title>
+      <para>The DHCPv6 protocol uses a "server identifier" (also known
+      as a DUID) for clients to be able to discriminate between several
+      servers present on the same link.  There are several types of
+      DUIDs defined, but RFC 3315 instructs servers to use DUID-LLT if
+      possible. This format consists of a link-layer (MAC) address and a
+      timestamp. When started for the first time, the DHCPv6 server will
+      automatically generate such a DUID and store the chosen value to
+      a file (The file is named b10-dhcp6-serverid and is stored in the
+      "local state directory".  This is set during installation when
+      "configure" is run, and can be changed by using "--localstatedir"
+      on the "configure" command line.)  That file will be read by the server
+      and the contained value used whenever the server is subsequently started.
+      </para>
+      <para>
+        It is unlikely that this parameter needs to be changed. If such a need
+        arises, please stop the server, edit the file and start the server
+        again. It is a text file that contains double digit hexadecimal values
+        separated by colons. This format is similar to typical MAC address
+        format. Spaces are ignored. No extra characters are allowed in this
+        file.
+      </para>
+    </section>
+
     <section id="dhcp6-std">
       <title>Supported DHCPv6 Standards</title>
       <para>The following standards and draft standards are currently
@@ -3871,7 +3901,7 @@ Dhcp6/subnet6	         []     list    (default)</screen>
     <section id="dhcp6-limit">
       <title>DHCPv6 Server Limitations</title>
       <para> These are the current limitations and known problems
-      with the the DHCPv6 server
+      with the DHCPv6 server
       software. Most of them are reflections of the early stage of
       development and should be treated as <quote>not implemented
       yet</quote>, rather than actual limitations.</para>
@@ -4156,7 +4186,7 @@ specify module-wide logging and see what appears...
           If there are multiple logger specifications in the
           configuration that might match a particular logger, the
           specification with the more specific logger name takes
-          precedence. For example, if there are entries for for
+          precedence. For example, if there are entries for
           both <quote>*</quote> and <quote>Resolver</quote>, the
           resolver module &mdash; and all libraries it uses &mdash;
           will log messages according to the configuration in the

examples/README

@@ -15,7 +15,7 @@ the "m4" subdirectory as a template for your own project.  The key is
 to call the AX_ISC_BIND10 function (as the sample configure.ac does)
 from your configure.ac.  Then it will check the availability of
 necessary stuff and set some corresponding AC variables.  You can then
-use the resulting variables in your Makefile.in or Makefile.ac.
+use the resulting variables in your Makefile.in or Makefile.am.
 
 If you use automake, don't forget adding the following line to the top
 level Makefile.am:

m4macros/ax_boost_for_bind10.m4

@@ -0,0 +1,112 @@
+dnl @synopsis AX_BOOST_FOR_BIND10
+dnl
+dnl Test for the Boost C++ header files intended to be used within BIND 10
+dnl
+dnl If no path to the installed boost header files is given via the
+dnl --with-boost-include option,  the macro searchs under
+dnl /usr/local /usr/pkg /opt /opt/local directories.
+dnl If it cannot detect any workable path for Boost, this macro treats it
+dnl as a fatal error (so it cannot be called if the availability of Boost
+dnl is optional).
+dnl
+dnl This macro also tries to identify some known portability issues, and
+dnl sets corresponding variables so the caller can react to (or ignore,
+dnl depending on other configuration) specific issues appropriately.
+dnl
+dnl This macro calls:
+dnl
+dnl   AC_SUBST(BOOST_INCLUDES)
+dnl
+dnl And possibly sets:
+dnl   CPPFLAGS_BOOST_THREADCONF should be added to CPPFLAGS by caller
+dnl   BOOST_OFFSET_PTR_WOULDFAIL set to "yes" if offset_ptr would cause build
+dnl                              error; otherwise set to "no"
+dnl   BOOST_NUMERIC_CAST_WOULDFAIL set to "yes" if numeric_cast would cause
+dnl                                build error; otherwise set to "no"
+dnl
+
+AC_DEFUN([AX_BOOST_FOR_BIND10], [
+AC_LANG_SAVE
+AC_LANG([C++])
+
+#
+# Configure Boost header path
+#
+# If explicitly specified, use it.
+AC_ARG_WITH([boost-include],
+  AC_HELP_STRING([--with-boost-include=PATH],
+    [specify exact directory for Boost headers]),
+    [boost_include_path="$withval"])
+# If not specified, try some common paths.
+if test -z "$with_boost_include"; then
+	boostdirs="/usr/local /usr/pkg /opt /opt/local"
+	for d in $boostdirs
+	do
+		if test -f $d/include/boost/shared_ptr.hpp; then
+			boost_include_path=$d/include
+			break
+		fi
+	done
+fi
+
+# Check the path with some specific headers.
+CPPFLAGS_SAVED="$CPPFLAGS"
+if test "${boost_include_path}" ; then
+	BOOST_INCLUDES="-I${boost_include_path}"
+	CPPFLAGS="$CPPFLAGS $BOOST_INCLUDES"
+fi
+AC_CHECK_HEADERS([boost/shared_ptr.hpp boost/foreach.hpp boost/interprocess/sync/interprocess_upgradable_mutex.hpp boost/date_time/posix_time/posix_time_types.hpp boost/bind.hpp boost/function.hpp],,
+  AC_MSG_ERROR([Missing required header files.]))
+
+# Detect whether Boost tries to use threads by default, and, if not,
+# make it sure explicitly.  In some systems the automatic detection
+# may depend on preceding header files, and if inconsistency happens
+# it could lead to a critical disruption.
+AC_MSG_CHECKING([whether Boost tries to use threads])
+AC_TRY_COMPILE([
+#include <boost/config.hpp>
+#ifdef BOOST_HAS_THREADS
+#error "boost will use threads"
+#endif],,
+[AC_MSG_RESULT(no)
+ CPPFLAGS_BOOST_THREADCONF="-DBOOST_DISABLE_THREADS=1"],
+[AC_MSG_RESULT(yes)])
+
+# Boost offset_ptr is known to not compile on some platforms, depending on
+# boost version, its local configuration, and compiler.  Detect it.
+AC_MSG_CHECKING([Boost offset_ptr compiles])
+AC_TRY_COMPILE([
+#include <boost/interprocess/offset_ptr.hpp>
+],,
+[AC_MSG_RESULT(yes)
+ BOOST_OFFSET_PTR_WOULDFAIL=no],
+[AC_MSG_RESULT(no)
+ BOOST_OFFSET_PTR_WOULDFAIL=yes])
+
+# Detect build failure case known to happen with Boost installed via
+# FreeBSD ports
+if test "X$GXX" = "Xyes"; then
+   CXXFLAGS_SAVED="$CXXFLAGS"
+   CXXFLAGS="$CXXFLAGS -Werror"
+
+   AC_MSG_CHECKING([Boost numeric_cast compiles with -Werror])
+   AC_TRY_COMPILE([
+   #include <boost/numeric/conversion/cast.hpp>
+   ],[
+   return (boost::numeric_cast<short>(0));
+   ],[AC_MSG_RESULT(yes)
+      BOOST_NUMERIC_CAST_WOULDFAIL=no],
+   [AC_MSG_RESULT(no)
+    BOOST_NUMERIC_CAST_WOULDFAIL=yes])
+
+   CXXFLAGS="$CXXFLAGS_SAVED"
+else
+  # This doesn't matter for non-g++
+  BOOST_NUMERIC_CAST_WOULDFAIL=no
+fi
+
+AC_SUBST(BOOST_INCLUDES)
+
+CPPFLAGS="$CPPFLAGS_SAVED"
+AC_LANG_RESTORE
+])dnl AX_BOOST_FOR_BIND10

src/bin/auth/auth_messages.mes

@@ -141,7 +141,7 @@ The specific problem is printed in the log message.
 The thread for maintaining data source clients has received a command to
 reconfigure, and has now started this process.
 
-% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_SUCCESS data source reconfiguration completed succesfully
+% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_SUCCESS data source reconfiguration completed successfully
 The thread for maintaining data source clients has finished reconfiguring
 the data source clients, and is now running with the new configuration.
 
@@ -169,7 +169,7 @@ probably better to stop and restart it.
 
 % AUTH_DATA_SOURCE data source database file: %1
 This is a debug message produced by the authoritative server when it accesses a
-datebase data source, listing the file that is being accessed.
+database data source, listing the file that is being accessed.
 
 % AUTH_DNS_SERVICES_CREATED DNS services created
 This is a debug message indicating that the component that will handling
@@ -184,7 +184,7 @@ reason for the failure is given in the message.) The server will drop the
 packet.
 
 % AUTH_INVALID_STATISTICS_DATA invalid specification of statistics data specified
-An error was encountered when the authoritiative server specified
+An error was encountered when the authoritative server specified
 statistics data which is invalid for the auth specification file.
 
 % AUTH_LOAD_TSIG loading TSIG keys
@@ -208,7 +208,7 @@ requests to b10-ddns) to handle it, but it failed.  The authoritative
 server returns SERVFAIL to the client on behalf of the separate
 process.  The error could be configuration mismatch between b10-auth
 and the recipient component, or it may be because the requests are
-coming too fast and the receipient process cannot keep up with the
+coming too fast and the recipient process cannot keep up with the
 rate, or some system level failure.  In either case this means the
 BIND 10 system is not working as expected, so the administrator should
 look into the cause and address the issue.  The log message includes

src/bin/auth/tests/.gitignore

@@ -1 +1,3 @@
 /run_unittests
+/example_base_inc.cc
+/example_nsec3_inc.cc

src/bin/auth/tests/Makefile.am

@@ -7,7 +7,8 @@ AM_CPPFLAGS += -I$(top_builddir)/src/lib/cc
 AM_CPPFLAGS += $(BOOST_INCLUDES)
 AM_CPPFLAGS += -DAUTH_OBJ_DIR=\"$(abs_top_builddir)/src/bin/auth\"
 AM_CPPFLAGS += -DTEST_DATA_DIR=\"$(abs_top_srcdir)/src/lib/testutils/testdata\"
-AM_CPPFLAGS += -DTEST_OWN_DATA_DIR=\"$(abs_top_srcdir)/src/bin/auth/tests/testdata\"
+AM_CPPFLAGS += -DTEST_OWN_DATA_DIR=\"$(abs_srcdir)/testdata\"
+AM_CPPFLAGS += -DTEST_OWN_DATA_BUILDDIR=\"$(abs_builddir)/testdata\"
 AM_CPPFLAGS += -DTEST_DATA_BUILDDIR=\"$(abs_top_builddir)/src/lib/testutils/testdata\"
 AM_CPPFLAGS += -DDSRC_DIR=\"$(abs_top_builddir)/src/lib/datasrc\"
 AM_CPPFLAGS += -DPLUGIN_DATA_PATH=\"$(abs_top_builddir)/src/bin/cfgmgr/plugins\"
@@ -50,7 +51,6 @@ run_unittests_SOURCES += config_syntax_unittest.cc
 run_unittests_SOURCES += command_unittest.cc
 run_unittests_SOURCES += common_unittest.cc
 run_unittests_SOURCES += query_unittest.cc
-run_unittests_SOURCES += query_inmemory_unittest.cc
 run_unittests_SOURCES += statistics_unittest.cc
 run_unittests_SOURCES += test_datasrc_clients_mgr.h test_datasrc_clients_mgr.cc
 run_unittests_SOURCES += datasrc_clients_builder_unittest.cc
@@ -81,6 +81,39 @@ run_unittests_LDADD += $(top_builddir)/src/lib/util/threads/libb10-threads.la
 run_unittests_LDADD += $(GTEST_LDADD)
 run_unittests_LDADD += $(SQLITE_LIBS)
 
+# The following are definitions for auto-generating test data for query
+# tests.
+BUILT_SOURCES = example_base_inc.cc example_nsec3_inc.cc
+BUILT_SOURCES += testdata/example-base.sqlite3
+BUILT_SOURCES += testdata/example-nsec3.sqlite3
+
+EXTRA_DIST = gen-query-testdata.py
+
+CLEANFILES += example_base_inc.cc example_nsec3_inc.cc
+
+example_base_inc.cc: $(srcdir)/testdata/example-base-inc.zone
+	$(PYTHON) $(srcdir)/gen-query-testdata.py \
+		$(srcdir)/testdata/example-base-inc.zone example_base_inc.cc
+
+example_nsec3_inc.cc: $(srcdir)/testdata/example-nsec3-inc.zone
+	$(PYTHON) $(srcdir)/gen-query-testdata.py \
+		$(srcdir)/testdata/example-nsec3-inc.zone example_nsec3_inc.cc
+
+testdata/example-common-inc.zone: $(srcdir)/testdata/example-common-inc-template.zone
+	$(top_srcdir)/install-sh -c \
+		$(srcdir)/testdata/example-common-inc-template.zone \
+		testdata/example-common-inc.zone
+
+testdata/example-base.sqlite3: testdata/example-base.zone testdata/example-common-inc.zone
+	$(SHELL) $(top_builddir)/src/bin/loadzone/run_loadzone.sh \
+		-c "{\"database_file\": \"$(builddir)/testdata/example-base.sqlite3\"}" \
+		example.com testdata/example-base.zone
+
+testdata/example-nsec3.sqlite3: testdata/example-nsec3.zone testdata/example-common-inc.zone
+	$(SHELL) $(top_builddir)/src/bin/loadzone/run_loadzone.sh \
+		-c "{\"database_file\": \"$(builddir)/testdata/example-nsec3.sqlite3\"}" \
+		example.com testdata/example-nsec3.zone
+
 check-local:
 	B10_FROM_BUILD=${abs_top_builddir} ./run_unittests
 

src/bin/auth/tests/auth_srv_unittest.cc

@@ -1286,12 +1286,12 @@ TEST_F(AuthSrvTest, queryCounterUnexpected) {
     createRequestPacket(request_message, IPPROTO_UDP);
 
     // Modify the message.
-    delete io_message;
-    endpoint = IOEndpoint::create(IPPROTO_UDP,
-                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 53210);
-    io_message = new IOMessage(request_renderer.getData(),
-                               request_renderer.getLength(),
-                               getDummyUnknownSocket(), *endpoint);
+    endpoint.reset(IOEndpoint::create(IPPROTO_UDP,
+                                      IOAddress(DEFAULT_REMOTE_ADDRESS),
+                                      53210));
+    io_message.reset(new IOMessage(request_renderer.getData(),
+                                   request_renderer.getLength(),
+                                   getDummyUnknownSocket(), *endpoint));
 
     EXPECT_FALSE(dnsserv.hasAnswer());
 }
@@ -1716,9 +1716,20 @@ void
 checkAddrPort(const struct sockaddr& actual_sa,
               const string& expected_addr, uint16_t expected_port)
 {
+    // ASIO does not set as_len, which is not a problem on most
+    // systems, but it will make getnameinfo() fail on NetBSD 4
+    // So we make a copy and if the field is available, we set it
+    const socklen_t sa_len = getSALength(actual_sa);
+    struct sockaddr_storage ss;
+    memcpy(&ss, &actual_sa, sa_len);
+
+    struct sockaddr* sa = convertSockAddr(&ss);
+#ifdef HAVE_SA_LEN
+    sa->sa_len = sa_len;
+#endif
     char hbuf[NI_MAXHOST], sbuf[NI_MAXSERV];
-    const int error = getnameinfo(&actual_sa, getSALength(actual_sa), hbuf,
-                                  sizeof(hbuf), sbuf, sizeof(sbuf),
+    const int error = getnameinfo(sa, sa_len, hbuf, sizeof(hbuf),
+                                  sbuf, sizeof(sbuf),
                                   NI_NUMERICHOST | NI_NUMERICSERV);
     if (error != 0) {
         isc_throw(isc::Unexpected, "getnameinfo failed: " <<

src/bin/auth/tests/gen-query-testdata.py

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2012  Internet Systems Consortium, Inc. ("ISC")
+#
+# Permission to use, copy, modify, and/or distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+# AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+# PERFORMANCE OF THIS SOFTWARE.
+
+"""\
+This is a supplemental script to auto generate test data in the form of
+C++ source code from a DNS zone file.
+
+Usage: python gen-query-testdata.py source_file output-cc-file
+
+The usage doesn't matter much, though, because it's expected to be invoked
+from Makefile, and that would be only use case of this script.
+"""
+
+import sys
+import re
+
+# Markup for variable definition
+re_start_rr = re.compile('^;var=(.*)')
+
+# Skip lines starting with ';' (comments) or empty lines.  re_start_rr
+# will also match this expression, so it should be checked first.
+re_skip = re.compile('(^;)|(^\s*$)')
+
+def parse_input(input_file):
+    '''Build an internal list of RR data from the input source file.
+
+    It generates a list of (variable_name, list of RR) tuples, where
+    variable_name is the expected C++ variable name for the subsequent RRs
+    if they are expected to be named.  It can be an empty string if the RRs
+    are only expected to appear in the zone file.
+    The second element of the tuple is a list of strings, each of which
+    represents a single RR, e.g., "example.com 3600 IN A 192.0.2.1".
+
+    '''
+    result = []
+    rrs = None
+    with open(input_file) as f:
+        for line in f:
+            m = re_start_rr.match(line)
+            if m:
+                if rrs is not None:
+                    result.append((rr_varname, rrs))
+                rrs = []
+                rr_varname = m.group(1)
+            elif re_skip.match(line):
+                continue
+            else:
+                rrs.append(line.rstrip('\n'))
+
+        # if needed, store the last RRs (they are not followed by 'var=' mark)
+        if rrs is not None:
+            result.append((rr_varname, rrs))
+
+    return result
+
+def generate_variables(out_file, rrsets_data):
+    '''Generate a C++ source file containing a C-string variables for RRs.
+
+    This produces a definition of C-string for each RRset that is expected
+    to be named as follows:
+    const char* const var_name =
+        "example.com. 3600 IN A 192.0.2.1\n"
+        "example.com. 3600 IN A 192.0.2.2\n";
+
+    Escape character '\' in the string will be further escaped so it will
+    compile.
+
+    '''
+    with open(out_file, 'w') as out:
+        for (var_name, rrs) in rrsets_data:
+            if len(var_name) > 0:
+                out.write('const char* const ' + var_name + ' =\n')
+                # Combine all RRs, escaping '\' as a C-string
+                out.write('\n'.join(['    \"%s\\n\"' %
+                                     (rr.replace('\\', '\\\\'))
+                                     for rr in rrs]))
+                out.write(';\n')
+
+if __name__ == "__main__":
+    if len(sys.argv) < 3:
+        sys.stderr.write('gen-query-testdata.py require 2 args\n')
+        sys.exit(1)
+    rrsets_data = parse_input(sys.argv[1])
+    generate_variables(sys.argv[2], rrsets_data)
+

src/bin/auth/tests/query_inmemory_unittest.cc

@@ -1,123 +0,0 @@
-// Copyright (C) 2012  Internet Systems Consortium, Inc. ("ISC")
-//
-// Permission to use, copy, modify, and/or distribute this software for any
-// purpose with or without fee is hereby granted, provided that the above
-// copyright notice and this permission notice appear in all copies.
-//
-// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
-// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
-// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
-// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
-// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
-// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
-// PERFORMANCE OF THIS SOFTWARE.
-
-#include <dns/name.h>
-#include <dns/message.h>
-#include <dns/rcode.h>
-#include <dns/opcode.h>
-
-#include <cc/data.h>
-
-#include <datasrc/client_list.h>
-
-#include <auth/query.h>
-
-#include <testutils/dnsmessage_test.h>
-
-#include <gtest/gtest.h>
-
-#include <string>
-
-using namespace isc::dns;
-using namespace isc::auth;
-using namespace isc::testutils;
-using isc::datasrc::ConfigurableClientList;
-using std::string;
-
-namespace {
-
-// The DNAME to do tests against
-const char* const dname_txt =
-    "dname.example.com. 3600 IN DNAME "
-    "somethinglong.dnametarget.example.com.\n";
-// This is not inside the zone, this is created at runtime
-const char* const synthetized_cname_txt =
-    "www.dname.example.com. 3600 IN CNAME "
-    "www.somethinglong.dnametarget.example.com.\n";
-
-// This is a subset of QueryTest using (subset of) the same test data, but
-// with the production in-memory data source.  Both tests should be eventually
-// unified to avoid duplicates.
-class InMemoryQueryTest : public ::testing::Test {
-protected:
-    InMemoryQueryTest() : list(RRClass::IN()), response(Message::RENDER) {
-        response.setRcode(Rcode::NOERROR());
-        response.setOpcode(Opcode::QUERY());
-        list.configure(isc::data::Element::fromJSON(
-                           "[{\"type\": \"MasterFiles\","
-                           "  \"cache-enable\": true, "
-                           "  \"params\": {\"example.com\": \"" +
-                           string(TEST_OWN_DATA_DIR "/example.zone") +
-                           "\"}}]"), true);
-    }
-
-    ConfigurableClientList list;
-    Message response;
-    Query query;
-};
-
-// A wrapper to check resulting response message commonly used in
-// tests below.
-// check_origin needs to be specified only when the authority section has
-// an SOA RR.  The interface is not generic enough but should be okay
-// for our test cases in practice.
-void
-responseCheck(Message& response, const isc::dns::Rcode& rcode,
-              unsigned int flags, const unsigned int ancount,
-              const unsigned int nscount, const unsigned int arcount,
-              const char* const expected_answer,
-              const char* const expected_authority,
-              const char* const expected_additional,
-              const Name& check_origin = Name::ROOT_NAME())
-{
-    // In our test cases QID, Opcode, and QDCOUNT should be constant, so
-    // we don't bother the test cases specifying these values.
-    headerCheck(response, response.getQid(), rcode, Opcode::QUERY().getCode(),
-                flags, 0, ancount, nscount, arcount);
-    if (expected_answer != NULL) {
-        rrsetsCheck(expected_answer,
-                    response.beginSection(Message::SECTION_ANSWER),
-                    response.endSection(Message::SECTION_ANSWER),
-                    check_origin);
-    }
-    if (expected_authority != NULL) {
-        rrsetsCheck(expected_authority,
-                    response.beginSection(Message::SECTION_AUTHORITY),
-                    response.endSection(Message::SECTION_AUTHORITY),
-                    check_origin);
-    }
-    if (expected_additional != NULL) {
-        rrsetsCheck(expected_additional,
-                    response.beginSection(Message::SECTION_ADDITIONAL),
-                    response.endSection(Message::SECTION_ADDITIONAL));
-    }
-}
-
-/*
- * Test a query under a domain with DNAME. We should get a synthetized CNAME
- * as well as the DNAME.
- *
- * TODO: Once we have CNAME chaining, check it works with synthetized CNAMEs
- * as well. This includes tests pointing inside the zone, outside the zone,
- * pointing to NXRRSET and NXDOMAIN cases (similarly as with CNAME).
- */
-TEST_F(InMemoryQueryTest, DNAME) {
-    query.process(list, Name("www.dname.example.com"), RRType::A(),
-                  response);
-
-    responseCheck(response, Rcode::NOERROR(), AA_FLAG, 2, 0, 0,
-        (string(dname_txt) + synthetized_cname_txt).c_str(),
-        NULL, NULL);
-}
-}

src/bin/auth/tests/testdata/.gitignore

@@ -6,3 +6,13 @@
 /shortanswer_fromWire.wire
 /simplequery_fromWire.wire
 /simpleresponse_fromWire.wire
+/example-base.sqlite3
+/example-base.sqlite3.copied
+/example-base.zone
+/example-base.zone
+/example-common-inc.zone
+/example-nsec3-inc.zone
+/example-nsec3.sqlite3
+/example-nsec3.sqlite3.copied
+/example-nsec3.zone
+/example.zone

src/bin/auth/tests/testdata/Makefile.am

@@ -1,4 +1,6 @@
-CLEANFILES = *.wire
+CLEANFILES = *.wire *.copied
+CLEANFILES += example-base.sqlite3 example-nsec3.sqlite3
+CLEANFILES += example-common-inc.zone
 
 BUILT_SOURCES = badExampleQuery_fromWire.wire examplequery_fromWire.wire
 BUILT_SOURCES += iqueryresponse_fromWire.wire multiquestion_fromWire.wire
@@ -24,5 +26,8 @@ EXTRA_DIST += example.com
 EXTRA_DIST += example.zone
 EXTRA_DIST += example.sqlite3
 
+EXTRA_DIST += example-base-inc.zone example-nsec3-inc.zone
+EXTRA_DIST += example-common-inc-template.zone
+
 .spec.wire:
 	$(PYTHON) $(top_builddir)/src/lib/util/python/gen_wiredata.py -o $@ $<

src/bin/auth/tests/testdata/example-base-inc.zone

@@ -0,0 +1,236 @@
+;; This file defines a set of RRs commonly used in query tests in the
+;; form of standard master zone file.
+;;
+;; It's a sequence of the following pattern:
+;; ;var=<var_name>
+;; RR_1
+;; RR_2
+;; ..
+;; RR_n
+;;
+;; where var_name is a string that can be used as a variable name in a
+;; C/C++ source file or an empty string.  RR_x is a single-line
+;; textual representation of an arbitrary DNS RR.
+;;
+;; If var_name is non empty, the generator script will define a C
+;; variable of C-string type for that set of RRs so that it can be referred
+;; to in the test source file.
+;;
+;; Note that lines beginning ';var=' is no different from other
+;; comment lines as a zone file.  It has special meaning only for the
+;; generator script.  Obviously, real comment lines cannot begin with
+;; ';var=' (which should be less likely to happen in practice though).
+;;
+;; These RRs will be loaded into in-memory data source in that order.
+;; Note that it may impose stricter restriction on the order of RRs.
+;; In general, each RRset of the same name and type and its RRSIG (if
+;; any) is expected to be grouped.
+
+;var=soa_txt
+example.com. 3600 IN SOA . . 1 0 0 0 0
+;var=zone_ns_txt
+example.com. 3600 IN NS glue.delegation.example.com.
+example.com. 3600 IN NS noglue.example.com.
+example.com. 3600 IN NS example.net.
+
+;var=
+example.com. 3600 IN RRSIG SOA 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+example.com. 3600 IN RRSIG NS 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; Note: the position of the next RR is tricky.  It's placed here to
+;; be grouped with the subsequent A RR of the name.  But we also want
+;; to group the A RR with other RRs of a different owner name, so the RRSIG
+;; cannot be placed after the A RR.  The empty 'var=' specification is
+;; not necessary here, but in case we want to reorganize the ordering
+;; (in which case it's more likely to be needed), we keep it here.
+;var=
+noglue.example.com. 3600 IN RRSIG A 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;var=ns_addrs_txt
+noglue.example.com. 3600 IN A 192.0.2.53
+glue.delegation.example.com. 3600 IN A 192.0.2.153
+glue.delegation.example.com. 3600 IN AAAA 2001:db8::53
+
+;var=
+glue.delegation.example.com. 3600 IN RRSIG A 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+glue.delegation.example.com. 3600 IN RRSIG AAAA 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;var=delegation_txt
+delegation.example.com. 3600 IN NS glue.delegation.example.com.
+delegation.example.com. 3600 IN NS noglue.example.com.
+delegation.example.com. 3600 IN NS cname.example.com.
+delegation.example.com. 3600 IN NS example.org.
+
+;var=
+delegation.example.com. 3600 IN RRSIG DS 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; Borrowed from the RFC4035
+;var=delegation_ds_txt
+delegation.example.com. 3600 IN DS 57855 5 1 B6DCD485719ADCA18E5F3D48A2331627FDD3 636B
+;var=mx_txt
+mx.example.com. 3600 IN MX 10 www.example.com.
+mx.example.com. 3600 IN MX 20 mailer.example.org.
+mx.example.com. 3600 IN MX 30 mx.delegation.example.com.
+;var=www_a_txt
+www.example.com. 3600 IN A 192.0.2.80
+
+;var=
+www.example.com. 3600 IN RRSIG A 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;var=cname_txt
+cname.example.com. 3600 IN CNAME www.example.com.
+;var=cname_nxdom_txt
+cnamenxdom.example.com. 3600 IN CNAME nxdomain.example.com.
+;; CNAME Leading out of zone
+;var=cname_out_txt
+cnameout.example.com. 3600 IN CNAME www.example.org.
+;; The DNAME to do tests against
+;var=dname_txt
+dname.example.com. 3600 IN DNAME somethinglong.dnametarget.example.com.
+;; Some data at the dname node (allowed by RFC 2672)
+;var=dname_a_txt
+dname.example.com. 3600 IN A 192.0.2.5
+;; This is not inside the zone, this is created at runtime
+;; www.dname.example.com. 3600 IN CNAME www.somethinglong.dnametarget.example.com.
+;; The rest of data won't be referenced from the test cases.
+;var=other_zone_rrs
+cnamemailer.example.com. 3600 IN CNAME www.example.com.
+cnamemx.example.com. 3600 IN MX 10 cnamemailer.example.com.
+mx.delegation.example.com. 3600 IN A 192.0.2.100
+;; Wildcards
+;var=wild_txt
+*.wild.example.com. 3600 IN A 192.0.2.7
+;var=nsec_wild_txt
+*.wild.example.com. 3600 IN NSEC www.example.com. A NSEC RRSIG
+
+;var=
+*.wild.example.com. 3600 IN RRSIG A 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+*.wild.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;var=cnamewild_txt
+*.cnamewild.example.com. 3600 IN CNAME www.example.org.
+;var=nsec_cnamewild_txt
+*.cnamewild.example.com. 3600 IN NSEC delegation.example.com. CNAME NSEC RRSIG
+
+;var=
+*.cnamewild.example.com. 3600 IN RRSIG CNAME 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+*.cnamewild.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; Wildcard_nxrrset
+;var=wild_txt_nxrrset
+*.uwild.example.com. 3600 IN A 192.0.2.9
+;var=nsec_wild_txt_nxrrset
+*.uwild.example.com. 3600 IN NSEC www.uwild.example.com. A NSEC RRSIG
+;var=
+*.uwild.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+;var=wild_txt_next
+www.uwild.example.com. 3600 IN A 192.0.2.11
+;var=
+www.uwild.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+;var=nsec_wild_txt_next
+www.uwild.example.com. 3600 IN NSEC *.wild.example.com. A NSEC RRSIG
+;; Wildcard empty
+;var=empty_txt
+b.*.t.example.com. 3600 IN A 192.0.2.13
+;var=nsec_empty_txt
+b.*.t.example.com. 3600 IN NSEC *.uwild.example.com. A NSEC RRSIG
+
+;var=
+b.*.t.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;var=empty_prev_txt
+t.example.com. 3600 IN A 192.0.2.15
+;var=nsec_empty_prev_txt
+t.example.com. 3600 IN NSEC b.*.t.example.com. A NSEC RRSIG
+
+;var=
+t.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; Used in NXDOMAIN proof test.  We are going to test some unusual case where
+;; the best possible wildcard is below the "next domain" of the NSEC RR that
+;; proves the NXDOMAIN, i.e.,
+;; mx.example.com. (exist)
+;; (.no.example.com. (qname, NXDOMAIN)
+;; ).no.example.com. (exist)
+;; *.no.example.com. (best possible wildcard, not exist)
+;var=no_txt
+\).no.example.com. 3600 IN AAAA 2001:db8::53
+;; NSEC records.
+;var=nsec_apex_txt
+example.com. 3600 IN NSEC cname.example.com. NS SOA NSEC RRSIG
+;var=
+example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+;var=nsec_mx_txt
+mx.example.com. 3600 IN NSEC \).no.example.com. MX NSEC RRSIG
+
+;var=
+mx.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;var=nsec_no_txt
+\).no.example.com. 3600 IN NSEC nz.no.example.com. AAAA NSEC RRSIG
+
+;var=
+\).no.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; We'll also test the case where a single NSEC proves both NXDOMAIN and the
+;; non existence of wildcard.  The following records will be used for that
+;; test.
+;; ).no.example.com. (exist, whose NSEC proves everything)
+;; *.no.example.com. (best possible wildcard, not exist)
+;; nx.no.example.com. (NXDOMAIN)
+;; nz.no.example.com. (exist)
+;var=nz_txt
+nz.no.example.com. 3600 IN AAAA 2001:db8::5300
+;var=nsec_nz_txt
+nz.no.example.com. 3600 IN NSEC noglue.example.com. AAAA NSEC RRSIG
+;var=nsec_nxdomain_txt
+noglue.example.com. 3600 IN NSEC nonsec.example.com. A
+
+;var=
+noglue.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; NSEC for the normal NXRRSET case
+;var=nsec_www_txt
+www.example.com. 3600 IN NSEC example.com. A NSEC RRSIG
+
+;var=
+www.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; Authoritative data without NSEC
+;var=nonsec_a_txt
+nonsec.example.com. 3600 IN A 192.0.2.0
+
+;; (Secure) delegation data; Delegation with DS record
+;var=signed_delegation_txt
+signed-delegation.example.com. 3600 IN NS ns.example.net.
+;var=signed_delegation_ds_txt
+signed-delegation.example.com. 3600 IN DS 12345 8 2 764501411DE58E8618945054A3F620B36202E115D015A7773F4B78E0F952CECA
+
+;var=
+signed-delegation.example.com. 3600 IN RRSIG DS 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; (Secure) delegation data; Delegation without DS record (and both NSEC
+;; and NSEC3 denying its existence)
+;var=unsigned_delegation_txt
+unsigned-delegation.example.com. 3600 IN NS ns.example.net.
+;var=unsigned_delegation_nsec_txt
+unsigned-delegation.example.com. 3600 IN NSEC unsigned-delegation-optout.example.com. NS RRSIG NSEC
+
+;var=
+unsigned-delegation.example.com. 3600 IN RRSIG NSEC 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+
+;; Delegation without DS record, and no direct matching NSEC3 record
+;var=unsigned_delegation_optout_txt
+unsigned-delegation-optout.example.com. 3600 IN NS ns.example.net.
+;var=unsigned_delegation_optout_nsec_txt
+unsigned-delegation-optout.example.com. 3600 IN NSEC *.uwild.example.com. NS RRSIG NSEC
+
+;; (Secure) delegation data; Delegation where the DS lookup will raise an
+;; exception.
+;var=bad_delegation_txt
+bad-delegation.example.com. 3600 IN NS ns.example.net.
+
+;; Delegation from an unsigned parent.  There's no DS, and there's no NSEC
+;; or NSEC3 that proves it.
+;var=nosec_delegation_txt
+nosec-delegation.example.com. 3600 IN NS ns.nosec.example.net.

src/bin/auth/tests/testdata/example-base.zone.in

@@ -0,0 +1,7 @@
+;;
+;; This is a complete (but crafted and somewhat broken) zone file used
+;; in query tests.
+;;
+
+$INCLUDE @abs_srcdir@/example-base-inc.zone
+$INCLUDE @abs_builddir@/example-common-inc.zone

src/bin/auth/tests/testdata/example-common-inc-template.zone

@@ -0,0 +1,5 @@
+;;
+;; This is an initial template of part of test zone file used in query test
+;; and expected to be included from other zone files.  This is
+;; intentionally kept empty.
+;;

src/bin/auth/tests/testdata/example-nsec3-inc.zone

@@ -0,0 +1,16 @@
+;; See query_testzone_data.txt for general notes.
+
+;; NSEC3PARAM.  This is needed for database-based data source to
+;; signal the zone is NSEC3-signed
+;var=
+example.com. 3600 IN NSEC3PARAM 1 1 12 aabbccdd
+
+;; NSEC3 RRs.  You may also need to add mapping to MockZoneFinder::hash_map_.
+;var=nsec3_apex_txt
+0p9mhaveqvm6t7vbl5lop2u3t2rp3tom.example.com. 3600 IN NSEC3 1 1 12 aabbccdd 2t7b4g4vsa5smi47k61mv5bv1a22bojr NS SOA NSEC3PARAM RRSIG
+;var=nsec3_apex_rrsig_txt
+0p9mhaveqvm6t7vbl5lop2u3t2rp3tom.example.com. 3600 IN RRSIG NSEC3 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
+;var=nsec3_www_txt
+q04jkcevqvmu85r014c7dkba38o0ji5r.example.com. 3600 IN NSEC3 1 1 12 aabbccdd r53bq7cc2uvmubfu5ocmm6pers9tk9en A RRSIG
+;var=nsec3_www_rrsig_txt
+q04jkcevqvmu85r014c7dkba38o0ji5r.example.com. 3600 IN RRSIG NSEC3 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE

src/bin/auth/tests/testdata/example-nsec3.zone.in

@@ -0,0 +1,8 @@
+;;
+;; This is a complete (but crafted and somewhat broken) zone file used
+;; in query tests including NSEC3 records, making the zone is "NSEC3 signed".
+;;
+
+$INCLUDE @abs_srcdir@/example-base-inc.zone
+$INCLUDE @abs_srcdir@/example-nsec3-inc.zone
+$INCLUDE @abs_builddir@/example-common-inc.zone

src/bin/auth/tests/testdata/example.zone

@@ -1,121 +0,0 @@
-;;
-;; This is a complete (but crafted and somewhat broken) zone file used
-;; in query tests.
-;;
-
-example.com. 3600 IN SOA . . 0 0 0 0 0
-example.com. 3600 IN NS glue.delegation.example.com.
-example.com. 3600 IN NS noglue.example.com.
-example.com. 3600 IN NS example.net.
-example.com. 3600 IN DS 57855 5 1 B6DCD485719ADCA18E5F3D48A2331627FDD3 636B
-glue.delegation.example.com. 3600 IN A 192.0.2.153
-glue.delegation.example.com. 3600 IN AAAA 2001:db8::53
-noglue.example.com. 3600 IN A 192.0.2.53
-delegation.example.com. 3600 IN NS glue.delegation.example.com.
-delegation.example.com. 3600 IN NS noglue.example.com.
-delegation.example.com. 3600 IN NS cname.example.com.
-delegation.example.com. 3600 IN NS example.org.
-;; Borrowed from the RFC4035
-delegation.example.com. 3600 IN DS 57855 5 1 B6DCD485719ADCA18E5F3D48A2331627FDD3 636B
-mx.example.com. 3600 IN MX 10 www.example.com.
-mx.example.com. 3600 IN MX 20 mailer.example.org.
-mx.example.com. 3600 IN MX 30 mx.delegation.example.com.
-www.example.com. 3600 IN A 192.0.2.80
-cname.example.com. 3600 IN CNAME www.example.com.
-cnamenxdom.example.com. 3600 IN CNAME nxdomain.example.com.
-;; CNAME Leading out of zone
-cnameout.example.com. 3600 IN CNAME www.example.org.
-;; The DNAME to do tests against
-dname.example.com. 3600 IN DNAME somethinglong.dnametarget.example.com.
-;; Some data at the dname node (allowed by RFC 2672)
-dname.example.com. 3600 IN A 192.0.2.5
-;; The rest of data won't be referenced from the test cases.
-cnamemailer.example.com. 3600 IN CNAME www.example.com.
-cnamemx.example.com. 3600 IN MX 10 cnamemailer.example.com.
-mx.delegation.example.com. 3600 IN A 192.0.2.100
-;; Wildcards
-*.wild.example.com. 3600 IN A 192.0.2.7
-*.wild.example.com. 3600 IN NSEC www.example.com. A NSEC RRSIG
-*.cnamewild.example.com. 3600 IN CNAME www.example.org.
-*.cnamewild.example.com. 3600 IN NSEC delegation.example.com. CNAME NSEC RRSIG
-;; Wildcard_nxrrset
-*.uwild.example.com. 3600 IN A 192.0.2.9
-*.uwild.example.com. 3600 IN NSEC www.uwild.example.com. A NSEC RRSIG
-www.uwild.example.com. 3600 IN A 192.0.2.11
-www.uwild.example.com. 3600 IN NSEC *.wild.example.com. A NSEC RRSIG
-;; Wildcard empty
-b.*.t.example.com. 3600 IN A 192.0.2.13
-b.*.t.example.com. 3600 IN NSEC *.uwild.example.com. A NSEC RRSIG
-t.example.com. 3600 IN A 192.0.2.15
-t.example.com. 3600 IN NSEC b.*.t.example.com. A NSEC RRSIG
-;; Used in NXDOMAIN proof test.  We are going to test some unusual case where
-;; the best possible wildcard is below the "next domain" of the NSEC RR that
-;; proves the NXDOMAIN, i.e.,
-;; mx.example.com. (exist)
-;; (.no.example.com. (qname, NXDOMAIN)
-;; ).no.example.com. (exist)
-;; *.no.example.com. (best possible wildcard, not exist)
-\).no.example.com. 3600 IN AAAA 2001:db8::53
-;; NSEC records.
-example.com. 3600 IN NSEC cname.example.com. NS SOA NSEC RRSIG
-mx.example.com. 3600 IN NSEC \).no.example.com. MX NSEC RRSIG
-\).no.example.com. 3600 IN NSEC nz.no.example.com. AAAA NSEC RRSIG
-;; We'll also test the case where a single NSEC proves both NXDOMAIN and the
-;; non existence of wildcard.  The following records will be used for that
-;; test.
-;; ).no.example.com. (exist, whose NSEC proves everything)
-;; *.no.example.com. (best possible wildcard, not exist)
-;; nx.no.example.com. (NXDOMAIN)
-;; nz.no.example.com. (exist)
-nz.no.example.com. 3600 IN AAAA 2001:db8::5300
-nz.no.example.com. 3600 IN NSEC noglue.example.com. AAAA NSEC RRSIG
-noglue.example.com. 3600 IN NSEC nonsec.example.com. A
-
-;; NSEC for the normal NXRRSET case
-www.example.com. 3600 IN NSEC example.com. A NSEC RRSIG
-
-;; Authoritative data without NSEC
-nonsec.example.com. 3600 IN A 192.0.2.0
-
-;; NSEC3 RRs.  You may also need to add mapping to MockZoneFinder::hash_map_.
-0p9mhaveqvm6t7vbl5lop2u3t2rp3tom.example.com. 3600 IN NSEC3 1 1 12 aabbccdd 2t7b4g4vsa5smi47k61mv5bv1a22bojr NS SOA NSEC3PARAM RRSIG
-0p9mhaveqvm6t7vbl5lop2u3t2rp3tom.example.com. 3600 IN RRSIG NSEC3 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
-q04jkcevqvmu85r014c7dkba38o0ji5r.example.com. 3600 IN NSEC3 1 1 12 aabbccdd r53bq7cc2uvmubfu5ocmm6pers9tk9en A RRSIG
-q04jkcevqvmu85r014c7dkba38o0ji5r.example.com. 3600 IN RRSIG NSEC3 5 3 3600 20000101000000 20000201000000 12345 example.com. FAKEFAKEFAKE
-
-;; NSEC3 for wild.example.com (used in wildcard tests, will be added on
-;; demand not to confuse other tests)
-ji6neoaepv8b5o6k4ev33abha8ht9fgc.example.com. 3600 IN NSEC3 1 1 12 aabbccdd r53bq7cc2uvmubfu5ocmm6pers9tk9en
-
-;; NSEC3 for cnamewild.example.com (used in wildcard tests, will be added on
-;; demand not to confuse other tests)
-k8udemvp1j2f7eg6jebps17vp3n8i58h.example.com. 3600 IN NSEC3 1 1 12 aabbccdd r53bq7cc2uvmubfu5ocmm6pers9tk9en
-
-;; NSEC3 for *.uwild.example.com (will be added on demand not to confuse
-;; other tests)
-b4um86eghhds6nea196smvmlo4ors995.example.com. 3600 IN NSEC3 1 1 12 aabbccdd r53bq7cc2uvmubfu5ocmm6pers9tk9en A RRSIG
-;; NSEC3 for uwild.example.com. (will be added on demand)
-t644ebqk9bibcna874givr6joj62mlhv.example.com. 3600 IN NSEC3 1 1 12 aabbccdd r53bq7cc2uvmubfu5ocmm6pers9tk9en A RRSIG
-
-;; (Secure) delegation data; Delegation with DS record
-signed-delegation.example.com. 3600 IN NS ns.example.net.
-signed-delegation.example.com. 3600 IN DS 12345 8 2 764501411DE58E8618945054A3F620B36202E115D015A7773F4B78E0F952CECA
-
-;; (Secure) delegation data; Delegation without DS record (and both NSEC
-;; and NSEC3 denying its existence)
-unsigned-delegation.example.com. 3600 IN NS ns.example.net.
-unsigned-delegation.example.com. 3600 IN NSEC unsigned-delegation-optout.example.com. NS RRSIG NSEC
-;; This one will be added on demand
-q81r598950igr1eqvc60aedlq66425b5.example.com. 3600 IN NSEC3 1 1 12 aabbccdd 0p9mhaveqvm6t7vbl5lop2u3t2rp3tom NS RRSIG
-
-;; Delegation without DS record, and no direct matching NSEC3 record
-unsigned-delegation-optout.example.com. 3600 IN NS ns.example.net.
-unsigned-delegation-optout.example.com. 3600 IN NSEC *.uwild.example.com. NS RRSIG NSEC
-
-;; (Secure) delegation data; Delegation where the DS lookup will raise an
-;; exception.
-bad-delegation.example.com. 3600 IN NS ns.example.net.
-
-;; Delegation from an unsigned parent.  There's no DS, and there's no NSEC
-;; or NSEC3 that proves it.
-nosec-delegation.example.com. 3600 IN NS ns.nosec.example.net.

src/bin/auth/tests/testdata/example.zone.in

@@ -0,0 +1,6 @@
+;;
+;; This is a complete (but crafted and somewhat broken) zone file used
+;; in query tests, excluding NSEC3 records.
+;;
+
+$INCLUDE @abs_builddir@/example-base.zone

src/bin/bind10/bind10_messages.mes

@@ -154,7 +154,7 @@ The boss module received the given signal.
 % BIND10_RESTART_COMPONENT_SKIPPED Skipped restarting a component %1
 The boss module tried to restart a component after it failed (crashed)
 unexpectedly, but the boss then found that the component had been removed
-from its local configuration of components to run.  This is an unusal
+from its local configuration of components to run.  This is an unusual
 situation but can happen if the administrator removes the component from
 the configuration after the component's crash and before the restart time.
 The boss module simply skipped restarting that module, and the whole system
@@ -262,7 +262,7 @@ indicated OS API function with given error.
 The boss forwards a request for a socket to the socket creator.
 
 % BIND10_STARTED_CC started configuration/command session
-Debug message given when BIND 10 has successfull started the object that
+Debug message given when BIND 10 has successfully started the object that
 handles configuration and commands.
 
 % BIND10_STARTED_PROCESS started %1
@@ -308,13 +308,6 @@ During the startup process, a number of messages are exchanged between the
 Boss process and the processes it starts.  This error is output when a
 message received by the Boss process is not recognised.
 
-% BIND10_START_AS_NON_ROOT_RESOLVER starting b10-resolver as a user, not root. This might fail.
-The resolver is being started or restarted without root privileges.
-If the module needs these privileges, it may have problems starting.
-Note that this issue should be resolved by the pending 'socket-creator'
-process; once that has been implemented, modules should not need root
-privileges anymore. See tickets #800 and #801 for more information.
-
 % BIND10_STOP_PROCESS asking %1 to shut down
 The boss module is sending a shutdown command to the given module over
 the message channel.

src/bin/bind10/bind10_src.py.in

@@ -103,8 +103,31 @@ VERSION = "bind10 20110223 (BIND 10 @PACKAGE_VERSION@)"
 # This is for boot_time of Boss
 _BASETIME = time.gmtime()
 
+# Detailed error message commonly used on startup failure, possibly due to
+# permission issue regarding log lock file.  We dump verbose message because
+# it may not be clear exactly what to do if it simply says
+# "failed to open <filename>: permission denied"
+NOTE_ON_LOCK_FILE = """\
+TIP: if this is about permission error for a lock file, check if the directory
+of the file is writable for the user of the bind10 process; often you need
+to start bind10 as a super user.  Also, if you specify the -u option to
+change the user and group, the directory must be writable for the group,
+and the created lock file must be writable for that user. Finally, make sure
+the lock file is not left in the directly before restarting.
+"""
+
 class ProcessInfoError(Exception): pass
 
+class ChangeUserError(Exception):
+    '''Exception raised when setuid/setgid fails.
+
+    When raised, it's expected to be propagated via underlying component
+    management modules to the top level so that it will help provide useful
+    fatal error message.
+
+    '''
+    pass
+
 class ProcessInfo:
     """Information about a process"""
 
@@ -206,8 +229,8 @@ class BoB:
         # restart. Components manage their own restart schedule now
         self.components_to_restart = []
         self.runnable = False
-        self.uid = setuid
-        self.gid = setgid
+        self.__uid = setuid
+        self.__gid = setgid
         self.username = username
         self.verbose = verbose
         self.nokill = nokill
@@ -269,6 +292,31 @@ class BoB:
         # Update the configuration
         self._component_configurator.reconfigure(comps)
 
+    def change_user(self):
+        '''Change the user and group to those specified on construction.
+
+        This method is expected to be called by a component on initial
+        startup when the system is ready to switch the user and group
+        (i.e., once all components that need the privilege of the original
+        user have started).
+        '''
+        try:
+            if self.__gid is not None:
+                logger.info(BIND10_SETGID, self.__gid)
+                posix.setgid(self.__gid)
+        except Exception as ex:
+            raise ChangeUserError('failed to change group: ' + str(ex))
+
+        try:
+            if self.__uid is not None:
+                posix.setuid(self.__uid)
+                # We use one-shot logger after setuid here.  This will
+                # detect any permission issue regarding logging due to the
+                # result of setuid at the earliest opportunity.
+                isc.log.Logger("boss").info(BIND10_SETUID, self.__uid)
+        except Exception as ex:
+            raise ChangeUserError('failed to change user: ' + str(ex))
+
     def config_handler(self, new_config):
         # If this is initial update, don't do anything now, leave it to startup
         if not self.runnable:
@@ -443,6 +491,8 @@ class BoB:
 
             # if we have been trying for "a while" give up
             if (time.time() - cc_connect_start) > self.msgq_timeout:
+                if msgq_proc.process:
+                    msgq_proc.process.kill()
                 logger.error(BIND10_CONNECTING_TO_CC_FAIL)
                 raise CChannelConnectError("Unable to connect to c-channel after 5 seconds")
 
@@ -578,8 +628,6 @@ class BoB:
             are pure speculation.  As with the auth daemon, they should be
             read from the configuration database.
         """
-        if self.uid is not None and self.__started:
-            logger.warn(BIND10_START_AS_NON_ROOT_RESOLVER)
         self.curproc = "b10-resolver"
         # XXX: this must be read from the configuration manager in the future
         resargs = ['b10-resolver']
@@ -646,6 +694,9 @@ class BoB:
         try:
             self.c_channel_env = c_channel_env
             self.start_all_components()
+        except ChangeUserError as e:
+            self.kill_started_components()
+            return str(e) + '; ' + NOTE_ON_LOCK_FILE.replace('\n', ' ')
         except Exception as e:
             self.kill_started_components()
             return "Unable to start " + self.curproc + ": " + str(e)
@@ -1155,7 +1206,19 @@ def remove_lock_files():
     for f in lockfiles:
         fname = lpath + '/' + f
         if os.path.isfile(fname):
-            os.unlink(fname)
+            try:
+                os.unlink(fname)
+            except OSError as e:
+                # We catch and ignore permission related error on unlink.
+                # This can happen if bind10 started with -u, created a lock
+                # file as a privileged user, but the directory is not writable
+                # for the changed user.  This setup will cause immediate
+                # start failure, and we leave verbose error message including
+                # the leftover lock file, so it should be acceptable to ignore
+                # it (note that it doesn't make sense to log this event at
+                # this poitn)
+                if e.errno != errno.EPERM and e.errno != errno.EACCES:
+                    raise
 
     return
 
@@ -1173,13 +1236,7 @@ def main():
     except RuntimeError as e:
         sys.stderr.write('ERROR: failed to write the initial log: %s\n' %
                          str(e))
-        sys.stderr.write("""\
-TIP: if this is about permission error for a lock file, check if the directory
-of the file is writable for the user of the bind10 process; often you need
-to start bind10 as a super user.  Also, if you specify the -u option to
-change the user and group, the directory must be writable for the group,
-and the created lock file must be writable for that user.
-""")
+        sys.stderr.write(NOTE_ON_LOCK_FILE)
         sys.exit(1)
 
     # Check user ID.

src/bin/bind10/tests/bind10_test.py.in

@@ -341,6 +341,18 @@ class TestCacheCommands(unittest.TestCase):
 
 
 class TestBoB(unittest.TestCase):
+    def setUp(self):
+        # Save original values that may be tweaked in some tests
+        self.__orig_setgid = bind10_src.posix.setgid
+        self.__orig_setuid = bind10_src.posix.setuid
+        self.__orig_logger_class = isc.log.Logger
+
+    def tearDown(self):
+        # Restore original values saved in setUp()
+        bind10_src.posix.setgid = self.__orig_setgid
+        bind10_src.posix.setuid = self.__orig_setuid
+        isc.log.Logger = self.__orig_logger_class
+
     def test_init(self):
         bob = BoB()
         self.assertEqual(bob.verbose, False)
@@ -349,10 +361,56 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.ccs, None)
         self.assertEqual(bob.components, {})
         self.assertEqual(bob.runnable, False)
-        self.assertEqual(bob.uid, None)
         self.assertEqual(bob.username, None)
         self.assertIsNone(bob._socket_cache)
 
+    def __setgid(self, gid):
+        self.__gid_set = gid
+
+    def __setuid(self, uid):
+        self.__uid_set = uid
+
+    def test_change_user(self):
+        bind10_src.posix.setgid = self.__setgid
+        bind10_src.posix.setuid = self.__setuid
+
+        self.__gid_set = None
+        self.__uid_set = None
+        bob = BoB()
+        bob.change_user()
+        # No gid/uid set in boss, nothing called.
+        self.assertIsNone(self.__gid_set)
+        self.assertIsNone(self.__uid_set)
+
+        BoB(setuid=42, setgid=4200).change_user()
+        # This time, it get's called
+        self.assertEqual(4200, self.__gid_set)
+        self.assertEqual(42, self.__uid_set)
+
+        def raising_set_xid(gid_or_uid):
+            ex = OSError()
+            ex.errno, ex.strerror = errno.EPERM, 'Operation not permitted'
+            raise ex
+
+        # Let setgid raise an exception
+        bind10_src.posix.setgid = raising_set_xid
+        bind10_src.posix.setuid = self.__setuid
+        self.assertRaises(bind10_src.ChangeUserError,
+                          BoB(setuid=42, setgid=4200).change_user)
+
+        # Let setuid raise an exception
+        bind10_src.posix.setgid = self.__setgid
+        bind10_src.posix.setuid = raising_set_xid
+        self.assertRaises(bind10_src.ChangeUserError,
+                          BoB(setuid=42, setgid=4200).change_user)
+
+        # Let initial log output after setuid raise an exception
+        bind10_src.posix.setgid = self.__setgid
+        bind10_src.posix.setuid = self.__setuid
+        isc.log.Logger = raising_set_xid
+        self.assertRaises(bind10_src.ChangeUserError,
+                          BoB(setuid=42, setgid=4200).change_user)
+
     def test_set_creator(self):
         """
         Test the call to set_creator. First time, the cache is created
@@ -423,7 +481,6 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.ccs, None)
         self.assertEqual(bob.components, {})
         self.assertEqual(bob.runnable, False)
-        self.assertEqual(bob.uid, None)
         self.assertEqual(bob.username, None)
 
     def test_command_handler(self):
@@ -2021,8 +2078,10 @@ class TestBossComponents(unittest.TestCase):
 
             def start_all_components(self):
                 self.started = True
-                if self.throw:
+                if self.throw is True:
                     raise Exception('Assume starting components has failed.')
+                elif self.throw:
+                    raise self.throw
 
             def kill_started_components(self):
                 self.killed = True
@@ -2067,6 +2126,12 @@ class TestBossComponents(unittest.TestCase):
         r = bob.startup()
         self.assertEqual({'BIND10_MSGQ_SOCKET_FILE': 'foo'}, bob.c_channel_env)
 
+        # Check failure of changing user results in a different message
+        bob = MockBobStartup(bind10_src.ChangeUserError('failed to chusr'))
+        r = bob.startup()
+        self.assertIn('failed to chusr', r)
+        self.assertTrue(bob.killed)
+
         # Check the case when socket file already exists
         isc.cc.Session = DummySessionSocketExists
         bob = MockBobStartup(False)
@@ -2277,11 +2342,15 @@ class TestFunctions(unittest.TestCase):
         self.assertFalse(os.path.exists(self.lockfile_testpath))
         os.mkdir(self.lockfile_testpath)
         self.assertTrue(os.path.isdir(self.lockfile_testpath))
+        self.__isfile_orig = bind10_src.os.path.isfile
+        self.__unlink_orig = bind10_src.os.unlink
 
     def tearDown(self):
         os.rmdir(self.lockfile_testpath)
         self.assertFalse(os.path.isdir(self.lockfile_testpath))
         os.environ["B10_LOCKFILE_DIR_FROM_BUILD"] = "@abs_top_builddir@"
+        bind10_src.os.path.isfile = self.__isfile_orig
+        bind10_src.os.unlink = self.__unlink_orig
 
     def test_remove_lock_files(self):
         os.environ["B10_LOCKFILE_DIR_FROM_BUILD"] = self.lockfile_testpath
@@ -2305,6 +2374,28 @@ class TestFunctions(unittest.TestCase):
         # second call should not assert anyway
         bind10_src.remove_lock_files()
 
+    def test_remove_lock_files_fail(self):
+        # Permission error on unlink is ignored; other exceptions are really
+        # unexpected and propagated.
+        def __raising_unlink(unused, ex):
+            raise ex
+
+        bind10_src.os.path.isfile = lambda _: True
+        os_error = OSError()
+        bind10_src.os.unlink = lambda f: __raising_unlink(f, os_error)
+
+        os_error.errno = errno.EPERM
+        bind10_src.remove_lock_files() # no disruption
+
+        os_error.errno = errno.EACCES
+        bind10_src.remove_lock_files() # no disruption
+
+        os_error.errno = errno.ENOENT
+        self.assertRaises(OSError, bind10_src.remove_lock_files)
+
+        bind10_src.os.unlink = lambda f: __raising_unlink(f, Exception('bad'))
+        self.assertRaises(Exception, bind10_src.remove_lock_files)
+
     def test_get_signame(self):
         # just test with some samples
         signame = bind10_src.get_signame(signal.SIGTERM)

src/bin/bindctl/bindcmd.py

@@ -208,12 +208,13 @@ WARNING: Python readline module isn't available, so the command line editor
         return True
 
     def login_to_cmdctl(self):
-        '''Login to cmdctl with the username and password inputted
-        from user. After the login is sucessful, the username and
+        '''Login to cmdctl with the username and password given by
+        the user. After the login is sucessful, the username and
         password will be saved in 'default_user.csv', when run the next
         time, username and password saved in 'default_user.csv' will be
         used first.
         '''
+        # Look at existing username/password combinations and try to log in
         users = self._get_saved_user_info(self.csv_file_dir, CSV_FILE_NAME)
         for row in users:
             param = {'username': row[0], 'password' : row[1]}
@@ -230,15 +231,18 @@ WARNING: Python readline module isn't available, so the command line editor
                     print(data + ' login as ' + row[0])
                 return True
 
+        # No valid logins were found, prompt the user for a username/password
         count = 0
-        print("[TEMP MESSAGE]: username :root  password :bind10")
+        print('No stored password file found, please see sections '
+              '"Configuration specification for b10-cmdctl" and "bindctl '
+              'command-line options" of the BIND 10 guide.')
         while True:
             count = count + 1
             if count > 3:
                 print("Too many authentication failures")
                 return False
 
-            username = input("Username:")
+            username = input("Username: ")
             passwd = getpass.getpass()
             param = {'username': username, 'password' : passwd}
             try:

src/bin/bindctl/run_bindctl.sh.in

@@ -20,7 +20,10 @@ export PYTHON_EXEC
 
 BINDCTL_PATH=@abs_top_builddir@/src/bin/bindctl
 
-PYTHONPATH=@abs_top_srcdir@/src/bin:@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/bin:@abs_top_srcdir@/src/lib/python
+# Note: lib/dns/python/.libs is necessary because __init__.py of isc package
+# automatically imports isc.datasrc, which then requires the DNS loadable
+# module.  #2145 should eliminate the need for it.
+PYTHONPATH=@abs_top_srcdir@/src/bin:@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/bin:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs
 export PYTHONPATH
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries

src/bin/dbutil/dbutil_messages.mes

@@ -53,7 +53,7 @@ inconsistent state, and it is advised to restore it from the backup that was
 created when b10-dbutil started.
 
 % DBUTIL_EXECUTE Executing SQL statement: %1
-Debug message; the given SQL statement is executed
+Debug message; the given SQL statement is executed.
 
 % DBUTIL_FILE Database file: %1
 The database file that is being checked.
@@ -67,7 +67,7 @@ The given database statement failed to execute. The error is shown in the
 message.
 
 % DBUTIL_TOO_MANY_ARGUMENTS too many arguments to the command, maximum of one expected
-There were too many command-line arguments to b10-dbutil
+There were too many command-line arguments to b10-dbutil.
 
 % DBUTIL_UPGRADE_CANCELED upgrade canceled; database has not been changed
 The user aborted the upgrade, and b10-dbutil will now exit.
@@ -95,7 +95,7 @@ again.
 
 % DBUTIL_UPGRADE_PREPARATION_FAILED upgrade preparation failed: %1
 An unexpected error occurred while b10-dbutil was preparing to upgrade the
-database schema. The error is shown in the message
+database schema. The error is shown in the message.
 
 % DBUTIL_UPGRADE_SUCCESFUL database upgrade successfully completed
 The database schema update was completed successfully.

src/bin/dbutil/run_dbutil.sh.in

@@ -20,7 +20,10 @@ export PYTHON_EXEC
 
 DBUTIL_PATH=@abs_top_builddir@/src/bin/dbutil
 
-PYTHONPATH=@abs_top_srcdir@/src/bin:@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/bin:@abs_top_srcdir@/src/lib/python
+# Note: lib/dns/python/.libs is necessary because __init__.py of isc package
+# automatically imports isc.datasrc, which then requires the DNS loadable
+# module.  #2145 should eliminate the need for it.
+PYTHONPATH=@abs_top_srcdir@/src/bin:@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/bin:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs
 export PYTHONPATH
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries

src/bin/dhcp4/config_parser.h

@@ -1,4 +1,4 @@
-// Copyright (C) 2012  Internet Systems Consortium, Inc. ("ISC")
+// Copyright (C) 2012-2013 Internet Systems Consortium, Inc. ("ISC")
 //
 // Permission to use, copy, modify, and/or distribute this software for any
 // purpose with or without fee is hereby granted, provided that the above
@@ -28,117 +28,12 @@ namespace dhcp {
 
 class Dhcpv4Srv;
 
-/// @brief a collection of elements that store uint32 values (e.g. renew-timer = 900)
-typedef std::map<std::string, uint32_t> Uint32Storage;
-
-/// @brief a collection of elements that store string values
-typedef std::map<std::string, std::string> StringStorage;
-
-/// An exception that is thrown if an error occurs while configuring an
-/// \c Dhcpv4Srv object.
-class Dhcp4ConfigError : public isc::Exception {
-public:
-
-    /// @brief constructor
-    ///
-    /// @param file name of the file, where exception occurred
-    /// @param line line of the file, where exception occurred
-    /// @param what text description of the issue that caused exception
-    Dhcp4ConfigError(const char* file, size_t line, const char* what)
-        : isc::Exception(file, line, what) {}
-};
-
-/// @brief Base abstract class for all DHCPv4 parsers
+/// @brief Configure DHCPv4 server (@c Dhcpv4Srv) with a set of configuration values.
 ///
-/// Each instance of a class derived from this class parses one specific config
-/// element. Sometimes elements are simple (e.g. a string) and sometimes quite
-/// complex (e.g. a subnet). In such case, it is likely that a parser will
-/// spawn child parsers to parse child elements in the configuration.
-/// @todo: Merge this class with DhcpConfigParser in src/bin/dhcp6
-class Dhcp4ConfigParser {
-    ///
-    /// \name Constructors and Destructor
-    ///
-    /// Note: The copy constructor and the assignment operator are
-    /// intentionally defined as private to make it explicit that this is a
-    /// pure base class.
-    //@{
-private:
-
-    // Private construtor and assignment operator assures that nobody
-    // will be able to copy or assign a parser. There are no defined
-    // bodies for them.
-    Dhcp4ConfigParser(const Dhcp4ConfigParser& source);
-    Dhcp4ConfigParser& operator=(const Dhcp4ConfigParser& source);
-protected:
-    /// \brief The default constructor.
-    ///
-    /// This is intentionally defined as \c protected as this base class should
-    /// never be instantiated (except as part of a derived class).
-    Dhcp4ConfigParser() {}
-public:
-    /// The destructor.
-    virtual ~Dhcp4ConfigParser() {}
-    //@}
-
-    /// \brief Prepare configuration value.
-    ///
-    /// This method parses the "value part" of the configuration identifier
-    /// that corresponds to this derived class and prepares a new value to
-    /// apply to the server.
-    ///
-    /// This method must validate the given value both in terms of syntax
-    /// and semantics of the configuration, so that the server will be
-    /// validly configured at the time of \c commit().  Note: the given
-    /// configuration value is normally syntactically validated, but the
-    /// \c build() implementation must also expect invalid input.  If it
-    /// detects an error it may throw an exception of a derived class
-    /// of \c isc::Exception.
-    ///
-    /// Preparing a configuration value will often require resource
-    /// allocation.  If it fails, it may throw a corresponding standard
-    /// exception.
-    ///
-    /// This method is not expected to be called more than once in the
-    /// life of the object. Although multiple calls are not prohibited
-    /// by the interface, the behavior is undefined.
-    ///
-    /// \param config_value The configuration value for the identifier
-    /// corresponding to the derived class.
-    virtual void build(isc::data::ConstElementPtr config_value) = 0;
-
-    /// \brief Apply the prepared configuration value to the server.
-    ///
-    /// This method is expected to be exception free, and, as a consequence,
-    /// it should normally not involve resource allocation.
-    /// Typically it would simply perform exception free assignment or swap
-    /// operation on the value prepared in \c build().
-    /// In some cases, however, it may be very difficult to meet this
-    /// condition in a realistic way, while the failure case should really
-    /// be very rare.  In such a case it may throw, and, if the parser is
-    /// called via \c configureDhcp4Server(), the caller will convert the
-    /// exception as a fatal error.
-    ///
-    /// This method is expected to be called after \c build(), and only once.
-    /// The result is undefined otherwise.
-    virtual void commit() = 0;
-};
-
-/// @brief a pointer to configuration parser
-typedef boost::shared_ptr<Dhcp4ConfigParser> ParserPtr;
-
-/// @brief a collection of parsers
-///
-/// This container is used to store pointer to parsers for a given scope.
-typedef std::vector<ParserPtr> ParserCollection;
-
-
-/// \brief Configure DHCPv4 server (\c Dhcpv4Srv) with a set of configuration values.
-///
-/// This function parses configuration information stored in \c config_set
-/// and configures the \c server by applying the configuration to it.
+/// This function parses configuration information stored in @c config_set
+/// and configures the @c server by applying the configuration to it.
 /// It provides the strong exception guarantee as long as the underlying
-/// derived class implementations of \c DhcpConfigParser meet the assumption,
+/// derived class implementations of @c DhcpConfigParser meet the assumption,
 /// that is, it ensures that either configuration is fully applied or the
 /// state of the server is intact.
 ///
@@ -154,7 +49,8 @@ typedef std::vector<ParserPtr> ParserCollection;
 /// reconfiguration statuses. It may return the following response codes:
 /// 0 - configuration successful
 /// 1 - malformed configuration (parsing failed)
-/// 2 - logical error (parsing was successful, but the values are invalid)
+/// 2 - commit failed (parsing was successful, but failed to store the
+/// values in to server's configuration)
 ///
 /// @param config_set a new configuration (JSON) for DHCPv4 server
 /// @return answer that contains result of reconfiguration
@@ -162,6 +58,16 @@ isc::data::ConstElementPtr
 configureDhcp4Server(Dhcpv4Srv&,
                      isc::data::ConstElementPtr config_set);
 
+
+/// @brief Returns the global uint32_t values storage.
+///
+/// This function must be only used by unit tests that need
+/// to access uint32_t global storage to verify that the
+/// Uint32Parser works as expected.
+///
+/// @return a reference to a global uint32 values storage.
+const std::map<std::string, uint32_t>& getUint32Defaults();
+
 }; // end of isc::dhcp namespace
 }; // end of isc namespace
 

src/bin/dhcp4/ctrl_dhcp4_srv.cc

@@ -19,6 +19,7 @@
 #include <cc/session.h>
 #include <config/ccsession.h>
 #include <dhcp/iface_mgr.h>
+#include <dhcpsrv/dhcp_config_parser.h>
 #include <dhcp4/ctrl_dhcp4_srv.h>
 #include <dhcp4/dhcp4_log.h>
 #include <dhcp4/spec_config.h>
@@ -121,7 +122,7 @@ void ControlledDhcpv4Srv::establishSession() {
 
     try {
         configureDhcp4Server(*this, config_session_->getFullConfig());
-    } catch (const Dhcp4ConfigError& ex) {
+    } catch (const DhcpConfigError& ex) {
         LOG_ERROR(dhcp4_logger, DHCP4_CONFIG_LOAD_FAIL).arg(ex.what());
     }
 

src/bin/dhcp4/dhcp4.spec

@@ -34,6 +34,56 @@
         "item_default": 4000
       },
 
+      { "item_name": "option-def",
+        "item_type": "list",
+        "item_optional": false,
+        "item_default": [],
+        "list_item_spec":
+        {
+          "item_name": "single-option-def",
+          "item_type": "map",
+          "item_optional": false,
+          "item_default": {},
+          "map_item_spec": [
+          {
+            "item_name": "name",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": ""
+          },
+
+          { "item_name": "code",
+            "item_type": "integer",
+            "item_optional": false,
+            "item_default": 0,
+          },
+
+          { "item_name": "type",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": "",
+          },
+
+          { "item_name": "array",
+            "item_type": "boolean",
+            "item_optional": false,
+            "item_default": False
+          },
+
+          { "item_name": "record_types",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": "",
+          },
+
+          { "item_name": "space",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": ""
+          } ]
+        }
+      },
+
       { "item_name": "option-data",
         "item_type": "list",
         "item_optional": false,
@@ -61,6 +111,16 @@
             "item_type": "string",
             "item_optional": false,
             "item_default": ""
+          },
+          { "item_name": "csv-format",
+            "item_type": "boolean",
+            "item_optional": false,
+            "item_default": False
+          },
+          { "item_name": "space",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": "dhcp4"
           } ]
         }
       },
@@ -141,6 +201,16 @@
                       "item_type": "string",
                       "item_optional": false,
                       "item_default": ""
+                    },
+                    { "item_name": "csv-format",
+                      "item_type": "boolean",
+                      "item_optional": false,
+                      "item_default": False
+                      },
+                    { "item_name": "space",
+                      "item_type": "string",
+                      "item_optional": false,
+                      "item_default": "dhcp4"
                     } ]
                   }
                 } ]

src/bin/dhcp4/dhcp4_messages.mes

@@ -26,12 +26,6 @@ to establish a session with the BIND 10 control channel.
 A debug message listing the command (and possible arguments) received
 from the BIND 10 control system by the IPv4 DHCP server.
 
-% DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: %1
-This is an informational message announcing the successful processing of a
-new configuration. it is output during server startup, and when an updated
-configuration is committed by the administrator.  Additional information
-may be provided.
-
 % DHCP4_CONFIG_LOAD_FAIL failed to load configuration: %1
 This critical error message indicates that the initial DHCPv4
 configuration has failed. The server will start, but nothing will be
@@ -41,19 +35,52 @@ served until the configuration has been corrected.
 This is an informational message reporting that the configuration has
 been extended to include the specified IPv4 subnet.
 
+% DHCP4_CONFIG_OPTION_DUPLICATE multiple options with the code %1 added to the subnet %2
+This warning message is issued on an attempt to configure multiple options
+with the same option code for a particular subnet. Adding multiple options
+is uncommon for DHCPv4, but is not prohibited.
+
+% DHCP4_CONFIG_UPDATE updated configuration received: %1
+A debug message indicating that the IPv4 DHCP server has received an
+updated configuration from the BIND 10 configuration system.
+
 % DHCP4_CONFIG_START DHCPv4 server is processing the following configuration: %1
 This is a debug message that is issued every time the server receives a
 configuration. That happens at start up and also when a server configuration
 change is committed by the administrator.
 
-% DHCP4_CONFIG_UPDATE updated configuration received: %1
-A debug message indicating that the IPv4 DHCP server has received an
-updated configuration from the BIND 10 configuration system.
+% DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: %1
+This is an informational message announcing the successful processing of a
+new configuration. It is output during server startup, and when an updated
+configuration is committed by the administrator.  Additional information
+may be provided.
 
-% DHCP4_CONFIG_OPTION_DUPLICATE multiple options with the code: %1 added to the subnet: %2
-This warning message is issued on an attempt to configure multiple options with the
-same option code for the particular subnet. Adding multiple options is uncommon
-for DHCPv4, but it is not prohibited.
+% DHCP4_DB_BACKEND_STARTED lease database started (type: %1, name: %2)
+This informational message is printed every time DHCPv4 server is started
+and gives both the type and name of the database being used to store
+lease and other information.
+
+% DHCP4_LEASE_ADVERT lease %1 advertised (client client-id %2, hwaddr %3)
+This debug message indicates that the server successfully advertised
+a lease. It is up to the client to choose one server out of othe advertised
+and continue allocation with that server. This is a normal behavior and
+indicates successful operation.
+
+% DHCP4_LEASE_ADVERT_FAIL failed to advertise a lease for client client-id %1, hwaddr %2
+This message indicates that the server has failed to offer a lease to
+the specified client after receiving a DISCOVER message from it. There are
+many possible reasons for such a failure.
+
+% DHCP4_LEASE_ALLOC lease %1 has been allocated for client-id %2, hwaddr %3
+This debug message indicates that the server successfully granted a lease
+in response to client's REQUEST message. This is a normal behavior and
+incicates successful operation.
+
+% DHCP4_LEASE_ALLOC_FAIL failed to grant a lease for client-id %1, hwaddr %2
+This message indicates that the server failed to grant a lease to the
+specified client after receiving a REQUEST message from it.  There are many
+possible reasons for such a failure. Additional messages will indicate the
+reason.
 
 % DHCP4_NOT_RUNNING IPv4 DHCP server is not running
 A warning message is issued when an attempt is made to shut down the
@@ -75,7 +102,7 @@ may well be a valid DHCP packet, just a type not expected by the server
 
 % DHCP4_PACKET_RECEIVE_FAIL error on attempt to receive packet: %1
 The IPv4 DHCP server tried to receive a packet but an error
-occured during this attempt. The reason for the error is included in
+occurred during this attempt. The reason for the error is included in
 the message.
 
 % DHCP4_PACKET_SEND_FAIL failed to send DHCPv4 packet: %1
@@ -91,6 +118,43 @@ to be a programming error: please raise a bug report.
 % DHCP4_QUERY_DATA received packet type %1, data is <%2>
 A debug message listing the data received from the client.
 
+% DHCP4_RELEASE address %1 belonging to client-id %2, hwaddr %3 was released properly.
+This debug message indicates that an address was released properly. It
+is a normal operation during client shutdown.
+
+% DHCP4_RELEASE_EXCEPTION exception %1 while trying to release address %2
+This message is output when an error was encountered during an attempt
+to process a RELEASE message. The error will not affect the client,
+which does not expect any response from the server for RELEASE
+messages. Depending on the nature of problem, it may affect future
+server operation.
+
+% DHCP4_RELEASE_FAIL failed to remove lease for address %1 for duid %2, hwaddr %3
+This error message indicates that the software failed to remove a
+lease from the lease database. It is probably due to an error during a
+database operation: resolution will most likely require administrator
+intervention (e.g. check if DHCP process has sufficient privileges to
+update the database). It may also be triggered if a lease was manually
+removed from the database during RELEASE message processing.
+
+% DHCP4_RELEASE_FAIL_NO_LEASE client (client-id %2) tried to release address %1, but there is no lease for such address.
+This warning message is printed when client attempts to release a lease,
+but no such lease is known to the server.
+
+% DHCP4_RELEASE_FAIL_WRONG_CLIENT_ID client (client-id %2) tried to release address %1, but it belongs to client (client-id %3)
+This warning message indicates that client tried to release an address
+that belongs to a different client. This should not happen in normal
+circumstances and may indicate a misconfiguration of the client.  However,
+since the client releasing the address will stop using it anyway, there
+is a good chance that the situation will correct itself.
+
+% DHCP4_RELEASE_FAIL_WRONG_HWADDR client (client-id %2) tried to release address %1, but sent from a wrong hardware address (%3)
+This warning message indicates that client tried to release an address
+that does belong to it, but the lease information was associated with
+a different hardware address. One possible reason for using different
+hardware address is that a cloned virtual machine was not updated and
+both clones use the same client-id.
+
 % DHCP4_RESPONSE_DATA responding with packet type %1, data is <%2>
 A debug message listing the data returned to the client.
 
@@ -98,6 +162,26 @@ A debug message listing the data returned to the client.
 The IPv4 DHCP server has encountered a fatal error and is terminating.
 The reason for the failure is included in the message.
 
+% DHCP4_SERVERID_GENERATED server-id %1 has been generated and will be stored in %2
+This informational messages indicates that the server was not able to
+read its server identifier and has generated a new one. This server-id
+will be stored in a file and will be read (and used) whenever the server
+is restarted. This is normal behavior when the server is started for the
+first time. If this message is printed every time the server is started,
+please check that the server has sufficient permission to write its
+server-id file and that the file is not corrupt.
+
+% DHCP4_SERVERID_LOADED server-id %1 has been loaded from file %2
+This debug message indicates that the server loaded its server identifier.
+That value is sent in all server responses and clients use it to
+discriminate between servers. This is a part of normal startup or
+reconfiguration procedure.
+
+% DHCP4_SERVERID_WRITE_FAIL server was not able to write its ID to file %1
+This warning message indicates that server was not able to write its
+server identifier to a file. The most likely cause is is that the server
+does not have permissions to write the server id file.
+
 % DHCP4_SESSION_FAIL failed to establish BIND 10 session (%1), running stand-alone
 The server has failed to establish communication with the rest of BIND
 10 and is running in stand-alone mode.  (This behavior will change once
@@ -131,3 +215,13 @@ processed any command-line switches and is starting.
 This is a debug message issued during the IPv4 DHCP server startup.
 It lists some information about the parameters with which the server
 is running.
+
+% DHCP4_SUBNET_SELECTED the %1 subnet was selected for client assignment
+This is a debug message noting the selection of a subnet to be used for
+address and option assignment.  Subnet selection is one of the early
+steps in the processing of incoming client message.
+
+% DHCP4_SUBNET_SELECTION_FAILED failed to select a subnet for incoming packet, src: %1, type: %2
+This warning message is output when a packet was received from a subnet
+for which the DHCPv4 server has not been configured. The most probable
+cause is a misconfiguration of the server.

src/bin/dhcp4/dhcp4_srv.cc

@@ -16,9 +16,24 @@
 #include <dhcp/dhcp4.h>
 #include <dhcp/iface_mgr.h>
 #include <dhcp/option4_addrlst.h>
+#include <dhcp/option_int.h>
 #include <dhcp/pkt4.h>
+#include <dhcp/duid.h>
+#include <dhcp/hwaddr.h>
 #include <dhcp4/dhcp4_log.h>
 #include <dhcp4/dhcp4_srv.h>
+#include <dhcpsrv/utils.h>
+#include <dhcpsrv/cfgmgr.h>
+#include <dhcpsrv/lease_mgr.h>
+#include <dhcpsrv/lease_mgr_factory.h>
+#include <dhcpsrv/subnet.h>
+#include <dhcpsrv/utils.h>
+#include <dhcpsrv/addr_utilities.h>
+
+#include <boost/algorithm/string/erase.hpp>
+
+#include <iomanip>
+#include <fstream>
 
 using namespace isc;
 using namespace isc::asiolink;
@@ -28,25 +43,48 @@ using namespace std;
 
 // These are hardcoded parameters. Currently this is a skeleton server that only
 // grants those options and a single, fixed, hardcoded lease.
-const std::string HARDCODED_LEASE = "192.0.2.222"; // assigned lease
-const std::string HARDCODED_NETMASK = "255.255.255.0";
-const uint32_t    HARDCODED_LEASE_TIME = 60; // in seconds
 const std::string HARDCODED_GATEWAY = "192.0.2.1";
 const std::string HARDCODED_DNS_SERVER = "192.0.2.2";
 const std::string HARDCODED_DOMAIN_NAME = "isc.example.com";
-const std::string HARDCODED_SERVER_ID = "192.0.2.1";
 
-Dhcpv4Srv::Dhcpv4Srv(uint16_t port) {
+Dhcpv4Srv::Dhcpv4Srv(uint16_t port, const char* dbconfig) {
     LOG_DEBUG(dhcp4_logger, DBG_DHCP4_START, DHCP4_OPEN_SOCKET).arg(port);
     try {
         // First call to instance() will create IfaceMgr (it's a singleton)
         // it may throw something if things go wrong
         IfaceMgr::instance();
 
-        /// @todo: instantiate LeaseMgr here once it is imlpemented.
-        IfaceMgr::instance().openSockets4(port);
+        if (port) {
+            // open sockets only if port is non-zero. Port 0 is used
+            // for non-socket related testing.
+            IfaceMgr::instance().openSockets4(port);
+        }
+
+        string srvid_file = CfgMgr::instance().getDataDir() + "/" + string(SERVER_ID_FILE);
+        if (loadServerID(srvid_file)) {
+            LOG_DEBUG(dhcp4_logger, DBG_DHCP4_START, DHCP4_SERVERID_LOADED)
+                .arg(srvid_file);
+        } else {
+            generateServerID();
+            LOG_INFO(dhcp4_logger, DHCP4_SERVERID_GENERATED)
+                .arg(srvidToString(getServerID()))
+                .arg(srvid_file);
+
+            if (!writeServerID(srvid_file)) {
+                LOG_WARN(dhcp4_logger, DHCP4_SERVERID_WRITE_FAIL)
+                    .arg(srvid_file);
+            }
+
+        }
+
+        // Instantiate LeaseMgr
+        LeaseMgrFactory::create(dbconfig);
+        LOG_INFO(dhcp4_logger, DHCP4_DB_BACKEND_STARTED)
+            .arg(LeaseMgrFactory::instance().getType())
+            .arg(LeaseMgrFactory::instance().getName());
 
-        setServerID();
+        // Instantiate allocation engine
+        alloc_engine_.reset(new AllocEngine(AllocEngine::ALLOC_ITERATIVE, 100));
 
     } catch (const std::exception &e) {
         LOG_ERROR(dhcp4_logger, DHCP4_SRV_CONSTRUCT_ERROR).arg(e.what());
@@ -157,27 +195,109 @@ Dhcpv4Srv::run() {
                 }
             }
         }
+    }
+
+    return (true);
+}
 
-        // TODO add support for config session (see src/bin/auth/main.cc)
-        //      so this daemon can be controlled from bob
+bool Dhcpv4Srv::loadServerID(const std::string& file_name) {
+
+    // load content of the file into a string
+    fstream f(file_name.c_str(), ios::in);
+    if (!f.is_open()) {
+        return (false);
+    }
+
+    string hex_string;
+    f >> hex_string;
+    f.close();
+
+    // remove any spaces
+    boost::algorithm::erase_all(hex_string, " ");
+
+    try {
+        IOAddress addr(hex_string);
+
+        if (!addr.isV4()) {
+            return (false);
+        }
+
+        // Now create server-id option
+        serverid_.reset(new Option4AddrLst(DHO_DHCP_SERVER_IDENTIFIER, addr));
+
+    } catch(...) {
+        // any kind of malformed input (empty string, IPv6 address, complete
+        // garbate etc.)
+        return (false);
     }
 
     return (true);
 }
 
-void
-Dhcpv4Srv::setServerID() {
-    /// TODO implement this for real once interface detection (ticket 1237)
-    /// is done. Use hardcoded server-id for now.
-
-#if 0
-    // uncomment this once ticket 1350 is merged.
-    IOAddress srvId("127.0.0.1");
-    serverid_ = OptionPtr(
-      new Option4AddrLst(Option::V4, DHO_DHCP_SERVER_IDENTIFIER, srvId));
-#endif
+void Dhcpv4Srv::generateServerID() {
+
+    const IfaceMgr::IfaceCollection& ifaces = IfaceMgr::instance().getIfaces();
+
+    // Let's find suitable interface.
+    for (IfaceMgr::IfaceCollection::const_iterator iface = ifaces.begin();
+         iface != ifaces.end(); ++iface) {
+
+        // Let's don't use loopback.
+        if (iface->flag_loopback_) {
+            continue;
+        }
+
+        // Let's skip downed interfaces. It is better to use working ones.
+        if (!iface->flag_up_) {
+            continue;
+        }
+
+        const IfaceMgr::AddressCollection addrs = iface->getAddresses();
+
+        for (IfaceMgr::AddressCollection::const_iterator addr = addrs.begin();
+             addr != addrs.end(); ++addr) {
+            if (addr->getFamily() != AF_INET) {
+                continue;
+            }
+
+            serverid_ = OptionPtr(new Option4AddrLst(DHO_DHCP_SERVER_IDENTIFIER,
+                                                     *addr));
+            return;
+        }
+
+
+    }
+
+    isc_throw(BadValue, "No suitable interfaces for server-identifier found");
+}
+
+bool Dhcpv4Srv::writeServerID(const std::string& file_name) {
+    fstream f(file_name.c_str(), ios::out | ios::trunc);
+    if (!f.good()) {
+        return (false);
+    }
+    f << srvidToString(getServerID());
+    f.close();
 }
 
+string Dhcpv4Srv::srvidToString(const OptionPtr& srvid) {
+    if (!srvid) {
+        isc_throw(BadValue, "NULL pointer passed to srvidToString()");
+    }
+    boost::shared_ptr<Option4AddrLst> generated =
+        boost::dynamic_pointer_cast<Option4AddrLst>(srvid);
+    if (!srvid) {
+        isc_throw(BadValue, "Pointer to invalid option passed to srvidToString()");
+    }
+
+    Option4AddrLst::AddressContainer addrs = generated->getAddresses();
+    if (addrs.size() != 1) {
+        isc_throw(BadValue, "Malformed option passed to srvidToString(). "
+                  << "Expected to contain a single IPv4 address.");
+    }
+
+    return (addrs[0].toText());
+}
 
 void Dhcpv4Srv::copyDefaultFields(const Pkt4Ptr& question, Pkt4Ptr& answer) {
     answer->setIface(question->getIface());
@@ -188,9 +308,7 @@ void Dhcpv4Srv::copyDefaultFields(const Pkt4Ptr& question, Pkt4Ptr& answer) {
     answer->setHops(question->getHops());
 
     // copy MAC address
-    vector<uint8_t> mac(question->getChaddr(),
-                        question->getChaddr() + Pkt4::MAX_CHADDR_LEN);
-    answer->setHWAddr(question->getHtype(), question->getHlen(), mac);
+    answer->setHWAddr(question->getHWAddr());
 
     // relay address
     answer->setGiaddr(question->getGiaddr());
@@ -203,21 +321,21 @@ void Dhcpv4Srv::copyDefaultFields(const Pkt4Ptr& question, Pkt4Ptr& answer) {
         answer->setRemoteAddr(question->getRemoteAddr());
     }
 
+    // Let's copy client-id to response. See RFC6842.
+    OptionPtr client_id = question->getOption(DHO_DHCP_CLIENT_IDENTIFIER);
+    if (client_id) {
+        answer->addOption(client_id);
+    }
 }
 
 void Dhcpv4Srv::appendDefaultOptions(Pkt4Ptr& msg, uint8_t msg_type) {
     OptionPtr opt;
 
     // add Message Type Option (type 53)
-    std::vector<uint8_t> tmp;
-    tmp.push_back(static_cast<uint8_t>(msg_type));
-    opt = OptionPtr(new Option(Option::V4, DHO_DHCP_MESSAGE_TYPE, tmp));
-    msg->addOption(opt);
+    msg->setType(msg_type);
 
     // DHCP Server Identifier (type 54)
-    opt = OptionPtr
-        (new Option4AddrLst(DHO_DHCP_SERVER_IDENTIFIER, IOAddress(HARDCODED_SERVER_ID)));
-    msg->addOption(opt);
+    msg->addOption(getServerID());
 
     // more options will be added here later
 }
@@ -237,25 +355,109 @@ void Dhcpv4Srv::appendRequestedOptions(Pkt4Ptr& msg) {
     msg->addOption(opt);
 }
 
-void Dhcpv4Srv::tryAssignLease(Pkt4Ptr& msg) {
-    OptionPtr opt;
+void Dhcpv4Srv::assignLease(const Pkt4Ptr& question, Pkt4Ptr& answer) {
+
+    // We need to select a subnet the client is connected in.
+    Subnet4Ptr subnet = selectSubnet(question);
+    if (!subnet) {
+        // This particular client is out of luck today. We do not have
+        // information about the subnet he is connected to. This likely means
+        // misconfiguration of the server (or some relays). We will continue to
+        // process this message, but our response will be almost useless: no
+        // addresses or prefixes, no subnet specific configuration etc. The only
+        // thing this client can get is some global information (like DNS
+        // servers).
+
+        // perhaps this should be logged on some higher level? This is most likely
+        // configuration bug.
+        LOG_ERROR(dhcp4_logger, DHCP4_SUBNET_SELECTION_FAILED)
+            .arg(question->getRemoteAddr().toText())
+            .arg(serverReceivedPacketName(question->getType()));
+        answer->setType(DHCPNAK);
+        answer->setYiaddr(IOAddress("0.0.0.0"));
+        return;
+    }
 
-    // TODO: Implement actual lease assignment here
-    msg->setYiaddr(IOAddress(HARDCODED_LEASE));
+    LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL_DATA, DHCP4_SUBNET_SELECTED)
+        .arg(subnet->toText());
 
-    // IP Address Lease time (type 51)
-    opt = OptionPtr(new Option(Option::V4, DHO_DHCP_LEASE_TIME));
-    opt->setUint32(HARDCODED_LEASE_TIME);
-    msg->addOption(opt);
-    // TODO: create Option_IntArray that holds list of integers, similar to Option4_AddrLst
+    // Get client-id option
+    ClientIdPtr client_id;
+    OptionPtr opt = question->getOption(DHO_DHCP_CLIENT_IDENTIFIER);
+    if (opt) {
+        client_id = ClientIdPtr(new ClientId(opt->getData()));
+    }
+    // client-id is not mandatory in DHCPv4
+
+    IOAddress hint = question->getYiaddr();
+
+    HWAddrPtr hwaddr = question->getHWAddr();
+
+    // "Fake" allocation is processing of DISCOVER message. We pretend to do an
+    // allocation, but we do not put the lease in the database. That is ok,
+    // because we do not guarantee that the user will get that exact lease. If
+    // the user selects this server to do actual allocation (i.e. sends REQUEST)
+    // it should include this hint. That will help us during the actual lease
+    // allocation.
+    bool fake_allocation = (question->getType() == DHCPDISCOVER);
+
+    // Use allocation engine to pick a lease for this client. Allocation engine
+    // will try to honour the hint, but it is just a hint - some other address
+    // may be used instead. If fake_allocation is set to false, the lease will
+    // be inserted into the LeaseMgr as well.
+    Lease4Ptr lease = alloc_engine_->allocateAddress4(subnet, client_id, hwaddr,
+                                                      hint, fake_allocation);
+
+    if (lease) {
+        // We have a lease! Let's set it in the packet and send it back to
+        // the client.
+        LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL, fake_allocation?
+                  DHCP4_LEASE_ADVERT:DHCP4_LEASE_ALLOC)
+            .arg(lease->addr_.toText())
+            .arg(client_id?client_id->toText():"(no client-id)")
+            .arg(hwaddr?hwaddr->toText():"(no hwaddr info)");
+
+        answer->setYiaddr(lease->addr_);
+
+        // IP Address Lease time (type 51)
+        opt = OptionPtr(new Option(Option::V4, DHO_DHCP_LEASE_TIME));
+        opt->setUint32(lease->valid_lft_);
+        answer->addOption(opt);
+
+        // @todo: include real router information here
+        // Router (type 3)
+        opt = OptionPtr(new Option4AddrLst(DHO_ROUTERS, IOAddress(HARDCODED_GATEWAY)));
+        answer->addOption(opt);
+
+        // Subnet mask (type 1)
+        answer->addOption(getNetmaskOption(subnet));
+
+        // @todo: send renew timer option (T1, option 58)
+        // @todo: send rebind timer option (T2, option 59)
 
-    // Subnet mask (type 1)
-    opt = OptionPtr(new Option4AddrLst(DHO_SUBNET_MASK, IOAddress(HARDCODED_NETMASK)));
-    msg->addOption(opt);
+    } else {
+        // Allocation engine did not allocate a lease. The engine logged
+        // cause of that failure. The only thing left is to insert
+        // status code to pass the sad news to the client.
+
+        LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL, fake_allocation?
+                  DHCP4_LEASE_ADVERT_FAIL:DHCP4_LEASE_ALLOC_FAIL)
+            .arg(client_id?client_id->toText():"(no client-id)")
+            .arg(hwaddr?hwaddr->toText():"(no hwaddr info)")
+            .arg(hint.toText());
+
+        answer->setType(DHCPNAK);
+        answer->setYiaddr(IOAddress("0.0.0.0"));
+    }
+}
 
-    // Router (type 3)
-    opt = OptionPtr(new Option4AddrLst(DHO_ROUTERS, IOAddress(HARDCODED_GATEWAY)));
-    msg->addOption(opt);
+OptionPtr Dhcpv4Srv::getNetmaskOption(const Subnet4Ptr& subnet) {
+    uint32_t netmask = getNetmask4(subnet->get().second);
+
+    OptionPtr opt(new OptionInt<uint32_t>(Option::V4,
+                  DHO_SUBNET_MASK, netmask));
+
+    return (opt);
 }
 
 Pkt4Ptr Dhcpv4Srv::processDiscover(Pkt4Ptr& discover) {
@@ -266,7 +468,7 @@ Pkt4Ptr Dhcpv4Srv::processDiscover(Pkt4Ptr& discover) {
     appendDefaultOptions(offer, DHCPOFFER);
     appendRequestedOptions(offer);
 
-    tryAssignLease(offer);
+    assignLease(discover, offer);
 
     return (offer);
 }
@@ -279,13 +481,77 @@ Pkt4Ptr Dhcpv4Srv::processRequest(Pkt4Ptr& request) {
     appendDefaultOptions(ack, DHCPACK);
     appendRequestedOptions(ack);
 
-    tryAssignLease(ack);
+    assignLease(request, ack);
 
     return (ack);
 }
 
 void Dhcpv4Srv::processRelease(Pkt4Ptr& release) {
-    /// TODO: Implement this.
+
+    // Try to find client-id
+    ClientIdPtr client_id;
+    OptionPtr opt = release->getOption(DHO_DHCP_CLIENT_IDENTIFIER);
+    if (opt) {
+        client_id = ClientIdPtr(new ClientId(opt->getData()));
+    }
+
+    try {
+        // Do we have a lease for that particular address?
+        Lease4Ptr lease = LeaseMgrFactory::instance().getLease4(release->getYiaddr());
+
+        if (!lease) {
+            // No such lease - bogus release
+            LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL, DHCP4_RELEASE_FAIL_NO_LEASE)
+                .arg(release->getYiaddr().toText())
+                .arg(release->getHWAddr()->toText())
+                .arg(client_id ? client_id->toText() : "(no client-id)");
+            return;
+        }
+
+        // Does the hardware address match? We don't want one client releasing
+        // second client's leases.
+        if (lease->hwaddr_ != release->getHWAddr()->hwaddr_) {
+            // @todo: Print hwaddr from lease as part of ticket #2589
+            LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL, DHCP4_RELEASE_FAIL_WRONG_HWADDR)
+                .arg(release->getYiaddr().toText())
+                .arg(client_id ? client_id->toText() : "(no client-id)")
+                .arg(release->getHWAddr()->toText());
+            return;
+        }
+
+        // Does the lease have client-id info? If it has, then check it with what
+        // the client sent us.
+        if (lease->client_id_ && client_id && *lease->client_id_ != *client_id) {
+            LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL, DHCP4_RELEASE_FAIL_WRONG_CLIENT_ID)
+                .arg(release->getYiaddr().toText())
+                .arg(client_id->toText())
+                .arg(lease->client_id_->toText());
+            return;
+        }
+
+        // Ok, hw and client-id match - let's release the lease.
+        if (LeaseMgrFactory::instance().deleteLease(lease->addr_)) {
+
+            // Release successful - we're done here
+            LOG_DEBUG(dhcp4_logger, DBG_DHCP4_DETAIL, DHCP4_RELEASE)
+                .arg(lease->addr_.toText())
+                .arg(client_id ? client_id->toText() : "(no client-id)")
+                .arg(release->getHWAddr()->toText());
+        } else {
+
+            // Release failed -
+            LOG_ERROR(dhcp4_logger, DHCP4_RELEASE_FAIL)
+                .arg(lease->addr_.toText())
+                .arg(client_id ? client_id->toText() : "(no client-id)")
+                .arg(release->getHWAddr()->toText());
+        }
+    } catch (const isc::Exception& ex) {
+        // Rethrow the exception with a bit more data.
+        LOG_ERROR(dhcp4_logger, DHCP4_RELEASE_EXCEPTION)
+            .arg(ex.what())
+            .arg(release->getYiaddr());
+    }
+
 }
 
 void Dhcpv4Srv::processDecline(Pkt4Ptr& decline) {
@@ -327,3 +593,42 @@ Dhcpv4Srv::serverReceivedPacketName(uint8_t type) {
     }
     return (UNKNOWN);
 }
+
+Subnet4Ptr Dhcpv4Srv::selectSubnet(const Pkt4Ptr& question) {
+
+    // Is this relayed message?
+    IOAddress relay = question->getGiaddr();
+    if (relay.toText() == "0.0.0.0") {
+
+        // Yes: Use relay address to select subnet
+        return (CfgMgr::instance().getSubnet4(relay));
+    } else {
+
+        // No: Use client's address to select subnet
+        return (CfgMgr::instance().getSubnet4(question->getRemoteAddr()));
+    }
+}
+
+void Dhcpv4Srv::sanityCheck(const Pkt4Ptr& pkt, RequirementLevel serverid) {
+    OptionPtr server_id = pkt->getOption(DHO_DHCP_SERVER_IDENTIFIER);
+    switch (serverid) {
+    case FORBIDDEN:
+        if (server_id) {
+            isc_throw(RFCViolation, "Server-id option was not expected, but "
+                      << "received in " << serverReceivedPacketName(pkt->getType()));
+        }
+        break;
+
+    case MANDATORY:
+        if (!server_id) {
+            isc_throw(RFCViolation, "Server-id option was expected, but not "
+                      " received in message "
+                      << serverReceivedPacketName(pkt->getType()));
+        }
+        break;
+
+    case OPTIONAL:
+        // do nothing here
+        ;
+    }
+}

src/bin/dhcp4/dhcp4_srv.h

@@ -18,14 +18,25 @@
 #include <dhcp/dhcp4.h>
 #include <dhcp/pkt4.h>
 #include <dhcp/option.h>
+#include <dhcpsrv/subnet.h>
+#include <dhcpsrv/alloc_engine.h>
 
 #include <boost/noncopyable.hpp>
 
 #include <iostream>
 
 namespace isc {
-
 namespace dhcp {
+
+/// @brief file name of a server-id file
+///
+/// Server must store its server identifier in persistent storage that must not
+/// change between restarts. This is name of the file that is created in dataDir
+/// (see isc::dhcp::CfgMgr::getDataDir()). It is a text file that uses
+/// regular IPv4 address, e.g. 192.0.2.1. Server will create it during
+/// first run and then use it afterwards.
+static const char* SERVER_ID_FILE = "b10-dhcp4-serverid";
+
 /// @brief DHCPv4 server service.
 ///
 /// This singleton class represents DHCPv4 server. It contains all
@@ -44,6 +55,14 @@ namespace dhcp {
 class Dhcpv4Srv : public boost::noncopyable {
 
     public:
+
+    /// @brief defines if certain option may, must or must not appear
+    typedef enum {
+        FORBIDDEN,
+        MANDATORY,
+        OPTIONAL
+    } RequirementLevel;
+
     /// @brief Default constructor.
     ///
     /// Instantiates necessary services, required to run DHCPv4 server.
@@ -54,7 +73,10 @@ class Dhcpv4Srv : public boost::noncopyable {
     /// for testing purposes.
     ///
     /// @param port specifies port number to listen on
-    Dhcpv4Srv(uint16_t port = DHCP4_SERVER_PORT);
+    /// @param dbconfig Lease manager configuration string.  The default
+    ///        of the "memfile" manager is used for testing.
+    Dhcpv4Srv(uint16_t port = DHCP4_SERVER_PORT,
+              const char* dbconfig = "type=memfile");
 
     /// @brief Destructor. Used during DHCPv4 service shutdown.
     ~Dhcpv4Srv();
@@ -82,6 +104,8 @@ class Dhcpv4Srv : public boost::noncopyable {
     /// As the operation of the method does not depend on any server state, it
     /// is declared static.
     ///
+    /// @todo: This should be named static Pkt4::getName()
+    ///
     /// @param type DHCPv4 packet type
     ///
     /// @return Pointer to "const" string containing the packet name.
@@ -90,6 +114,17 @@ class Dhcpv4Srv : public boost::noncopyable {
     static const char* serverReceivedPacketName(uint8_t type);
 
 protected:
+
+    /// @brief verifies if specified packet meets RFC requirements
+    ///
+    /// Checks if mandatory option is really there, that forbidden option
+    /// is not there, and that client-id or server-id appears only once.
+    ///
+    /// @param pkt packet to be checked
+    /// @param serverid expectation regarding server-id option
+    /// @throw RFCViolation if any issues are detected
+    void sanityCheck(const Pkt4Ptr& pkt, RequirementLevel serverid);
+
     /// @brief Processes incoming DISCOVER and returns response.
     ///
     /// Processes received DISCOVER message and verifies that its sender
@@ -156,11 +191,19 @@ protected:
     /// client and assigning it. Options corresponding to the lease
     /// are added to specific message.
     ///
-    /// Note: Lease manager is not implemented yet, so this method
-    /// used fixed, hardcoded lease.
+    /// @param question DISCOVER or REQUEST message from client
+    /// @param answer OFFER or ACK/NAK message (lease options will be added here)
+    void assignLease(const Pkt4Ptr& question, Pkt4Ptr& answer);
+
+    /// @brief Attempts to renew received addresses
+    ///
+    /// Attempts to renew existing lease. This typically includes finding a lease that
+    /// corresponds to the received address. If no such lease is found, a status code
+    /// response is generated.
     ///
-    /// @param msg OFFER or ACK message (lease options will be added here)
-    void tryAssignLease(Pkt4Ptr& msg);
+    /// @param renew client's message asking for renew
+    /// @param reply server's response (ACK or NAK)
+    void renewLease(const Pkt4Ptr& renew, Pkt4Ptr& reply);
 
     /// @brief Appends default options to a message
     ///
@@ -182,7 +225,38 @@ protected:
     ///
     /// @throws isc::Unexpected Failed to obtain server identifier (i.e. no
     //          previously stored configuration and no network interfaces available)
-    void setServerID();
+    void generateServerID();
+
+    /// @brief attempts to load server-id from a file
+    ///
+    /// Tries to load duid from a text file. If the load is successful,
+    /// it creates server-id option and stores it in serverid_ (to be used
+    /// later by getServerID()).
+    ///
+    /// @param file_name name of the server-id file to load
+    /// @return true if load was successful, false otherwise
+    bool loadServerID(const std::string& file_name);
+
+    /// @brief attempts to write server-id to a file
+    /// Tries to write server-id content (stored in serverid_) to a text file.
+    ///
+    /// @param file_name name of the server-id file to write
+    /// @return true if write was successful, false otherwise
+    bool writeServerID(const std::string& file_name);
+
+    /// @brief converts server-id to text
+    /// Converts content of server-id option to a text representation, e.g.
+    /// "192.0.2.1"
+    ///
+    /// @param opt option that contains server-id
+    /// @return string representation
+    static std::string srvidToString(const OptionPtr& opt);
+
+    /// @brief Selects a subnet for a given client's packet.
+    ///
+    /// @param question client's message
+    /// @return selected subnet (or NULL if no suitable subnet was found)
+    isc::dhcp::Subnet4Ptr selectSubnet(const Pkt4Ptr& question);
 
     /// server DUID (to be sent in server-identifier option)
     OptionPtr serverid_;
@@ -190,6 +264,21 @@ protected:
     /// indicates if shutdown is in progress. Setting it to true will
     /// initiate server shutdown procedure.
     volatile bool shutdown_;
+
+    private:
+
+    /// @brief Constructs netmask option based on subnet4
+    /// @param subnet subnet for which the netmask will be calculated
+    ///
+    /// @return Option that contains netmask information
+    static OptionPtr getNetmaskOption(const Subnet4Ptr& subnet);
+
+    /// @brief Allocation Engine.
+    /// Pointer to the allocation engine that we are currently using
+    /// It must be a pointer, because we will support changing engines
+    /// during normal operation (e.g. to use different allocators)
+    boost::shared_ptr<AllocEngine> alloc_engine_;
+
 };
 
 }; // namespace isc::dhcp

src/bin/dhcp4/tests/config_parser_unittest.cc

@@ -1,4 +1,4 @@
-// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC")
+// Copyright (C) 2012-2013 Internet Systems Consortium, Inc. ("ISC")
 //
 // Permission to use, copy, modify, and/or distribute this software for any
 // purpose with or without fee is hereby granted, provided that the above
@@ -17,9 +17,10 @@
 #include <arpa/inet.h>
 #include <gtest/gtest.h>
 
+#include <config/ccsession.h>
 #include <dhcp4/dhcp4_srv.h>
 #include <dhcp4/config_parser.h>
-#include <config/ccsession.h>
+#include <dhcp/option4_addrlst.h>
 #include <dhcpsrv/subnet.h>
 #include <dhcpsrv/cfgmgr.h>
 #include <boost/foreach.hpp>
@@ -35,12 +36,6 @@ using namespace isc::asiolink;
 using namespace isc::data;
 using namespace isc::config;
 
-namespace isc {
-namespace dhcp {
-extern Uint32Storage uint32_defaults;
-}
-}
-
 namespace {
 
 class Dhcp4ParserTest : public ::testing::Test {
@@ -55,7 +50,9 @@ public:
 
     // Checks if global parameter of name have expected_value
     void checkGlobalUint32(string name, uint32_t expected_value) {
-        Uint32Storage::const_iterator it = uint32_defaults.find(name);
+        const std::map<std::string, uint32_t>& uint32_defaults = getUint32Defaults();
+        std::map<std::string, uint32_t>::const_iterator it =
+            uint32_defaults.find(name);
         if (it == uint32_defaults.end()) {
             ADD_FAILURE() << "Expected uint32 with name " << name
                           << " not found";
@@ -81,7 +78,8 @@ public:
     /// @brief Create the simple configuration with single option.
     ///
     /// This function allows to set one of the parameters that configure
-    /// option value. These parameters are: "name", "code" and "data".
+    /// option value. These parameters are: "name", "code", "data",
+    /// "csv-format" and "space".
     ///
     /// @param param_value string holiding option parameter value to be
     /// injected into the configuration string.
@@ -94,16 +92,34 @@ public:
         std::map<std::string, std::string> params;
         if (parameter == "name") {
             params["name"] = param_value;
+            params["space"] = "dhcp4";
             params["code"] = "56";
             params["data"] = "AB CDEF0105";
+            params["csv-format"] = "False";
+        } else if (parameter == "space") {
+            params["name"] = "dhcp-message";
+            params["space"] = param_value;
+            params["code"] = "56";
+            params["data"] = "AB CDEF0105";
+            params["csv-format"] = "False";
         } else if (parameter == "code") {
-            params["name"] = "option_foo";
+            params["name"] = "dhcp-message";
+            params["space"] = "dhcp4";
             params["code"] = param_value;
             params["data"] = "AB CDEF0105";
+            params["csv-format"] = "False";
         } else if (parameter == "data") {
-            params["name"] = "option_foo";
+            params["name"] = "dhcp-message";
+            params["space"] = "dhcp4";
             params["code"] = "56";
             params["data"] = param_value;
+            params["csv-format"] = "False";
+        } else if (parameter == "csv-format") {
+            params["name"] = "dhcp-message";
+            params["space"] = "dhcp4";
+            params["code"] = "56";
+            params["data"] = "AB CDEF0105";
+            params["csv-format"] = param_value;
         }
         return (createConfigWithOption(params));
     }
@@ -136,10 +152,14 @@ public:
             }
             if (param.first == "name") {
                 stream << "\"name\": \"" << param.second << "\"";
+            } else if (param.first == "space") {
+                stream << "\"space\": \"" << param.second << "\"";
             } else if (param.first == "code") {
                 stream << "\"code\": " << param.second << "";
             } else if (param.first == "data") {
                 stream << "\"data\": \"" << param.second << "\"";
+            } else if (param.first == "csv-format") {
+                stream << "\"csv-format\": " << param.second;
             }
         }
         stream <<
@@ -226,6 +246,7 @@ public:
             "\"renew-timer\": 1000, "
             "\"valid-lifetime\": 4000, "
             "\"subnet4\": [ ], "
+            "\"option-def\": [ ], "
             "\"option-data\": [ ] }";
 
         try {
@@ -393,9 +414,9 @@ TEST_F(Dhcp4ParserTest, poolOutOfSubnet) {
 
     EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
 
-    // returned value must be 2 (values error)
+    // returned value must be 1 (values error)
     // as the pool does not belong to that subnet
-    checkResult(status, 2);
+    checkResult(status, 1);
 }
 
 // Goal of this test is to verify if pools can be defined
@@ -428,6 +449,363 @@ TEST_F(Dhcp4ParserTest, poolPrefixLen) {
     EXPECT_EQ(4000, subnet->getValid());
 }
 
+// The goal of this test is to check whether an option definition
+// that defines an option carrying an IPv4 address can be created.
+TEST_F(Dhcp4ParserTest, optionDefIpv4Address) {
+
+    // Configuration string.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"ipv4-address\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Make sure that the particular option definition does not exist.
+    OptionDefinitionPtr def = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_FALSE(def);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+
+    // The option definition should now be available in the CfgMgr.
+    def = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_TRUE(def);
+
+    // Verify that the option definition data is valid.
+    EXPECT_EQ("foo", def->getName());
+    EXPECT_EQ(100, def->getCode());
+    EXPECT_FALSE(def->getArrayType());
+    EXPECT_EQ(OPT_IPV4_ADDRESS_TYPE, def->getType());
+}
+
+// The goal of this test is to check whether an option definiiton
+// that defines an option carrying a record of data fields can
+// be created.
+TEST_F(Dhcp4ParserTest, optionDefRecord) {
+
+    // Configuration string.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"record\","
+        "      \"array\": False,"
+        "      \"record-types\": \"uint16, ipv4-address, ipv6-address, string\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Make sure that the particular option definition does not exist.
+    OptionDefinitionPtr def = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_FALSE(def);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    checkResult(status, 0);
+
+    // The option definition should now be available in the CfgMgr.
+    def = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_TRUE(def);
+
+    // Check the option data.
+    EXPECT_EQ("foo", def->getName());
+    EXPECT_EQ(100, def->getCode());
+    EXPECT_EQ(OPT_RECORD_TYPE, def->getType());
+    EXPECT_FALSE(def->getArrayType());
+
+    // The option comprises the record of data fields. Verify that all
+    // fields are present and they are of the expected types.
+    const OptionDefinition::RecordFieldsCollection& record_fields =
+        def->getRecordFields();
+    ASSERT_EQ(4, record_fields.size());
+    EXPECT_EQ(OPT_UINT16_TYPE, record_fields[0]);
+    EXPECT_EQ(OPT_IPV4_ADDRESS_TYPE, record_fields[1]);
+    EXPECT_EQ(OPT_IPV6_ADDRESS_TYPE, record_fields[2]);
+    EXPECT_EQ(OPT_STRING_TYPE, record_fields[3]);
+}
+
+// The goal of this test is to verify that multiple option definitions
+// can be created.
+TEST_F(Dhcp4ParserTest, optionDefMultiple) {
+    // Configuration string.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"uint32\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  },"
+        "  {"
+        "      \"name\": \"foo-2\","
+        "      \"code\": 101,"
+        "      \"type\": \"ipv4-address\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Make sure that the option definitions do not exist yet.
+    ASSERT_FALSE(CfgMgr::instance().getOptionDef("isc", 100));
+    ASSERT_FALSE(CfgMgr::instance().getOptionDef("isc", 101));
+
+    // Use the configuration string to create new option definitions.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    checkResult(status, 0);
+
+    // Check the first definition we have created.
+    OptionDefinitionPtr def1 = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_TRUE(def1);
+
+    // Check the option data.
+    EXPECT_EQ("foo", def1->getName());
+    EXPECT_EQ(100, def1->getCode());
+    EXPECT_EQ(OPT_UINT32_TYPE, def1->getType());
+    EXPECT_FALSE(def1->getArrayType());
+
+    // Check the second option definition we have created.
+    OptionDefinitionPtr def2 = CfgMgr::instance().getOptionDef("isc", 101);
+    ASSERT_TRUE(def2);
+
+    // Check the option data.
+    EXPECT_EQ("foo-2", def2->getName());
+    EXPECT_EQ(101, def2->getCode());
+    EXPECT_EQ(OPT_IPV4_ADDRESS_TYPE, def2->getType());
+    EXPECT_FALSE(def2->getArrayType());
+}
+
+// The goal of this test is to verify that the duplicated option
+// definition is not accepted.
+TEST_F(Dhcp4ParserTest, optionDefDuplicate) {
+
+    // Configuration string. Both option definitions have
+    // the same code and belong to the same option space.
+    // This configuration should not be accepted.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"uint32\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  },"
+        "  {"
+        "      \"name\": \"foo-2\","
+        "      \"code\": 100,"
+        "      \"type\": \"ipv4-address\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Make sure that the option definition does not exist yet.
+    ASSERT_FALSE(CfgMgr::instance().getOptionDef("isc", 100));
+
+    // Use the configuration string to create new option definitions.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    checkResult(status, 1);
+}
+
+// The goal of this test is to verify that the option definition
+// comprising an array of uint32 values can be created.
+TEST_F(Dhcp4ParserTest, optionDefArray) {
+
+    // Configuration string. Created option definition should
+    // comprise an array of uint32 values.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"uint32\","
+        "      \"array\": True,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Make sure that the particular option definition does not exist.
+    OptionDefinitionPtr def = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_FALSE(def);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    checkResult(status, 0);
+
+    // The option definition should now be available in the CfgMgr.
+    def = CfgMgr::instance().getOptionDef("isc", 100);
+    ASSERT_TRUE(def);
+
+    // Check the option data.
+    EXPECT_EQ("foo", def->getName());
+    EXPECT_EQ(100, def->getCode());
+    EXPECT_EQ(OPT_UINT32_TYPE, def->getType());
+    EXPECT_TRUE(def->getArrayType());
+}
+
+/// The purpose of this test is to verify that the option definition
+/// with invalid name is not accepted.
+TEST_F(Dhcp4ParserTest, optionDefInvalidName) {
+    // Configuration string. The option name is invalid as it
+    // contains the % character.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"invalid%name\","
+        "      \"code\": 100,"
+        "      \"type\": \"string\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    // Expecting parsing error (error code 1).
+    checkResult(status, 1);
+}
+
+/// The purpose of this test is to verify that the option definition
+/// with invalid type is not accepted.
+TEST_F(Dhcp4ParserTest, optionDefInvalidType) {
+    // Configuration string. The option type is invalid. It is
+    // "sting" instead of "string".
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"sting\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    // Expecting parsing error (error code 1).
+    checkResult(status, 1);
+}
+
+/// The purpose of this test is to verify that the option definition
+/// with invalid type is not accepted.
+TEST_F(Dhcp4ParserTest, optionDefInvalidRecordType) {
+    // Configuration string. The third of the record fields
+    // is invalid. It is "sting" instead of "string".
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"record\","
+        "      \"array\": False,"
+        "      \"record-types\": \"uint32,uint8,sting\","
+        "      \"space\": \"isc\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    // Expecting parsing error (error code 1).
+    checkResult(status, 1);
+}
+
+
+/// The purpose of this test is to verify that it is not allowed
+/// to override the standard option (that belongs to dhcp4 option
+/// space) and that it is allowed to define option in the dhcp4
+/// option space that has a code which is not used by any of the
+/// standard options.
+TEST_F(Dhcp4ParserTest, optionStandardDefOverride) {
+
+    // Configuration string. The option code 109 is unassigned
+    // so it can be used for a custom option definition in
+    // dhcp4 option space.
+    std::string config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 109,"
+        "      \"type\": \"string\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"dhcp4\""
+        "  } ]"
+        "}";
+    ElementPtr json = Element::fromJSON(config);
+
+    OptionDefinitionPtr def = CfgMgr::instance().getOptionDef("dhcp4", 109);
+    ASSERT_FALSE(def);
+
+    // Use the configuration string to create new option definition.
+    ConstElementPtr status;
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    checkResult(status, 0);
+
+    // The option definition should now be available in the CfgMgr.
+    def = CfgMgr::instance().getOptionDef("dhcp4", 109);
+    ASSERT_TRUE(def);
+
+    // Check the option data.
+    EXPECT_EQ("foo", def->getName());
+    EXPECT_EQ(109, def->getCode());
+    EXPECT_EQ(OPT_STRING_TYPE, def->getType());
+    EXPECT_FALSE(def->getArrayType());
+
+    // The combination of option space and code is
+    // invalid. The 'dhcp4' option space groups
+    // standard options and the code 100 is reserved
+    // for one of them.
+    config =
+        "{ \"option-def\": [ {"
+        "      \"name\": \"foo\","
+        "      \"code\": 100,"
+        "      \"type\": \"string\","
+        "      \"array\": False,"
+        "      \"record-types\": \"\","
+        "      \"space\": \"dhcp4\""
+        "  } ]"
+        "}";
+    json = Element::fromJSON(config);
+
+    // Use the configuration string to create new option definition.
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    // Expecting parsing error (error code 1).
+    checkResult(status, 1);
+}
+
 // Goal of this test is to verify that global option
 // data is configured for the subnet if the subnet
 // configuration does not include options configuration.
@@ -437,14 +815,18 @@ TEST_F(Dhcp4ParserTest, optionDataDefaults) {
         "\"rebind-timer\": 2000,"
         "\"renew-timer\": 1000,"
         "\"option-data\": [ {"
-        "    \"name\": \"option_foo\","
+        "    \"name\": \"dhcp-message\","
+        "    \"space\": \"dhcp4\","
         "    \"code\": 56,"
-        "    \"data\": \"AB CDEF0105\""
+        "    \"data\": \"AB CDEF0105\","
+        "    \"csv-format\": False"
         " },"
         " {"
-        "    \"name\": \"option_foo2\","
+        "    \"name\": \"default-ip-ttl\","
+        "    \"space\": \"dhcp4\","
         "    \"code\": 23,"
-        "    \"data\": \"01\""
+        "    \"data\": \"01\","
+        "    \"csv-format\": False"
         " } ],"
         "\"subnet4\": [ { "
         "    \"pool\": [ \"192.0.2.1 - 192.0.2.100\" ],"
@@ -461,11 +843,11 @@ TEST_F(Dhcp4ParserTest, optionDataDefaults) {
 
     Subnet4Ptr subnet = CfgMgr::instance().getSubnet4(IOAddress("192.0.2.200"));
     ASSERT_TRUE(subnet);
-    const Subnet::OptionContainer& options = subnet->getOptions();
-    ASSERT_EQ(2, options.size());
+    Subnet::OptionContainerPtr options = subnet->getOptionDescriptors("dhcp4");
+    ASSERT_EQ(2, options->size());
 
     // Get the search index. Index #1 is to search using option code.
-    const Subnet::OptionContainerTypeIndex& idx = options.get<1>();
+    const Subnet::OptionContainerTypeIndex& idx = options->get<1>();
 
     // Get the options for specified index. Expecting one option to be
     // returned but in theory we may have multiple options with the same
@@ -490,6 +872,74 @@ TEST_F(Dhcp4ParserTest, optionDataDefaults) {
     testOption(*range.first, 23, foo2_expected, sizeof(foo2_expected));
 }
 
+/// The goal of this test is to verify that two options having the same
+/// option code can be added to different option spaces.
+TEST_F(Dhcp4ParserTest, optionDataTwoSpaces) {
+
+    // This configuration string is to configure two options
+    // sharing the code 56 and having different definitions
+    // and belonging to the different option spaces.
+    // The option definition must be provided for the
+    // option that belongs to the 'isc' option space.
+    // The definition is not required for the option that
+    // belongs to the 'dhcp4' option space as it is the
+    // standard option.
+    string config = "{ \"interface\": [ \"all\" ],"
+        "\"rebind-timer\": 2000,"
+        "\"renew-timer\": 1000,"
+        "\"option-data\": [ {"
+        "    \"name\": \"dhcp-message\","
+        "    \"space\": \"dhcp4\","
+        "    \"code\": 56,"
+        "    \"data\": \"AB CDEF0105\","
+        "    \"csv-format\": False"
+        " },"
+        " {"
+        "    \"name\": \"foo\","
+        "    \"space\": \"isc\","
+        "    \"code\": 56,"
+        "    \"data\": \"1234\","
+        "    \"csv-format\": True"
+        " } ],"
+        "\"option-def\": [ {"
+        "    \"name\": \"foo\","
+        "    \"code\": 56,"
+        "    \"type\": \"uint32\","
+        "    \"array\": False,"
+        "    \"record-types\": \"\","
+        "    \"space\": \"isc\""
+        " } ],"
+        "\"subnet4\": [ { "
+        "    \"pool\": [ \"192.0.2.1 - 192.0.2.100\" ],"
+        "    \"subnet\": \"192.0.2.0/24\""
+        " } ]"
+        "}";
+
+    ConstElementPtr status;
+
+    ElementPtr json = Element::fromJSON(config);
+
+    EXPECT_NO_THROW(status = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(status);
+    checkResult(status, 0);
+
+    // Options should be now availabe for the subnet.
+    Subnet4Ptr subnet = CfgMgr::instance().getSubnet4(IOAddress("192.0.2.200"));
+    ASSERT_TRUE(subnet);
+    // Try to get the option from the space dhcp4.
+    Subnet::OptionDescriptor desc1 = subnet->getOptionDescriptor("dhcp4", 56);
+    ASSERT_TRUE(desc1.option);
+    EXPECT_EQ(56, desc1.option->getType());
+    // Try to get the option from the space isc.
+    Subnet::OptionDescriptor desc2 = subnet->getOptionDescriptor("isc", 56);
+    ASSERT_TRUE(desc2.option);
+    EXPECT_EQ(56, desc1.option->getType());
+    // Try to get the non-existing option from the non-existing
+    // option space and  expect that option is not returned.
+    Subnet::OptionDescriptor desc3 = subnet->getOptionDescriptor("non-existing", 56);
+    ASSERT_FALSE(desc3.option);
+}
+
 // Goal of this test is to verify options configuration
 // for a single subnet. In particular this test checks
 // that local options configuration overrides global
@@ -500,22 +950,28 @@ TEST_F(Dhcp4ParserTest, optionDataInSingleSubnet) {
         "\"rebind-timer\": 2000, "
         "\"renew-timer\": 1000, "
         "\"option-data\": [ {"
-        "      \"name\": \"option_foo\","
+        "      \"name\": \"dhcp-message\","
+        "      \"space\": \"dhcp4\","
         "      \"code\": 56,"
-        "      \"data\": \"AB\""
+        "      \"data\": \"AB\","
+        "      \"csv-format\": False"
         " } ],"
         "\"subnet4\": [ { "
         "    \"pool\": [ \"192.0.2.1 - 192.0.2.100\" ],"
         "    \"subnet\": \"192.0.2.0/24\", "
         "    \"option-data\": [ {"
-        "          \"name\": \"option_foo\","
+        "          \"name\": \"dhcp-message\","
+        "          \"space\": \"dhcp4\","
         "          \"code\": 56,"
-        "          \"data\": \"AB CDEF0105\""
+        "          \"data\": \"AB CDEF0105\","
+        "          \"csv-format\": False"
         "        },"
         "        {"
-        "          \"name\": \"option_foo2\","
+        "          \"name\": \"default-ip-ttl\","
+        "          \"space\": \"dhcp4\","
         "          \"code\": 23,"
-        "          \"data\": \"01\""
+        "          \"data\": \"01\","
+        "          \"csv-format\": False"
         "        } ]"
         " } ],"
         "\"valid-lifetime\": 4000 }";
@@ -529,11 +985,11 @@ TEST_F(Dhcp4ParserTest, optionDataInSingleSubnet) {
 
     Subnet4Ptr subnet = CfgMgr::instance().getSubnet4(IOAddress("192.0.2.24"));
     ASSERT_TRUE(subnet);
-    const Subnet::OptionContainer& options = subnet->getOptions();
-    ASSERT_EQ(2, options.size());
+    Subnet::OptionContainerPtr options = subnet->getOptionDescriptors("dhcp4");
+    ASSERT_EQ(2, options->size());
 
     // Get the search index. Index #1 is to search using option code.
-    const Subnet::OptionContainerTypeIndex& idx = options.get<1>();
+    const Subnet::OptionContainerTypeIndex& idx = options->get<1>();
 
     // Get the options for specified index. Expecting one option to be
     // returned but in theory we may have multiple options with the same
@@ -569,18 +1025,22 @@ TEST_F(Dhcp4ParserTest, optionDataInMultipleSubnets) {
         "    \"pool\": [ \"192.0.2.1 - 192.0.2.100\" ],"
         "    \"subnet\": \"192.0.2.0/24\", "
         "    \"option-data\": [ {"
-        "          \"name\": \"option_foo\","
+        "          \"name\": \"dhcp-message\","
+        "          \"space\": \"dhcp4\","
         "          \"code\": 56,"
-        "          \"data\": \"0102030405060708090A\""
+        "          \"data\": \"0102030405060708090A\","
+        "          \"csv-format\": False"
         "        } ]"
         " },"
         " {"
         "    \"pool\": [ \"192.0.3.101 - 192.0.3.150\" ],"
         "    \"subnet\": \"192.0.3.0/24\", "
         "    \"option-data\": [ {"
-        "          \"name\": \"option_foo2\","
+        "          \"name\": \"default-ip-ttl\","
+        "          \"space\": \"dhcp4\","
         "          \"code\": 23,"
-        "          \"data\": \"FF\""
+        "          \"data\": \"FF\","
+        "          \"csv-format\": False"
         "        } ]"
         " } ],"
         "\"valid-lifetime\": 4000 }";
@@ -594,11 +1054,11 @@ TEST_F(Dhcp4ParserTest, optionDataInMultipleSubnets) {
 
     Subnet4Ptr subnet1 = CfgMgr::instance().getSubnet4(IOAddress("192.0.2.100"));
     ASSERT_TRUE(subnet1);
-    const Subnet::OptionContainer& options1 = subnet1->getOptions();
-    ASSERT_EQ(1, options1.size());
+    Subnet::OptionContainerPtr options1 = subnet1->getOptionDescriptors("dhcp4");
+    ASSERT_EQ(1, options1->size());
 
     // Get the search index. Index #1 is to search using option code.
-    const Subnet::OptionContainerTypeIndex& idx1 = options1.get<1>();
+    const Subnet::OptionContainerTypeIndex& idx1 = options1->get<1>();
 
     // Get the options for specified index. Expecting one option to be
     // returned but in theory we may have multiple options with the same
@@ -618,10 +1078,10 @@ TEST_F(Dhcp4ParserTest, optionDataInMultipleSubnets) {
     // Test another subnet in the same way.
     Subnet4Ptr subnet2 = CfgMgr::instance().getSubnet4(IOAddress("192.0.3.102"));
     ASSERT_TRUE(subnet2);
-    const Subnet::OptionContainer& options2 = subnet2->getOptions();
-    ASSERT_EQ(1, options2.size());
+    Subnet::OptionContainerPtr options2 = subnet2->getOptionDescriptors("dhcp4");
+    ASSERT_EQ(1, options2->size());
 
-    const Subnet::OptionContainerTypeIndex& idx2 = options2.get<1>();
+    const Subnet::OptionContainerTypeIndex& idx2 = options2->get<1>();
     std::pair<Subnet::OptionContainerTypeIndex::const_iterator,
               Subnet::OptionContainerTypeIndex::const_iterator> range2 =
         idx2.equal_range(23);
@@ -703,11 +1163,11 @@ TEST_F(Dhcp4ParserTest, optionDataLowerCase) {
 
     Subnet4Ptr subnet = CfgMgr::instance().getSubnet4(IOAddress("192.0.2.5"));
     ASSERT_TRUE(subnet);
-    const Subnet::OptionContainer& options = subnet->getOptions();
-    ASSERT_EQ(1, options.size());
+    Subnet::OptionContainerPtr options = subnet->getOptionDescriptors("dhcp4");
+    ASSERT_EQ(1, options->size());
 
     // Get the search index. Index #1 is to search using option code.
-    const Subnet::OptionContainerTypeIndex& idx = options.get<1>();
+    const Subnet::OptionContainerTypeIndex& idx = options->get<1>();
 
     // Get the options for specified index. Expecting one option to be
     // returned but in theory we may have multiple options with the same
@@ -724,10 +1184,73 @@ TEST_F(Dhcp4ParserTest, optionDataLowerCase) {
     testOption(*range.first, 56, foo_expected, sizeof(foo_expected));
 }
 
+// Verify that specific option object is returned for standard
+// option which has dedicated option class derived from Option.
+TEST_F(Dhcp4ParserTest, stdOptionData) {
+    ConstElementPtr x;
+    std::map<std::string, std::string> params;
+    params["name"] = "nis-servers";
+    params["space"] = "dhcp4";
+    // Option code 41 means nis-servers.
+    params["code"] = "41";
+    // Specify option values in a CSV (user friendly) format.
+    params["data"] = "192.0.2.10, 192.0.2.1, 192.0.2.3";
+    params["csv-format"] = "True";
+
+    std::string config = createConfigWithOption(params);
+    ElementPtr json = Element::fromJSON(config);
+
+    EXPECT_NO_THROW(x = configureDhcp4Server(*srv_, json));
+    ASSERT_TRUE(x);
+    comment_ = parseAnswer(rcode_, x);
+    ASSERT_EQ(0, rcode_);
+
+    Subnet4Ptr subnet = CfgMgr::instance().getSubnet4(IOAddress("192.0.2.5"));
+    ASSERT_TRUE(subnet);
+    Subnet::OptionContainerPtr options =
+        subnet->getOptionDescriptors("dhcp4");
+    ASSERT_TRUE(options);
+    ASSERT_EQ(1, options->size());
+
+    // Get the search index. Index #1 is to search using option code.
+    const Subnet::OptionContainerTypeIndex& idx = options->get<1>();
+
+    // Get the options for specified index. Expecting one option to be
+    // returned but in theory we may have multiple options with the same
+    // code so we get the range.
+    std::pair<Subnet::OptionContainerTypeIndex::const_iterator,
+              Subnet::OptionContainerTypeIndex::const_iterator> range =
+        idx.equal_range(DHO_NIS_SERVERS);
+    // Expect single option with the code equal to NIS_SERVERS option code.
+    ASSERT_EQ(1, std::distance(range.first, range.second));
+    // The actual pointer to the option is held in the option field
+    // in the structure returned.
+    OptionPtr option = range.first->option;
+    ASSERT_TRUE(option);
+    // Option object returned for here is expected to be Option6IA
+    // which is derived from Option. This class is dedicated to
+    // represent standard option IA_NA.
+    boost::shared_ptr<Option4AddrLst> option_addrs =
+        boost::dynamic_pointer_cast<Option4AddrLst>(option);
+    // If cast is unsuccessful than option returned was of a
+    // differnt type than Option6IA. This is wrong.
+    ASSERT_TRUE(option_addrs);
+
+    // Get addresses from the option.
+    Option4AddrLst::AddressContainer addrs = option_addrs->getAddresses();
+    // Verify that the addresses have been configured correctly.
+    ASSERT_EQ(3, addrs.size());
+    EXPECT_EQ("192.0.2.10", addrs[0].toText());
+    EXPECT_EQ("192.0.2.1", addrs[1].toText());
+    EXPECT_EQ("192.0.2.3", addrs[2].toText());
+}
+
 /// This test checks if Uint32Parser can really parse the whole range
 /// and properly err of out of range values. As we can't call Uint32Parser
 /// directly, we are exploiting the fact that it is used to parse global
 /// parameter renew-timer and the results are stored in uint32_defaults.
+/// We get the uint32_defaults using a getUint32Defaults functions which
+/// is defined only to access the values from this test.
 TEST_F(Dhcp4ParserTest, DISABLED_Uint32Parser) {
 
     ConstElementPtr status;

src/bin/dhcp4/tests/dhcp4_unittests.cc

@@ -13,13 +13,16 @@
 // PERFORMANCE OF THIS SOFTWARE.
 
 #include <log/logger_support.h>
-
+#include <dhcp4/dhcp4_log.h>
 #include <gtest/gtest.h>
 
 int
 main(int argc, char* argv[]) {
 
     ::testing::InitGoogleTest(&argc, argv);
+
+    // See the documentation of the B10_* environment variables in
+    // src/lib/log/README for info on how to tweak logging
     isc::log::initLogger();
 
     int result = RUN_ALL_TESTS();

src/bin/dhcp6/config_parser.h

@@ -1,4 +1,4 @@
-// Copyright (C) 2012  Internet Systems Consortium, Inc. ("ISC")
+// Copyright (C) 2012-2013 Internet Systems Consortium, Inc. ("ISC")
 //
 // Permission to use, copy, modify, and/or distribute this software for any
 // purpose with or without fee is hereby granted, provided that the above
@@ -27,129 +27,25 @@ namespace dhcp {
 
 class Dhcpv6Srv;
 
-/// An exception that is thrown if an error occurs while configuring an
-/// \c Dhcpv6Srv object.
-class Dhcp6ConfigError : public isc::Exception {
-public:
-
-    /// @brief constructor
-    ///
-    /// @param file name of the file, where exception occurred
-    /// @param line line of the file, where exception occurred
-    /// @param what text description of the issue that caused exception
-    Dhcp6ConfigError(const char* file, size_t line, const char* what)
-        : isc::Exception(file, line, what) {}
-};
-
-/// @brief Base abstract class for all DHCPv6 parsers
-///
-/// Each instance of a class derived from this class parses one specific config
-/// element. Sometimes elements are simple (e.g. a string) and sometimes quite
-/// complex (e.g. a subnet). In such case, it is likely that a parser will
-/// spawn child parsers to parse child elements in the configuration.
-/// @todo: Merge this class with Dhcp4ConfigParser in src/bin/dhcp4
-class DhcpConfigParser {
-    ///
-    /// \name Constructors and Destructor
-    ///
-    /// Note: The copy constructor and the assignment operator are
-    /// intentionally defined as private to make it explicit that this is a
-    /// pure base class.
-    //@{
-private:
-    DhcpConfigParser(const DhcpConfigParser& source);
-    DhcpConfigParser& operator=(const DhcpConfigParser& source);
-protected:
-    /// \brief The default constructor.
-    ///
-    /// This is intentionally defined as \c protected as this base class should
-    /// never be instantiated (except as part of a derived class).
-    DhcpConfigParser() {}
-public:
-    /// The destructor.
-    virtual ~DhcpConfigParser() {}
-    //@}
-
-    /// \brief Prepare configuration value.
-    ///
-    /// This method parses the "value part" of the configuration identifier
-    /// that corresponds to this derived class and prepares a new value to
-    /// apply to the server.
-    ///
-    /// This method must validate the given value both in terms of syntax
-    /// and semantics of the configuration, so that the server will be
-    /// validly configured at the time of \c commit().  Note: the given
-    /// configuration value is normally syntactically validated, but the
-    /// \c build() implementation must also expect invalid input.  If it
-    /// detects an error it may throw an exception of a derived class
-    /// of \c isc::Exception.
-    ///
-    /// Preparing a configuration value will often require resource
-    /// allocation.  If it fails, it may throw a corresponding standard
-    /// exception.
-    ///
-    /// This method is not expected to be called more than once in the
-    /// life of the object. Although multiple calls are not prohibited
-    /// by the interface, the behavior is undefined.
-    ///
-    /// \param config_value The configuration value for the identifier
-    /// corresponding to the derived class.
-    virtual void build(isc::data::ConstElementPtr config_value) = 0;
-
-    /// \brief Apply the prepared configuration value to the server.
-    ///
-    /// This method is expected to be exception free, and, as a consequence,
-    /// it should normally not involve resource allocation.
-    /// Typically it would simply perform exception free assignment or swap
-    /// operation on the value prepared in \c build().
-    /// In some cases, however, it may be very difficult to meet this
-    /// condition in a realistic way, while the failure case should really
-    /// be very rare.  In such a case it may throw, and, if the parser is
-    /// called via \c configureDhcp6Server(), the caller will convert the
-    /// exception as a fatal error.
-    ///
-    /// This method is expected to be called after \c build(), and only once.
-    /// The result is undefined otherwise.
-    virtual void commit() = 0;
-};
-
-/// @brief a pointer to configuration parser
-typedef boost::shared_ptr<DhcpConfigParser> ParserPtr;
-
-/// @brief a collection of parsers
-///
-/// This container is used to store pointer to parsers for a given scope.
-typedef std::vector<ParserPtr> ParserCollection;
-
-
-/// \brief Configure an \c Dhcpv6Srv object with a set of configuration values.
+/// @brief Configures DHCPv6 server
 ///
-/// This function parses configuration information stored in \c config_set
-/// and configures the \c server by applying the configuration to it.
-/// It provides the strong exception guarantee as long as the underlying
-/// derived class implementations of \c DhcpConfigParser meet the assumption,
-/// that is, it ensures that either configuration is fully applied or the
-/// state of the server is intact.
+/// This function is called every time a new configuration is received. The extra
+/// parameter is a reference to DHCPv6 server component. It is currently not used
+/// and CfgMgr::instance() is accessed instead.
 ///
-/// If a syntax or semantics level error happens during the configuration
-/// (such as malformed configuration or invalid configuration parameter),
-/// this function throws an exception of class \c Dhcp6ConfigError.
-/// If the given configuration requires resource allocation and it fails,
-/// a corresponding standard exception will be thrown.
-/// Other exceptions may also be thrown, depending on the implementation of
-/// the underlying derived class of \c Dhcp6ConfigError.
-/// In any case the strong guarantee is provided as described above except
-/// in the very rare cases where the \c commit() method of a parser throws
-/// an exception.  If that happens this function converts the exception
-/// into a \c FatalError exception and rethrows it.  This exception is
-/// expected to be caught at the highest level of the application to terminate
-/// the program gracefully.
+/// This method does not throw. It catches all exceptions and returns them as
+/// reconfiguration statuses. It may return the following response codes:
+/// 0 - configuration successful
+/// 1 - malformed configuration (parsing failed)
+/// 2 - commit failed (parsing was successful, but the values could not be
+/// stored in the configuration).
 ///
-/// \param server The \c Dhcpv6Srv object to be configured.
-/// \param config_set A JSON style configuration to apply to \c server.
+/// @param server DHCPv6 server object.
+/// @param config_set a new configuration for DHCPv6 server.
+/// @return answer that contains result of the reconfiguration.
+/// @throw Dhcp6ConfigError if trying to create a parser for NULL config.
 isc::data::ConstElementPtr
-configureDhcp6Server(Dhcpv6Srv& server,
-                     isc::data::ConstElementPtr config_set);
+configureDhcp6Server(Dhcpv6Srv& server, isc::data::ConstElementPtr config_set);
 
 }; // end of isc::dhcp namespace
 }; // end of isc namespace

src/bin/dhcp6/ctrl_dhcp6_srv.cc

@@ -19,6 +19,7 @@
 #include <cc/session.h>
 #include <config/ccsession.h>
 #include <dhcp/iface_mgr.h>
+#include <dhcpsrv/dhcp_config_parser.h>
 #include <dhcp6/config_parser.h>
 #include <dhcp6/ctrl_dhcp6_srv.h>
 #include <dhcp6/dhcp6_log.h>
@@ -41,7 +42,6 @@ using namespace std;
 namespace isc {
 namespace dhcp {
 
-
 ControlledDhcpv6Srv* ControlledDhcpv6Srv::server_ = NULL;
 
 ConstElementPtr
@@ -121,7 +121,7 @@ void ControlledDhcpv6Srv::establishSession() {
 
     try {
         configureDhcp6Server(*this, config_session_->getFullConfig());
-    } catch (const Dhcp6ConfigError& ex) {
+    } catch (const DhcpConfigError& ex) {
         LOG_ERROR(dhcp6_logger, DHCP6_CONFIG_LOAD_FAIL).arg(ex.what());
     }
 

src/bin/dhcp6/dhcp6.dox

@@ -35,19 +35,19 @@
 
  This method iterates over list of received configuration elements and creates a
  list of parsers for each received entry. Parser is an object that is derived
- from a \ref isc::dhcp::DhcpConfigParser class. Once a parser is created
+ from a DhcpConfigParser class. Once a parser is created
  (constructor), its value is set (using build() method). Once all parsers are
  build, the configuration is then applied ("commited") and commit() method is
  called.
 
  All parsers are defined in src/bin/dhcp6/config_parser.cc file. Some of them
- are generic (e.g. \ref isc::dhcp::Uint32Parser that is able to handle any
- unsigned 32 bit integer), but some are very specialized (e.g. \ref
- isc::dhcp::Subnets6ListConfigParser parses definitions of Subnet6 lists). In
- some cases, e.g. subnet6 definitions, the configuration entry is not a simple
- value, but a map or a list itself. In such case, the parser iterates over all
- elements and creates parsers for a given scope. This process may be repeated
- (sort of) recursively.
+ are generic (e.g. Uint32Parser that is able to handle any
+ unsigned 32 bit integer), but some are very specialized (e.g.
+ Subnets6ListConfigParser parses definitions of Subnet6 lists). In some cases,
+ e.g. subnet6 definitions, the configuration entry is not a simple value, but
+ a map or a list itself. In such case, the parser iterates over all elements
+ and creates parsers for a given scope. This process may be repeated (sort of)
+ recursively.
 
  @section dhcpv6ConfigInherit DHCPv6 Configuration Inheritance
 
@@ -55,16 +55,16 @@
  For example, renew-timer value may be specified at a global scope and it then
  applies to all subnets. However, some subnets may have it overwritten with more
  specific values that takes precedence over global values that are considered
- defaults. Some parsers (e.g. \ref isc::dhcp::Uint32Parser and \ref
- isc::dhcp::StringParser) implement that inheritance. By default, they store
- values in global uint32_defaults and string_defaults storages. However, it is
- possible to instruct them to store parsed values in more specific
- storages. That capability is used, e.g. in \ref isc::dhcp::Subnet6ConfigParser
- that has its own storage that is unique for each subnet. Finally, during commit
- phase (commit() method), appropriate parsers can use apply parameter inheritance.
+ defaults. Some parsers (e.g. Uint32Parser and StringParser) implement that
+ inheritance. By default, they store values in global uint32_defaults and
+ string_defaults storages. However, it is possible to instruct them to store
+ parsed values in more specific storages. That capability is used, e.g. in
+ Subnet6ConfigParser that has its own storage that is unique for each subnet.
+ Finally, during commit phase (commit() method), appropriate parsers can use
+ apply parameter inheritance.
 
  Debugging configuration parser may be confusing. Therefore there is a special
- class called \ref isc::dhcp::DebugParser. It does not configure anything, but just
+ class called DebugParser. It does not configure anything, but just
  accepts any parameter of any type. If requested to commit configuration, it will
  print out received parameter name and its value. This class is not currently used,
  but it is convenient to have it every time a new parameter is added to DHCP

src/bin/dhcp6/dhcp6.spec

@@ -40,6 +40,56 @@
         "item_default": 4000
       },
 
+      { "item_name": "option-def",
+        "item_type": "list",
+        "item_optional": false,
+        "item_default": [],
+        "list_item_spec":
+        {
+          "item_name": "single-option-def",
+          "item_type": "map",
+          "item_optional": false,
+          "item_default": {},
+          "map_item_spec": [
+          {
+            "item_name": "name",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": ""
+          },
+
+          { "item_name": "code",
+            "item_type": "integer",
+            "item_optional": false,
+            "item_default": 0,
+          },
+
+          { "item_name": "type",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": "",
+          },
+
+          { "item_name": "array",
+            "item_type": "boolean",
+            "item_optional": false,
+            "item_default": False
+          },
+
+          { "item_name": "record_types",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": "",
+          },
+
+          { "item_name": "space",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": ""
+          } ]
+        }
+      },
+
       { "item_name": "option-data",
         "item_type": "list",
         "item_optional": false,
@@ -67,6 +117,16 @@
             "item_type": "string",
             "item_optional": false,
             "item_default": ""
+          },
+          { "item_name": "csv-format",
+            "item_type": "boolean",
+            "item_optional": false,
+            "item_default": False
+          },
+          { "item_name": "space",
+            "item_type": "string",
+            "item_optional": false,
+            "item_default": "dhcp6"
           } ]
         }
       },
@@ -89,6 +149,12 @@
                   "item_default": ""
                 },
 
+                { "item_name": "interface",
+                  "item_type": "string",
+                  "item_optional": false,
+                  "item_default": ""
+                },
+
                 { "item_name": "renew-timer",
                   "item_type": "integer",
                   "item_optional": false,
@@ -152,6 +218,16 @@
                       "item_type": "string",
                       "item_optional": false,
                       "item_default": ""
+                    },
+                    { "item_name": "csv-format",
+                      "item_type": "boolean",
+                      "item_optional": false,
+                      "item_default": False
+                    },
+                    { "item_name": "space",
+                      "item_type": "string",
+                      "item_optional": false,
+                      "item_default": "dhcp6"
                     } ]
                   }
                 } ]

src/bin/dhcp6/dhcp6_messages.mes

@@ -61,15 +61,22 @@ A debug message indicating that the IPv6 DHCP server has received an
 updated configuration from the BIND 10 configuration system.
 
 % DHCP6_DB_BACKEND_STARTED lease database started (type: %1, name: %2)
-This informational message is printed every time DHCPv6 is started.
-It indicates what database backend type is being to store lease and
-other information.
+This informational message is printed every time the IPv6 DHCP server
+is started.  It indicates what database backend type is being to store
+lease and other information.
+
+% DHCP6_LEASE_WITHOUT_DUID lease for address %1 does not have a DUID
+This error message indicates a database consistency failure. The lease
+database has an entry indicating that the given address is in use,
+but the lease does not contain any client identification. This is most
+likely due to a software error: please raise a bug report. As a temporary
+workaround, manually remove the lease entry from the database.
 
 % DHCP6_LEASE_ADVERT lease %1 advertised (client duid=%2, iaid=%3)
 This debug message indicates that the server successfully advertised
-a lease. It is up to the client to choose one server out of othe advertised
-and continue allocation with that server. This is a normal behavior and
-indicates successful operation.
+a lease. It is up to the client to choose one server out of the
+advertised servers and continue allocation with that server. This
+is a normal behavior and indicates successful operation.
 
 % DHCP6_LEASE_ADVERT_FAIL failed to advertise a lease for client duid=%1, iaid=%2
 This message indicates that the server failed to advertise (in response to
@@ -79,13 +86,44 @@ such failure. Each specific failure is logged in a separate log entry.
 % DHCP6_LEASE_ALLOC lease %1 has been allocated (client duid=%2, iaid=%3)
 This debug message indicates that the server successfully granted (in
 response to client's REQUEST message) a lease. This is a normal behavior
-and incicates successful operation.
+and indicates successful operation.
 
 % DHCP6_LEASE_ALLOC_FAIL failed to grant a lease for client duid=%1, iaid=%2
 This message indicates that the server failed to grant (in response to
 received REQUEST) a lease for a given client. There may be many reasons for
 such failure. Each specific failure is logged in a separate log entry.
 
+% DHCP6_RELEASE address %1 belonging to client duid=%2, iaid=%3 was released properly.
+This debug message indicates that an address was released properly. It
+is a normal operation during client shutdown.
+
+% DHCP6_RELEASE_FAIL failed to remove lease for address %1 for duid=%2, iaid=%3
+This error message indicates that the software failed to remove a
+lease from the lease database.  It probably due to an error during a
+database operation: resolution will most likely require administrator
+intervention (e.g. check if DHCP process has sufficient privileges to
+update the database). It may also be triggered if a lease was manually
+removed from the database during RELEASE message processing.
+
+% DHCP6_RELEASE_FAIL_WRONG_DUID client (duid=%1) tried to release address %2, but it belongs to client (duid=%3)
+This warning message indicates that client tried to release an address
+that belongs to a different client. This should not happen in normal
+circumstances and may indicate a misconfiguration of the client.  However,
+since the client releasing the address will stop using it anyway, there
+is a good chance that the situation will correct itself.
+
+% DHCP6_RELEASE_FAIL_WRONG_IAID client (duid=%1) tried to release address %2, but it used wrong IAID (expected %3, but got %4)
+This warning message indicates that client tried to release an address
+that does belong to it, but the address was expected to be in a different
+IA (identity association) container. This probably means that the client's
+support for multiple addresses is flawed.
+
+% DHCP6_RELEASE_MISSING_CLIENTID client (address=%1) sent RELEASE message without mandatory client-id
+This warning message indicates that client sent RELEASE message without
+mandatory client-id option. This is most likely caused by a buggy client
+(or a relay that malformed forwarded message). This request will not be
+processed and a response with error status code will be sent back.
+
 % DHCP6_NOT_RUNNING IPv6 DHCP server is not running
 A warning message is issued when an attempt is made to shut down the
 IPv6 DHCP server but it is not running.
@@ -121,7 +159,7 @@ a received OFFER packet as UNKNOWN).
 
 % DHCP6_PACKET_RECEIVE_FAIL error on attempt to receive packet: %1
 The IPv6 DHCP server tried to receive a packet but an error
-occured during this attempt. The reason for the error is included in
+occurred during this attempt. The reason for the error is included in
 the message.
 
 % DHCP6_PACKET_SEND_FAIL failed to send DHCPv6 packet: %1
@@ -156,6 +194,34 @@ A debug message listing the data returned to the client.
 The IPv6 DHCP server has encountered a fatal error and is terminating.
 The reason for the failure is included in the message.
 
+% DHCP6_SERVERID_GENERATED server-id %1 has been generated and will be stored in %2
+This informational messages indicates that the server was not able to read
+its server identifier (DUID) and has generated a new one. This server-id
+will be stored in a file and will be read and used during next restart. It
+is normal behavior when the server is started for the first time. If
+this message is printed every start, please check that the server have
+sufficient permission to write its server-id file and that the file is not
+corrupt.
+
+Changing the server identifier in a production environment is not
+recommended as existing clients will not recognize the server and may go
+through a rebind phase. However, they should be able to recover without
+losing their leases.
+
+% DHCP6_SERVERID_LOADED server-id %1 has been loaded from file %2
+This debug message indicates that the server loaded its server identifier.
+That value is sent in all server responses and clients use it to
+discriminate between servers. This is a part of normal startup or
+reconfiguration procedure.
+
+% DHCP6_SERVERID_WRITE_FAIL server was not able to write its ID to file %1
+This warning message indicates that server was not able to write its
+server identifier (DUID) to a file. This likely indicates lack of write
+permission to a given file or directory. This is not cricital and the
+server will continue to operate, but server will generate different DUID
+during every start and clients will need to go through a rebind phase
+to recover.
+
 % DHCP6_SESSION_FAIL failed to establish BIND 10 session (%1), running stand-alone
 The server has failed to establish communication with the rest of BIND
 10 and is running in stand-alone mode.  (This behavior will change once
@@ -216,3 +282,8 @@ recently and does not recognize its well-behaving clients. This is more
 probable if you see many such messages. Clients will recover from this,
 but they will most likely get a different IP addresses and experience
 a brief service interruption.
+
+% DHCP6_UNKNOWN_RELEASE received RELEASE from unknown client (duid=%1, iaid=%2)
+This warning message is printed when client attempts to release a lease,
+but no such lease is known by the server. See DHCP6_UNKNOWN_RENEW for
+possible reasons for such behavior.

src/bin/dhcp6/dhcp6_srv.cc

@@ -1,4 +1,4 @@
-// Copyright (C) 2011-2012 Internet Systems Consortium, Inc. ("ISC")
+// Copyright (C) 2011-2013 Internet Systems Consortium, Inc. ("ISC")
 //
 // Permission to use, copy, modify, and/or distribute this software for any
 // purpose with or without fee is hereby granted, provided that the above
@@ -32,14 +32,20 @@
 #include <dhcpsrv/lease_mgr.h>
 #include <dhcpsrv/lease_mgr_factory.h>
 #include <dhcpsrv/subnet.h>
+#include <dhcpsrv/utils.h>
 #include <exceptions/exceptions.h>
 #include <util/io_utilities.h>
 #include <util/range_utilities.h>
+#include <util/encode/hex.h>
 
 #include <boost/foreach.hpp>
+#include <boost/tokenizer.hpp>
+#include <boost/algorithm/string/erase.hpp>
 
 #include <stdlib.h>
 #include <time.h>
+#include <iomanip>
+#include <fstream>
 
 using namespace isc;
 using namespace isc::asiolink;
@@ -69,7 +75,22 @@ Dhcpv6Srv::Dhcpv6Srv(uint16_t port, const char* dbconfig)
             IfaceMgr::instance().openSockets6(port);
         }
 
-        setServerID();
+        string duid_file = CfgMgr::instance().getDataDir() + "/" + string(SERVER_DUID_FILE);
+        if (loadServerID(duid_file)) {
+            LOG_DEBUG(dhcp6_logger, DBG_DHCP6_START, DHCP6_SERVERID_LOADED)
+                .arg(duid_file);
+        } else {
+            generateServerID();
+            LOG_INFO(dhcp6_logger, DHCP6_SERVERID_GENERATED)
+                .arg(duidToString(getServerID()))
+                .arg(duid_file);
+
+            if (!writeServerID(duid_file)) {
+                LOG_WARN(dhcp6_logger, DHCP6_SERVERID_WRITE_FAIL)
+                    .arg(duid_file);
+            }
+
+        }
 
         // Instantiate LeaseMgr
         LeaseMgrFactory::create(dbconfig);
@@ -208,10 +229,67 @@ bool Dhcpv6Srv::run() {
     return (true);
 }
 
-void Dhcpv6Srv::setServerID() {
+bool Dhcpv6Srv::loadServerID(const std::string& file_name) {
+
+    // load content of the file into a string
+    fstream f(file_name.c_str(), ios::in);
+    if (!f.is_open()) {
+        return (false);
+    }
+
+    string hex_string;
+    f >> hex_string;
+    f.close();
+
+    // remove any spaces
+    boost::algorithm::erase_all(hex_string, " ");
+
+    // now remove :
+    /// @todo: We should check first if the format is sane.
+    /// Otherwise 1:2:3:4 will be converted to 0x12, 0x34
+    boost::algorithm::erase_all(hex_string, ":");
+
+    std::vector<uint8_t> bin;
+
+    // Decode the hex string and store it in bin (which happens
+    // to be OptionBuffer format)
+    isc::util::encode::decodeHex(hex_string, bin);
+
+    // Now create server-id option
+    serverid_.reset(new Option(Option::V6, D6O_SERVERID, bin));
+
+    return (true);
+}
+
+std::string Dhcpv6Srv::duidToString(const OptionPtr& opt) {
+    stringstream tmp;
 
-    /// @todo: DUID should be generated once and then stored, rather
-    /// than generated each time
+    OptionBuffer data = opt->getData();
+
+    bool colon = false;
+    for (OptionBufferConstIter it = data.begin(); it != data.end(); ++it) {
+        if (colon) {
+            tmp << ":";
+        }
+        tmp << hex << setw(2) << setfill('0') << static_cast<uint16_t>(*it);
+        if (!colon) {
+            colon = true;
+        }
+    }
+
+    return tmp.str();
+}
+
+bool Dhcpv6Srv::writeServerID(const std::string& file_name) {
+    fstream f(file_name.c_str(), ios::out | ios::trunc);
+    if (!f.good()) {
+        return (false);
+    }
+    f << duidToString(getServerID());
+    f.close();
+}
+
+void Dhcpv6Srv::generateServerID() {
 
     /// @todo: This code implements support for DUID-LLT (the recommended one).
     /// We should eventually add support for other DUID types: DUID-LL, DUID-EN
@@ -340,8 +418,8 @@ void Dhcpv6Srv::appendRequestedOptions(const Pkt6Ptr& question, Pkt6Ptr& answer)
     // Get the list of options that client requested.
     const std::vector<uint16_t>& requested_opts = option_oro->getValues();
     // Get the list of options configured for a subnet.
-    const Subnet::OptionContainer& options = subnet->getOptions();
-    const Subnet::OptionContainerTypeIndex& idx = options.get<1>();
+    Subnet::OptionContainerPtr options = subnet->getOptionDescriptors("dhcp6");
+    const Subnet::OptionContainerTypeIndex& idx = options->get<1>();
     // Try to match requested options with those configured for a subnet.
     // If match is found, append configured option to the answer message.
     BOOST_FOREACH(uint16_t opt, requested_opts) {
@@ -404,8 +482,8 @@ void Dhcpv6Srv::sanityCheck(const Pkt6Ptr& pkt, RequirementLevel clientid,
     Option::OptionCollection server_ids = pkt->getOptions(D6O_SERVERID);
     switch (serverid) {
     case FORBIDDEN:
-        if (server_ids.size() > 0) {
-            isc_throw(RFCViolation, "Exactly 1 server-id option expected, but "
+        if (!server_ids.empty()) {
+            isc_throw(RFCViolation, "Server-id option was not expected, but "
                       << server_ids.size() << " received in " << pkt->getName());
         }
         break;
@@ -427,7 +505,17 @@ void Dhcpv6Srv::sanityCheck(const Pkt6Ptr& pkt, RequirementLevel clientid,
 }
 
 Subnet6Ptr Dhcpv6Srv::selectSubnet(const Pkt6Ptr& question) {
-    Subnet6Ptr subnet = CfgMgr::instance().getSubnet6(question->getRemoteAddr());
+
+    /// @todo: pass interface information only if received direct (non-relayed) message
+
+    // Try to find a subnet if received packet from a directly connected client
+    Subnet6Ptr subnet = CfgMgr::instance().getSubnet6(question->getIface());
+    if (subnet) {
+        return (subnet);
+    }
+
+    // If no subnet was found, try to find it based on remote address
+    subnet = CfgMgr::instance().getSubnet6(question->getRemoteAddr());
 
     return (subnet);
 }
@@ -436,6 +524,8 @@ void Dhcpv6Srv::assignLeases(const Pkt6Ptr& question, Pkt6Ptr& answer) {
 
     // We need to allocate addresses for all IA_NA options in the client's
     // question (i.e. SOLICIT or REQUEST) message.
+    // @todo add support for IA_TA
+    // @todo add support for IA_PD
 
     // We need to select a subnet the client is connected in.
     Subnet6Ptr subnet = selectSubnet(question);
@@ -450,7 +540,10 @@ void Dhcpv6Srv::assignLeases(const Pkt6Ptr& question, Pkt6Ptr& answer) {
 
         // perhaps this should be logged on some higher level? This is most likely
         // configuration bug.
-        LOG_ERROR(dhcp6_logger, DHCP6_SUBNET_SELECTION_FAILED);
+        LOG_ERROR(dhcp6_logger, DHCP6_SUBNET_SELECTION_FAILED)
+            .arg(question->getRemoteAddr().toText())
+            .arg(question->getName());
+
     } else {
         LOG_DEBUG(dhcp6_logger, DBG_DHCP6_DETAIL_DATA, DHCP6_SUBNET_SELECTED)
             .arg(subnet->toText());
@@ -604,7 +697,7 @@ OptionPtr Dhcpv6Srv::renewIA_NA(const Subnet6Ptr& subnet, const DuidPtr& duid,
         boost::shared_ptr<Option6IA> ia_rsp(new Option6IA(D6O_IA_NA, ia->getIAID()));
 
         // Insert status code NoAddrsAvail.
-        ia_rsp->addOption(createStatusCode(STATUS_NoAddrsAvail,
+        ia_rsp->addOption(createStatusCode(STATUS_NoBinding,
                           "Sorry, no known leases for this duid/iaid."));
 
         LOG_DEBUG(dhcp6_logger, DBG_DHCP6_DETAIL, DHCP6_UNKNOWN_RENEW)
@@ -640,6 +733,8 @@ void Dhcpv6Srv::renewLeases(const Pkt6Ptr& renew, Pkt6Ptr& reply) {
 
     // We need to renew addresses for all IA_NA options in the client's
     // RENEW message.
+    // @todo add support for IA_TA
+    // @todo add support for IA_PD
 
     // We need to select a subnet the client is connected in.
     Subnet6Ptr subnet = selectSubnet(renew);
@@ -688,11 +783,176 @@ void Dhcpv6Srv::renewLeases(const Pkt6Ptr& renew, Pkt6Ptr& reply) {
             break;
         }
     }
+}
+
+void Dhcpv6Srv::releaseLeases(const Pkt6Ptr& release, Pkt6Ptr& reply) {
+
+    // We need to release addresses for all IA_NA options in the client's
+    // RELEASE message.
+    // @todo Add support for IA_TA
+    // @todo Add support for IA_PD
+    // @todo Consider supporting more than one address in a single IA_NA.
+    // That was envisaged by RFC3315, but it never happened. The only
+    // software that supports that is Dibbler, but its author seriously doubts
+    // if anyone is really using it. Clients that want more than one address
+    // just include more instances of IA_NA options.
+
+    // Let's find client's DUID. Client is supposed to include its client-id
+    // option almost all the time (the only exception is an anonymous inf-request,
+    // but that is mostly a theoretical case). Our allocation engine needs DUID
+    // and will refuse to allocate anything to anonymous clients.
+    OptionPtr opt_duid = release->getOption(D6O_CLIENTID);
+    if (!opt_duid) {
+        // This should not happen. We have checked this before.
+        // see sanityCheck() called from processRelease()
+        LOG_WARN(dhcp6_logger, DHCP6_RELEASE_MISSING_CLIENTID)
+            .arg(release->getRemoteAddr().toText());
+
+        reply->addOption(createStatusCode(STATUS_UnspecFail,
+                         "You did not include mandatory client-id"));
+        return;
+    }
+    DuidPtr duid(new DUID(opt_duid->getData()));
+
+    int general_status = STATUS_Success;
+    for (Option::OptionCollection::iterator opt = release->options_.begin();
+         opt != release->options_.end(); ++opt) {
+        switch (opt->second->getType()) {
+        case D6O_IA_NA: {
+            OptionPtr answer_opt = releaseIA_NA(duid, release, general_status,
+                                   boost::dynamic_pointer_cast<Option6IA>(opt->second));
+            if (answer_opt) {
+                reply->addOption(answer_opt);
+            }
+            break;
+        }
+        // @todo: add support for IA_PD
+        // @todo: add support for IA_TA
+        default:
+            // remaining options are stateless and thus ignored in this context
+            ;
+        }
+    }
+
+    // To be pedantic, we should also include status code in the top-level
+    // scope, not just in each IA_NA. See RFC3315, section 18.2.6.
+    // This behavior will likely go away in RFC3315bis.
+    reply->addOption(createStatusCode(general_status,
+                     "Summary status for all processed IA_NAs"));
+}
+
+OptionPtr Dhcpv6Srv::releaseIA_NA(const DuidPtr& duid, Pkt6Ptr question,
+                                  int& general_status,
+                                  boost::shared_ptr<Option6IA> ia) {
+    // Release can be done in one of two ways:
+    // Approach 1: extract address from client's IA_NA and see if it belongs
+    // to this particular client.
+    // Approach 2: find a subnet for this client, get a lease for
+    // this subnet/duid/iaid and check if its content matches to what the
+    // client is asking us to release.
+    //
+    // This method implements approach 1.
+
+    // That's our response
+    boost::shared_ptr<Option6IA> ia_rsp(new Option6IA(D6O_IA_NA, ia->getIAID()));
+
+    boost::shared_ptr<Option6IAAddr> release_addr = boost::dynamic_pointer_cast<Option6IAAddr>
+        (ia->getOption(D6O_IAADDR));
+    if (!release_addr) {
+        ia_rsp->addOption(createStatusCode(STATUS_NoBinding,
+                                           "You did not include address in your RELEASE"));
+        general_status = STATUS_NoBinding;
+        return (ia_rsp);
+    }
+
+    Lease6Ptr lease = LeaseMgrFactory::instance().getLease6(release_addr->getAddress());
+
+    if (!lease) {
+        // client releasing a lease that we don't know about.
+
+        // Insert status code NoAddrsAvail.
+        ia_rsp->addOption(createStatusCode(STATUS_NoBinding,
+                          "Sorry, no known leases for this duid/iaid, can't release."));
+        general_status = STATUS_NoBinding;
+
+        LOG_INFO(dhcp6_logger, DHCP6_UNKNOWN_RELEASE)
+            .arg(duid->toText())
+            .arg(ia->getIAID());
+
+        return (ia_rsp);
+    }
+
+    if (!lease->duid_) {
+        // Something is gravely wrong here. We do have a lease, but it does not
+        // have mandatory DUID information attached. Someone was messing with our
+        // database.
+
+        LOG_ERROR(dhcp6_logger, DHCP6_LEASE_WITHOUT_DUID)
+            .arg(release_addr->getAddress().toText());
+
+        general_status = STATUS_UnspecFail;
+        ia_rsp->addOption(createStatusCode(STATUS_UnspecFail,
+                          "Database consistency check failed when trying to RELEASE"));
+        return (ia_rsp);
+    }
+
+    if (*duid != *(lease->duid_)) {
+        // Sorry, it's not your address. You can't release it.
+
+        LOG_INFO(dhcp6_logger, DHCP6_RELEASE_FAIL_WRONG_DUID)
+            .arg(duid->toText())
+            .arg(release_addr->getAddress().toText())
+            .arg(lease->duid_->toText());
+
+        general_status = STATUS_NoBinding;
+        ia_rsp->addOption(createStatusCode(STATUS_NoBinding,
+                          "This address does not belong to you, you can't release it"));
+        return (ia_rsp);
+    }
 
+    if (ia->getIAID() != lease->iaid_) {
+        // This address belongs to this client, but to a different IA
+        LOG_WARN(dhcp6_logger, DHCP6_RELEASE_FAIL_WRONG_IAID)
+            .arg(duid->toText())
+            .arg(release_addr->getAddress().toText())
+            .arg(lease->iaid_)
+            .arg(ia->getIAID());
+        ia_rsp->addOption(createStatusCode(STATUS_NoBinding,
+                          "This is your address, but you used wrong IAID"));
+        general_status = STATUS_NoBinding;
+        return (ia_rsp);
+    }
+
+    // It is not necessary to check if the address matches as we used
+    // getLease6(addr) method that is supposed to return a proper lease.
 
+    // Ok, we've passed all checks. Let's release this address.
 
+    if (!LeaseMgrFactory::instance().deleteLease(lease->addr_)) {
+        ia_rsp->addOption(createStatusCode(STATUS_UnspecFail,
+                          "Server failed to release a lease"));
+
+        LOG_ERROR(dhcp6_logger, DHCP6_RELEASE_FAIL)
+            .arg(lease->addr_.toText())
+            .arg(duid->toText())
+            .arg(lease->iaid_);
+        general_status = STATUS_UnspecFail;
+
+        return (ia_rsp);
+    } else {
+        LOG_DEBUG(dhcp6_logger, DBG_DHCP6_DETAIL, DHCP6_RELEASE)
+            .arg(lease->addr_.toText())
+            .arg(duid->toText())
+            .arg(lease->iaid_);
+
+        ia_rsp->addOption(createStatusCode(STATUS_Success,
+                          "Lease released. Thank you, please come again."));
+
+        return (ia_rsp);
+    }
 }
 
+
 Pkt6Ptr Dhcpv6Srv::processSolicit(const Pkt6Ptr& solicit) {
 
     sanityCheck(solicit, MANDATORY, FORBIDDEN);
@@ -751,8 +1011,16 @@ Pkt6Ptr Dhcpv6Srv::processConfirm(const Pkt6Ptr& confirm) {
 }
 
 Pkt6Ptr Dhcpv6Srv::processRelease(const Pkt6Ptr& release) {
-    /// @todo: Implement this
+
+    sanityCheck(release, MANDATORY, MANDATORY);
+
     Pkt6Ptr reply(new Pkt6(DHCPV6_REPLY, release->getTransid()));
+
+    copyDefaultOptions(release, reply);
+    appendDefaultOptions(release, reply);
+
+    releaseLeases(release, reply);
+
     return reply;
 }
 

src/bin/dhcp6/dhcp6_srv.h

@@ -31,21 +31,15 @@
 namespace isc {
 namespace dhcp {
 
-/// An exception that is thrown if a DHCPv6 protocol violation occurs while
-/// processing a message (e.g. a mandatory option is missing)
-class RFCViolation : public isc::Exception {
-public:
-
-/// @brief constructor
+/// @brief file name of a server-id file
 ///
-/// @param file name of the file, where exception occurred
-/// @param line line of the file, where exception occurred
-/// @param what text description of the issue that caused exception
-RFCViolation(const char* file, size_t line, const char* what) :
-    isc::Exception(file, line, what) {}
-};
-
-
+/// Server must store its duid in persistent storage that must not change
+/// between restarts. This is name of the file that is created in dataDir
+/// (see isc::dhcp::CfgMgr::getDataDir()). It is a text file that uses
+/// double digit hex values separated by colons format, e.g.
+/// 01:ff:02:03:06:80:90:ab:cd:ef. Server will create it during first
+/// run and then use it afterwards.
+static const char* SERVER_DUID_FILE = "b10-dhcp6-serverid";
 
 /// @brief DHCPv6 server service.
 ///
@@ -83,7 +77,7 @@ public:
     /// @param dbconfig Lease manager configuration string.  The default
     ///        of the "memfile" manager is used for testing.
     Dhcpv6Srv(uint16_t port = DHCP6_SERVER_PORT,
-            const char* dbconfig = "type=memfile");
+              const char* dbconfig = "type=memfile");
 
     /// @brief Destructor. Used during DHCPv6 service shutdown.
     virtual ~Dhcpv6Srv();
@@ -212,17 +206,39 @@ protected:
 
     /// @brief Renews specific IA_NA option
     ///
-    /// Generates response to IA_NA. This typically includes finding a lease that
-    /// corresponds to the received address. If no such lease is found, an IA_NA
-    /// response is generated with an appropriate status code.
+    /// Generates response to IA_NA in Renew. This typically includes finding a
+    /// lease that corresponds to the received address. If no such lease is
+    /// found, an IA_NA response is generated with an appropriate status code.
     ///
     /// @param subnet subnet the sender belongs to
     /// @param duid client's duid
     /// @param question client's message
     /// @param ia IA_NA option that is being renewed
+    /// @return IA_NA option (server's response)
     OptionPtr renewIA_NA(const Subnet6Ptr& subnet, const DuidPtr& duid,
                          Pkt6Ptr question, boost::shared_ptr<Option6IA> ia);
 
+    /// @brief Releases specific IA_NA option
+    ///
+    /// Generates response to IA_NA in Release message. This covers finding and
+    /// removal of a lease that corresponds to the received address. If no such
+    /// lease is found, an IA_NA response is generated with an appropriate
+    /// status code.
+    ///
+    /// As RFC 3315 requires that a single status code be sent for the whole message,
+    /// this method may update the passed general_status: it is set to SUCCESS when
+    /// message processing begins, but may be updated to some error code if the
+    /// release process fails.
+    ///
+    /// @param duid client's duid
+    /// @param question client's message
+    /// @param general_status a global status (it may be updated in case of errors)
+    /// @param ia IA_NA option that is being renewed
+    /// @return IA_NA option (server's response)
+    OptionPtr releaseIA_NA(const DuidPtr& duid, Pkt6Ptr question,
+                           int& general_status,
+                           boost::shared_ptr<Option6IA> ia);
+
     /// @brief Copies required options from client message to server answer.
     ///
     /// Copies options that must appear in any server response (ADVERTISE, REPLY)
@@ -271,17 +287,52 @@ protected:
     /// @param reply server's response
     void renewLeases(const Pkt6Ptr& renew, Pkt6Ptr& reply);
 
+    /// @brief Attempts to release received addresses
+    ///
+    /// It iterates through received IA_NA options and attempts to release
+    /// received addresses. If no such leases are found, or the lease fails
+    /// proper checks (e.g. belongs to someone else), a proper status
+    /// code is added to reply message. Released addresses are not added
+    /// to REPLY packet, just its IA_NA containers.
+    /// @param release client's message asking to release
+    /// @param reply server's response
+    void releaseLeases(const Pkt6Ptr& release, Pkt6Ptr& reply);
+
     /// @brief Sets server-identifier.
     ///
-    /// This method attempts to set server-identifier DUID. It loads it
-    /// from a file. If file load fails, it generates new DUID using
-    /// interface link-layer addresses (EUI-64) + timestamp (DUID type
-    /// duid-llt, see RFC3315, section 9.2). If there are no suitable
+    /// This method attempts to generate server-identifier DUID. It generates a
+    /// new DUID using interface link-layer addresses (EUI-64) + timestamp (DUID
+    /// type duid-llt, see RFC3315, section 9.2). If there are no suitable
     /// interfaces present, exception it thrown
     ///
     /// @throws isc::Unexpected Failed to read DUID file and no suitable
     ///         interfaces for new DUID generation are detected.
-    void setServerID();
+    void generateServerID();
+
+    /// @brief attempts to load DUID from a file
+    ///
+    /// Tries to load duid from a text file. If the load is successful,
+    /// it creates server-id option and stores it in serverid_ (to be used
+    /// later by getServerID()).
+    ///
+    /// @param file_name name of the DUID file to load
+    /// @return true if load was successful, false otherwise
+    bool loadServerID(const std::string& file_name);
+
+    /// @brief attempts to write DUID to a file
+    /// Tries to write duid content (stored in serverid_) to a text file.
+    ///
+    /// @param file_name name of the DUID file to write
+    /// @return true if write was successful, false otherwise
+    bool writeServerID(const std::string& file_name);
+
+    /// @brief converts DUID to text
+    /// Converts content of DUID option to a text representation, e.g.
+    /// 01:ff:02:03:06:80:90:ab:cd:ef
+    ///
+    /// @param opt option that contains DUID
+    /// @return string representation
+    static std::string duidToString(const OptionPtr& opt);
 
 private:
     /// @brief Allocation Engine.
@@ -291,7 +342,7 @@ private:
     boost::shared_ptr<AllocEngine> alloc_engine_;
 
     /// Server DUID (to be sent in server-identifier option)
-    boost::shared_ptr<isc::dhcp::Option> serverid_;
+    OptionPtr serverid_;
 
     /// Indicates if shutdown is in progress. Setting it to true will
     /// initiate server shutdown procedure.

src/bin/dhcp6/tests/dhcp6_srv_unittest.cc

@@ -29,12 +29,13 @@
 #include <dhcpsrv/cfgmgr.h>
 #include <dhcpsrv/lease_mgr.h>
 #include <dhcpsrv/lease_mgr_factory.h>
+#include <dhcpsrv/utils.h>
 #include <util/buffer.h>
 #include <util/range_utilities.h>
 
 #include <boost/scoped_ptr.hpp>
 #include <gtest/gtest.h>
-
+#include <unistd.h>
 #include <fstream>
 #include <iostream>
 #include <sstream>
@@ -59,21 +60,32 @@ public:
     using Dhcpv6Srv::processSolicit;
     using Dhcpv6Srv::processRequest;
     using Dhcpv6Srv::processRenew;
+    using Dhcpv6Srv::processRelease;
     using Dhcpv6Srv::createStatusCode;
     using Dhcpv6Srv::selectSubnet;
     using Dhcpv6Srv::sanityCheck;
+    using Dhcpv6Srv::loadServerID;
+    using Dhcpv6Srv::writeServerID;
 };
 
+static const char* DUID_FILE = "server-id-test.txt";
+
 class Dhcpv6SrvTest : public ::testing::Test {
 public:
+    /// Name of the server-id file (used in server-id tests)
+
     // these are empty for now, but let's keep them around
     Dhcpv6SrvTest() : rcode_(-1) {
         subnet_ = Subnet6Ptr(new Subnet6(IOAddress("2001:db8:1::"), 48, 1000,
                                          2000, 3000, 4000));
         pool_ = Pool6Ptr(new Pool6(Pool6::TYPE_IA, IOAddress("2001:db8:1:1::"), 64));
-        subnet_->addPool6(pool_);
+        subnet_->addPool(pool_);
 
+        CfgMgr::instance().deleteSubnets6();
         CfgMgr::instance().addSubnet6(subnet_);
+
+        // it's ok if that fails. There should not be such a file anyway
+        unlink(DUID_FILE);
     }
 
     // Generate IA_NA option with specified parameters
@@ -143,11 +155,14 @@ public:
     }
 
     // Checks that server rejected IA_NA, i.e. that it has no addresses and
-    // that expected status code really appears there.
+    // that expected status code really appears there. In some limited cases
+    // (reply to RELEASE) it may be used to verify positive case, where
+    // IA_NA response is expected to not include address.
+    //
     // Status code indicates type of error encountered (in theory it can also
     // indicate success, but servers typically don't send success status
     // as this is the default result and it saves bandwidth)
-    void checkRejectedIA_NA(const boost::shared_ptr<Option6IA>& ia,
+    void checkIA_NAStatusCode(const boost::shared_ptr<Option6IA>& ia,
                             uint16_t expected_status_code) {
         // Make sure there is no address assigned.
         EXPECT_FALSE(ia->getOption(D6O_IAADDR));
@@ -158,6 +173,12 @@ public:
 
         boost::shared_ptr<OptionCustom> status =
             boost::dynamic_pointer_cast<OptionCustom>(ia->getOption(D6O_STATUS_CODE));
+
+        // It is ok to not include status success as this is the default behavior
+        if (expected_status_code == STATUS_Success && !status) {
+            return;
+        }
+
         EXPECT_TRUE(status);
 
         if (status) {
@@ -169,6 +190,26 @@ public:
         }
     }
 
+
+    void checkMsgStatusCode(const Pkt6Ptr& msg, uint16_t expected_status) {
+        boost::shared_ptr<OptionCustom> status =
+            boost::dynamic_pointer_cast<OptionCustom>(msg->getOption(D6O_STATUS_CODE));
+
+        // It is ok to not include status success as this is the default behavior
+        if (expected_status == STATUS_Success && !status) {
+            return;
+        }
+
+        EXPECT_TRUE(status);
+        if (status) {
+            // We don't have dedicated class for status code, so let's just interpret
+            // first 2 bytes as status. Remainder of the status code option content is
+            // just a text explanation what went wrong.
+            EXPECT_EQ(static_cast<uint16_t>(expected_status),
+                      status->readInteger<uint16_t>(0));
+        }
+    }
+
     // Check that generated IAADDR option contains expected address.
     void checkIAAddr(const boost::shared_ptr<Option6IAAddr>& addr,
                      const IOAddress& expected_addr,
@@ -214,6 +255,9 @@ public:
 
     ~Dhcpv6SrvTest() {
         CfgMgr::instance().deleteSubnets6();
+
+        // Let's clean up if there is such a file.
+        unlink(DUID_FILE);
     };
 
     // A subnet used in most tests
@@ -338,25 +382,27 @@ TEST_F(Dhcpv6SrvTest, advertiseOptions) {
         "    \"pool\": [ \"2001:db8:1::/64\" ],"
         "    \"subnet\": \"2001:db8:1::/48\", "
         "    \"option-data\": [ {"
-        "          \"name\": \"OPTION_DNS_SERVERS\","
+        "          \"name\": \"dns-servers\","
+        "          \"space\": \"dhcp6\","
         "          \"code\": 23,"
-        "          \"data\": \"2001 0DB8 1234 FFFF 0000 0000 0000 0001"
-        "2001 0DB8 1234 FFFF 0000 0000 0000 0002\""
+        "          \"data\": \"2001:db8:1234:FFFF::1, 2001:db8:1234:FFFF::2\","
+        "          \"csv-format\": True"
         "        },"
         "        {"
-        "          \"name\": \"OPTION_FOO\","
-        "          \"code\": 1000,"
-        "          \"data\": \"1234\""
+        "          \"name\": \"subscriber-id\","
+        "          \"space\": \"dhcp6\","
+        "          \"code\": 38,"
+        "          \"data\": \"1234\","
+        "          \"csv-format\": False"
         "        } ]"
         " } ],"
         "\"valid-lifetime\": 4000 }";
 
     ElementPtr json = Element::fromJSON(config);
 
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW(srv.reset(new NakedDhcpv6Srv(0)));
+    NakedDhcpv6Srv srv(0);
 
-    EXPECT_NO_THROW(x = configureDhcp6Server(*srv, json));
+    EXPECT_NO_THROW(x = configureDhcp6Server(srv, json));
     ASSERT_TRUE(x);
     comment_ = parseAnswer(rcode_, x);
 
@@ -369,23 +415,23 @@ TEST_F(Dhcpv6SrvTest, advertiseOptions) {
     sol->addOption(clientid);
 
     // Pass it to the server and get an advertise
-    boost::shared_ptr<Pkt6> adv = srv->processSolicit(sol);
+    boost::shared_ptr<Pkt6> adv = srv.processSolicit(sol);
 
     // check if we get response at all
     ASSERT_TRUE(adv);
 
-    // We have not requested option with code 1000 so it should not
+    // We have not requested any options so they should not
     // be included in the response.
-    ASSERT_FALSE(adv->getOption(1000));
+    ASSERT_FALSE(adv->getOption(D6O_SUBSCRIBER_ID));
     ASSERT_FALSE(adv->getOption(D6O_NAME_SERVERS));
 
-    // Let's now request option with code 1000.
-    // We expect that server will include this option in its reply.
+    // Let's now request some options. We expect that the server
+    // will include them in its response.
     boost::shared_ptr<OptionIntArray<uint16_t> >
         option_oro(new OptionIntArray<uint16_t>(Option::V6, D6O_ORO));
     // Create vector with two option codes.
     std::vector<uint16_t> codes(2);
-    codes[0] = 1000;
+    codes[0] = D6O_SUBSCRIBER_ID;
     codes[1] = D6O_NAME_SERVERS;
     // Pass this code to option.
     option_oro->setValues(codes);
@@ -393,7 +439,7 @@ TEST_F(Dhcpv6SrvTest, advertiseOptions) {
     sol->addOption(option_oro);
 
     // Need to process SOLICIT again after requesting new option.
-    adv = srv->processSolicit(sol);
+    adv = srv.processSolicit(sol);
     ASSERT_TRUE(adv);
 
     OptionPtr tmp = adv->getOption(D6O_NAME_SERVERS);
@@ -410,7 +456,7 @@ TEST_F(Dhcpv6SrvTest, advertiseOptions) {
 
     // There is a dummy option with code 1000 we requested from a server.
     // Expect that this option is in server's response.
-    tmp = adv->getOption(1000);
+    tmp = adv->getOption(D6O_SUBSCRIBER_ID);
     ASSERT_TRUE(tmp);
 
     // Check that the option contains valid data (from configuration).
@@ -444,8 +490,7 @@ TEST_F(Dhcpv6SrvTest, advertiseOptions) {
 // - server-id
 // - IA that includes IAADDR
 TEST_F(Dhcpv6SrvTest, SolicitBasic) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     Pkt6Ptr sol = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
     sol->setRemoteAddr(IOAddress("fe80::abcd"));
@@ -454,7 +499,7 @@ TEST_F(Dhcpv6SrvTest, SolicitBasic) {
     sol->addOption(clientid);
 
     // Pass it to the server and get an advertise
-    Pkt6Ptr reply = srv->processSolicit(sol);
+    Pkt6Ptr reply = srv.processSolicit(sol);
 
     // check if we get response at all
     checkResponse(reply, DHCPV6_ADVERTISE, 1234);
@@ -467,7 +512,7 @@ TEST_F(Dhcpv6SrvTest, SolicitBasic) {
     checkIAAddr(addr, addr->getAddress(), subnet_->getPreferred(), subnet_->getValid());
 
     // check DUIDs
-    checkServerId(reply, srv->getServerID());
+    checkServerId(reply, srv.getServerID());
     checkClientId(reply, clientid);
 }
 
@@ -487,8 +532,7 @@ TEST_F(Dhcpv6SrvTest, SolicitBasic) {
 // - server-id
 // - IA that includes IAADDR
 TEST_F(Dhcpv6SrvTest, SolicitHint) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     // Let's create a SOLICIT
     Pkt6Ptr sol = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
@@ -505,7 +549,7 @@ TEST_F(Dhcpv6SrvTest, SolicitHint) {
     sol->addOption(clientid);
 
     // Pass it to the server and get an advertise
-    Pkt6Ptr reply = srv->processSolicit(sol);
+    Pkt6Ptr reply = srv.processSolicit(sol);
 
     // check if we get response at all
     checkResponse(reply, DHCPV6_ADVERTISE, 1234);
@@ -521,7 +565,7 @@ TEST_F(Dhcpv6SrvTest, SolicitHint) {
     checkIAAddr(addr, hint, subnet_->getPreferred(), subnet_->getValid());
 
     // check DUIDs
-    checkServerId(reply, srv->getServerID());
+    checkServerId(reply, srv.getServerID());
     checkClientId(reply, clientid);
 }
 
@@ -541,8 +585,7 @@ TEST_F(Dhcpv6SrvTest, SolicitHint) {
 // - server-id
 // - IA that includes IAADDR
 TEST_F(Dhcpv6SrvTest, SolicitInvalidHint) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     // Let's create a SOLICIT
     Pkt6Ptr sol = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
@@ -557,7 +600,7 @@ TEST_F(Dhcpv6SrvTest, SolicitInvalidHint) {
     sol->addOption(clientid);
 
     // Pass it to the server and get an advertise
-    Pkt6Ptr reply = srv->processSolicit(sol);
+    Pkt6Ptr reply = srv.processSolicit(sol);
 
     // check if we get response at all
     checkResponse(reply, DHCPV6_ADVERTISE, 1234);
@@ -571,7 +614,7 @@ TEST_F(Dhcpv6SrvTest, SolicitInvalidHint) {
     EXPECT_TRUE(subnet_->inPool(addr->getAddress()));
 
     // check DUIDs
-    checkServerId(reply, srv->getServerID());
+    checkServerId(reply, srv.getServerID());
     checkClientId(reply, clientid);
 }
 
@@ -580,14 +623,13 @@ TEST_F(Dhcpv6SrvTest, SolicitInvalidHint) {
 
 // This test checks that the server is offering different addresses to different
 // clients in ADVERTISEs. Please note that ADVERTISE is not a guarantee that such
-// and address will be assigned. Had the pool was very small and contained only
+// an address will be assigned. Had the pool was very small and contained only
 // 2 addresses, the third client would get the same advertise as the first one
 // and this is a correct behavior. It is REQUEST that will fail for the third
 // client. ADVERTISE is basically saying "if you send me a request, you will
 // probably get an address like this" (there are no guarantees).
 TEST_F(Dhcpv6SrvTest, ManySolicits) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     Pkt6Ptr sol1 = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
     Pkt6Ptr sol2 = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 2345));
@@ -611,9 +653,9 @@ TEST_F(Dhcpv6SrvTest, ManySolicits) {
     sol3->addOption(clientid3);
 
     // Pass it to the server and get an advertise
-    Pkt6Ptr reply1 = srv->processSolicit(sol1);
-    Pkt6Ptr reply2 = srv->processSolicit(sol2);
-    Pkt6Ptr reply3 = srv->processSolicit(sol3);
+    Pkt6Ptr reply1 = srv.processSolicit(sol1);
+    Pkt6Ptr reply2 = srv.processSolicit(sol2);
+    Pkt6Ptr reply3 = srv.processSolicit(sol3);
 
     // check if we get response at all
     checkResponse(reply1, DHCPV6_ADVERTISE, 1234);
@@ -634,9 +676,9 @@ TEST_F(Dhcpv6SrvTest, ManySolicits) {
     checkIAAddr(addr3, addr3->getAddress(), subnet_->getPreferred(), subnet_->getValid());
 
     // check DUIDs
-    checkServerId(reply1, srv->getServerID());
-    checkServerId(reply2, srv->getServerID());
-    checkServerId(reply3, srv->getServerID());
+    checkServerId(reply1, srv.getServerID());
+    checkServerId(reply2, srv.getServerID());
+    checkServerId(reply3, srv.getServerID());
     checkClientId(reply1, clientid1);
     checkClientId(reply2, clientid2);
     checkClientId(reply3, clientid3);
@@ -666,8 +708,7 @@ TEST_F(Dhcpv6SrvTest, ManySolicits) {
 // - server-id
 // - IA that includes IAADDR
 TEST_F(Dhcpv6SrvTest, RequestBasic) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     // Let's create a REQUEST
     Pkt6Ptr req = Pkt6Ptr(new Pkt6(DHCPV6_REQUEST, 1234));
@@ -684,10 +725,10 @@ TEST_F(Dhcpv6SrvTest, RequestBasic) {
     req->addOption(clientid);
 
     // server-id is mandatory in REQUEST
-    req->addOption(srv->getServerID());
+    req->addOption(srv.getServerID());
 
     // Pass it to the server and hope for a REPLY
-    Pkt6Ptr reply = srv->processRequest(req);
+    Pkt6Ptr reply = srv.processRequest(req);
 
     // check if we get response at all
     checkResponse(reply, DHCPV6_REPLY, 1234);
@@ -703,7 +744,7 @@ TEST_F(Dhcpv6SrvTest, RequestBasic) {
     checkIAAddr(addr, hint, subnet_->getPreferred(), subnet_->getValid());
 
     // check DUIDs
-    checkServerId(reply, srv->getServerID());
+    checkServerId(reply, srv.getServerID());
     checkClientId(reply, clientid);
 
     // check that the lease is really in the database
@@ -720,8 +761,7 @@ TEST_F(Dhcpv6SrvTest, RequestBasic) {
 // client. ADVERTISE is basically saying "if you send me a request, you will
 // probably get an address like this" (there are no guarantees).
 TEST_F(Dhcpv6SrvTest, ManyRequests) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     Pkt6Ptr req1 = Pkt6Ptr(new Pkt6(DHCPV6_REQUEST, 1234));
     Pkt6Ptr req2 = Pkt6Ptr(new Pkt6(DHCPV6_REQUEST, 2345));
@@ -745,14 +785,14 @@ TEST_F(Dhcpv6SrvTest, ManyRequests) {
     req3->addOption(clientid3);
 
     // server-id is mandatory in REQUEST
-    req1->addOption(srv->getServerID());
-    req2->addOption(srv->getServerID());
-    req3->addOption(srv->getServerID());
+    req1->addOption(srv.getServerID());
+    req2->addOption(srv.getServerID());
+    req3->addOption(srv.getServerID());
 
     // Pass it to the server and get an advertise
-    Pkt6Ptr reply1 = srv->processRequest(req1);
-    Pkt6Ptr reply2 = srv->processRequest(req2);
-    Pkt6Ptr reply3 = srv->processRequest(req3);
+    Pkt6Ptr reply1 = srv.processRequest(req1);
+    Pkt6Ptr reply2 = srv.processRequest(req2);
+    Pkt6Ptr reply3 = srv.processRequest(req3);
 
     // check if we get response at all
     checkResponse(reply1, DHCPV6_REPLY, 1234);
@@ -773,9 +813,9 @@ TEST_F(Dhcpv6SrvTest, ManyRequests) {
     checkIAAddr(addr3, addr3->getAddress(), subnet_->getPreferred(), subnet_->getValid());
 
     // check DUIDs
-    checkServerId(reply1, srv->getServerID());
-    checkServerId(reply2, srv->getServerID());
-    checkServerId(reply3, srv->getServerID());
+    checkServerId(reply1, srv.getServerID());
+    checkServerId(reply2, srv.getServerID());
+    checkServerId(reply3, srv.getServerID());
     checkClientId(reply1, clientid1);
     checkClientId(reply2, clientid2);
     checkClientId(reply3, clientid3);
@@ -799,8 +839,7 @@ TEST_F(Dhcpv6SrvTest, ManyRequests) {
 // - returned REPLY message has IA that includes IAADDR
 // - lease is actually renewed in LeaseMgr
 TEST_F(Dhcpv6SrvTest, RenewBasic) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     const IOAddress addr("2001:db8:1:1::cafe:babe");
     const uint32_t iaid = 234;
@@ -841,10 +880,10 @@ TEST_F(Dhcpv6SrvTest, RenewBasic) {
     req->addOption(clientid);
 
     // Server-id is mandatory in RENEW
-    req->addOption(srv->getServerID());
+    req->addOption(srv.getServerID());
 
     // Pass it to the server and hope for a REPLY
-    Pkt6Ptr reply = srv->processRenew(req);
+    Pkt6Ptr reply = srv.processRenew(req);
 
     // Check if we get response at all
     checkResponse(reply, DHCPV6_REPLY, 1234);
@@ -860,7 +899,7 @@ TEST_F(Dhcpv6SrvTest, RenewBasic) {
     checkIAAddr(addr_opt, addr, subnet_->getPreferred(), subnet_->getValid());
 
     // Check DUIDs
-    checkServerId(reply, srv->getServerID());
+    checkServerId(reply, srv.getServerID());
     checkClientId(reply, clientid);
 
     // Check that the lease is really in the database
@@ -895,9 +934,7 @@ TEST_F(Dhcpv6SrvTest, RenewBasic) {
 // - returned REPLY message has IA that includes STATUS-CODE
 // - No lease in LeaseMgr
 TEST_F(Dhcpv6SrvTest, RenewReject) {
-
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     const IOAddress addr("2001:db8:1:1::dead");
     const uint32_t transid = 1234;
@@ -925,12 +962,12 @@ TEST_F(Dhcpv6SrvTest, RenewReject) {
     req->addOption(clientid);
 
     // Server-id is mandatory in RENEW
-    req->addOption(srv->getServerID());
+    req->addOption(srv.getServerID());
 
     // Case 1: No lease known to server
 
     // Pass it to the server and hope for a REPLY
-    Pkt6Ptr reply = srv->processRenew(req);
+    Pkt6Ptr reply = srv.processRenew(req);
 
     // Check if we get response at all
     checkResponse(reply, DHCPV6_REPLY, transid);
@@ -939,7 +976,7 @@ TEST_F(Dhcpv6SrvTest, RenewReject) {
     // Check that IA_NA was returned and that there's an address included
     ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
     ASSERT_TRUE(ia);
-    checkRejectedIA_NA(ia, STATUS_NoAddrsAvail);
+    checkIA_NAStatusCode(ia, STATUS_NoBinding);
 
     // Check that there is no lease added
     l = LeaseMgrFactory::instance().getLease6(addr);
@@ -956,14 +993,14 @@ TEST_F(Dhcpv6SrvTest, RenewReject) {
     ASSERT_TRUE(LeaseMgrFactory::instance().addLease(lease));
 
     // Pass it to the server and hope for a REPLY
-    reply = srv->processRenew(req);
+    reply = srv.processRenew(req);
     checkResponse(reply, DHCPV6_REPLY, transid);
     tmp = reply->getOption(D6O_IA_NA);
     ASSERT_TRUE(tmp);
     // Check that IA_NA was returned and that there's an address included
     ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
     ASSERT_TRUE(ia);
-    checkRejectedIA_NA(ia, STATUS_NoAddrsAvail);
+    checkIA_NAStatusCode(ia, STATUS_NoBinding);
 
     // There is a iaid mis-match, so server should respond that there is
     // no such address to renew.
@@ -975,14 +1012,14 @@ TEST_F(Dhcpv6SrvTest, RenewReject) {
     req->addOption(generateClientId(13)); // generate different DUID
                                           // (with length 13)
 
-    reply = srv->processRenew(req);
+    reply = srv.processRenew(req);
     checkResponse(reply, DHCPV6_REPLY, transid);
     tmp = reply->getOption(D6O_IA_NA);
     ASSERT_TRUE(tmp);
     // Check that IA_NA was returned and that there's an address included
     ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
     ASSERT_TRUE(ia);
-    checkRejectedIA_NA(ia, STATUS_NoAddrsAvail);
+    checkIA_NAStatusCode(ia, STATUS_NoBinding);
 
     lease = LeaseMgrFactory::instance().getLease6(addr);
     ASSERT_TRUE(lease);
@@ -992,10 +1029,198 @@ TEST_F(Dhcpv6SrvTest, RenewReject) {
     EXPECT_TRUE(LeaseMgrFactory::instance().deleteLease(addr));
 }
 
+// This test verifies that incoming (positive) RELEASE can be handled properly,
+// that a REPLY is generated, that the response has status code and that the
+// lease is indeed removed from the database.
+//
+// expected:
+// - returned REPLY message has copy of client-id
+// - returned REPLY message has server-id
+// - returned REPLY message has IA that does not include an IAADDR
+// - lease is actually removed from LeaseMgr
+TEST_F(Dhcpv6SrvTest, ReleaseBasic) {
+    NakedDhcpv6Srv srv(0);
+
+    const IOAddress addr("2001:db8:1:1::cafe:babe");
+    const uint32_t iaid = 234;
+
+    // Generate client-id also duid_
+    OptionPtr clientid = generateClientId();
+
+    // Check that the address we are about to use is indeed in pool
+    ASSERT_TRUE(subnet_->inPool(addr));
+
+    // Note that preferred, valid, T1 and T2 timers and CLTT are set to invalid
+    // value on purpose. They should be updated during RENEW.
+    Lease6Ptr lease(new Lease6(Lease6::LEASE_IA_NA, addr, duid_, iaid,
+                               501, 502, 503, 504, subnet_->getID(), 0));
+    lease->cltt_ = 1234;
+    ASSERT_TRUE(LeaseMgrFactory::instance().addLease(lease));
+
+    // Check that the lease is really in the database
+    Lease6Ptr l = LeaseMgrFactory::instance().getLease6(addr);
+    ASSERT_TRUE(l);
+
+    // Let's create a RELEASE
+    Pkt6Ptr req = Pkt6Ptr(new Pkt6(DHCPV6_RELEASE, 1234));
+    req->setRemoteAddr(IOAddress("fe80::abcd"));
+    boost::shared_ptr<Option6IA> ia = generateIA(iaid, 1500, 3000);
+
+    OptionPtr released_addr_opt(new Option6IAAddr(D6O_IAADDR, addr, 300, 500));
+    ia->addOption(released_addr_opt);
+    req->addOption(ia);
+    req->addOption(clientid);
+
+    // Server-id is mandatory in RELEASE
+    req->addOption(srv.getServerID());
+
+    // Pass it to the server and hope for a REPLY
+    Pkt6Ptr reply = srv.processRelease(req);
+
+    // Check if we get response at all
+    checkResponse(reply, DHCPV6_REPLY, 1234);
+
+    OptionPtr tmp = reply->getOption(D6O_IA_NA);
+    ASSERT_TRUE(tmp);
+
+    // Check that IA_NA was returned and that there's an address included
+    ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
+    checkIA_NAStatusCode(ia, STATUS_Success);
+    checkMsgStatusCode(reply, STATUS_Success);
+
+    // There should be no address returned in RELEASE (see RFC3315, 18.2.6)
+    EXPECT_FALSE(tmp->getOption(D6O_IAADDR));
+
+    // Check DUIDs
+    checkServerId(reply, srv.getServerID());
+    checkClientId(reply, clientid);
+
+    // Check that the lease is really gone in the database
+    // get lease by address
+    l = LeaseMgrFactory::instance().getLease6(addr);
+    ASSERT_FALSE(l);
+
+    // get lease by subnetid/duid/iaid combination
+    l = LeaseMgrFactory::instance().getLease6(*duid_, iaid, subnet_->getID());
+    ASSERT_FALSE(l);
+}
+
+// This test verifies that incoming (invalid) RELEASE can be handled properly.
+//
+// This test checks 3 scenarios:
+// 1. there is no such lease at all
+// 2. there is such a lease, but it is assigned to a different IAID
+// 3. there is such a lease, but it belongs to a different client
+//
+// expected:
+// - returned REPLY message has copy of client-id
+// - returned REPLY message has server-id
+// - returned REPLY message has IA that includes STATUS-CODE
+// - No lease in LeaseMgr
+TEST_F(Dhcpv6SrvTest, ReleaseReject) {
+
+    NakedDhcpv6Srv srv(0);
+
+    const IOAddress addr("2001:db8:1:1::dead");
+    const uint32_t transid = 1234;
+    const uint32_t valid_iaid = 234;
+    const uint32_t bogus_iaid = 456;
+
+    // Quick sanity check that the address we're about to use is ok
+    ASSERT_TRUE(subnet_->inPool(addr));
+
+    // GenerateClientId() also sets duid_
+    OptionPtr clientid = generateClientId();
+
+    // Check that the lease is NOT in the database
+    Lease6Ptr l = LeaseMgrFactory::instance().getLease6(addr);
+    ASSERT_FALSE(l);
+
+    // Let's create a RELEASE
+    Pkt6Ptr req = Pkt6Ptr(new Pkt6(DHCPV6_RELEASE, transid));
+    req->setRemoteAddr(IOAddress("fe80::abcd"));
+    boost::shared_ptr<Option6IA> ia = generateIA(bogus_iaid, 1500, 3000);
+
+    OptionPtr released_addr_opt(new Option6IAAddr(D6O_IAADDR, addr, 300, 500));
+    ia->addOption(released_addr_opt);
+    req->addOption(ia);
+    req->addOption(clientid);
+
+    // Server-id is mandatory in RENEW
+    req->addOption(srv.getServerID());
+
+    // Case 1: No lease known to server
+    SCOPED_TRACE("CASE 1: No lease known to server");
+
+    // Pass it to the server and hope for a REPLY
+    Pkt6Ptr reply = srv.processRelease(req);
+
+    // Check if we get response at all
+    checkResponse(reply, DHCPV6_REPLY, transid);
+    OptionPtr tmp = reply->getOption(D6O_IA_NA);
+    ASSERT_TRUE(tmp);
+    // Check that IA_NA was returned and that there's an address included
+    ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
+    ASSERT_TRUE(ia);
+    checkIA_NAStatusCode(ia, STATUS_NoBinding);
+    checkMsgStatusCode(reply, STATUS_NoBinding);
+
+    // Check that the lease is not there
+    l = LeaseMgrFactory::instance().getLease6(addr);
+    ASSERT_FALSE(l);
+
+    // CASE 2: Lease is known and belongs to this client, but to a different IAID
+    SCOPED_TRACE("CASE 2: Lease is known and belongs to this client, but to a different IAID");
+
+    Lease6Ptr lease(new Lease6(Lease6::LEASE_IA_NA, addr, duid_, valid_iaid,
+                               501, 502, 503, 504, subnet_->getID(), 0));
+    ASSERT_TRUE(LeaseMgrFactory::instance().addLease(lease));
+
+    // Pass it to the server and hope for a REPLY
+    reply = srv.processRelease(req);
+    checkResponse(reply, DHCPV6_REPLY, transid);
+    tmp = reply->getOption(D6O_IA_NA);
+    ASSERT_TRUE(tmp);
+    // Check that IA_NA was returned and that there's an address included
+    ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
+    ASSERT_TRUE(ia);
+    checkIA_NAStatusCode(ia, STATUS_NoBinding);
+    checkMsgStatusCode(reply, STATUS_NoBinding);
+
+    // Check that the lease is still there
+    l = LeaseMgrFactory::instance().getLease6(addr);
+    ASSERT_TRUE(l);
+
+    // CASE 3: Lease belongs to a client with different client-id
+    SCOPED_TRACE("CASE 3: Lease belongs to a client with different client-id");
+
+    req->delOption(D6O_CLIENTID);
+    ia = boost::dynamic_pointer_cast<Option6IA>(req->getOption(D6O_IA_NA));
+    ia->setIAID(valid_iaid); // Now iaid in renew matches that in leasemgr
+    req->addOption(generateClientId(13)); // generate different DUID
+                                          // (with length 13)
+
+    reply = srv.processRelease(req);
+    checkResponse(reply, DHCPV6_REPLY, transid);
+    tmp = reply->getOption(D6O_IA_NA);
+    ASSERT_TRUE(tmp);
+    // Check that IA_NA was returned and that there's an address included
+    ia = boost::dynamic_pointer_cast<Option6IA>(tmp);
+    ASSERT_TRUE(ia);
+    checkIA_NAStatusCode(ia, STATUS_NoBinding);
+    checkMsgStatusCode(reply, STATUS_NoBinding);
+
+    // Check that the lease is still there
+    l = LeaseMgrFactory::instance().getLease6(addr);
+    ASSERT_TRUE(l);
+
+    // Finally, let's cleanup the database
+    EXPECT_TRUE(LeaseMgrFactory::instance().deleteLease(addr));
+}
+
 // This test verifies if the status code option is generated properly.
 TEST_F(Dhcpv6SrvTest, StatusCode) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     // a dummy content for client-id
     uint8_t expected[] = {
@@ -1005,7 +1230,7 @@ TEST_F(Dhcpv6SrvTest, StatusCode) {
         0x41, 0x42, 0x43, 0x44, 0x45 // string value ABCDE
     };
     // Create the option.
-    OptionPtr status = srv->createStatusCode(3, "ABCDE");
+    OptionPtr status = srv.createStatusCode(3, "ABCDE");
     // Allocate an output buffer. We will store the option
     // in wire format here.
     OutputBuffer buf(sizeof(expected));
@@ -1019,34 +1244,34 @@ TEST_F(Dhcpv6SrvTest, StatusCode) {
 
 // This test verifies if the sanityCheck() really checks options presence.
 TEST_F(Dhcpv6SrvTest, sanityCheck) {
-    boost::scoped_ptr<NakedDhcpv6Srv> srv;
-    ASSERT_NO_THROW( srv.reset(new NakedDhcpv6Srv(0)) );
+    NakedDhcpv6Srv srv(0);
 
     Pkt6Ptr pkt = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
 
-    // check that the packets originating from local addresses can be
+    // Set link-local sender address, so appropriate subnet can be
+    // selected for this packet.
     pkt->setRemoteAddr(IOAddress("fe80::abcd"));
 
     // client-id is optional for information-request, so
-    EXPECT_NO_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::OPTIONAL));
+    EXPECT_NO_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::OPTIONAL));
 
     // empty packet, no client-id, no server-id
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::FORBIDDEN),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::FORBIDDEN),
                  RFCViolation);
 
     // This doesn't make much sense, but let's check it for completeness
-    EXPECT_NO_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::FORBIDDEN, Dhcpv6Srv::FORBIDDEN));
+    EXPECT_NO_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::FORBIDDEN, Dhcpv6Srv::FORBIDDEN));
 
     OptionPtr clientid = generateClientId();
     pkt->addOption(clientid);
 
     // client-id is mandatory, server-id is forbidden (as in SOLICIT or REBIND)
-    EXPECT_NO_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::FORBIDDEN));
+    EXPECT_NO_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::FORBIDDEN));
 
-    pkt->addOption(srv->getServerID());
+    pkt->addOption(srv.getServerID());
 
     // both client-id and server-id are mandatory (as in REQUEST, RENEW, RELEASE, DECLINE)
-    EXPECT_NO_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::MANDATORY));
+    EXPECT_NO_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::MANDATORY));
 
     // sane section ends here, let's do some negative tests as well
 
@@ -1054,13 +1279,13 @@ TEST_F(Dhcpv6SrvTest, sanityCheck) {
     pkt->addOption(clientid);
 
     // with more than one client-id it should throw, no matter what
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::OPTIONAL),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::OPTIONAL),
                  RFCViolation);
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::OPTIONAL),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::OPTIONAL),
                  RFCViolation);
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::MANDATORY),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::MANDATORY),
                  RFCViolation);
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::MANDATORY),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::MANDATORY),
                  RFCViolation);
 
     pkt->delOption(D6O_CLIENTID);
@@ -1069,20 +1294,166 @@ TEST_F(Dhcpv6SrvTest, sanityCheck) {
     // again we have only one client-id
 
     // let's try different type of insanity - several server-ids
-    pkt->addOption(srv->getServerID());
-    pkt->addOption(srv->getServerID());
+    pkt->addOption(srv.getServerID());
+    pkt->addOption(srv.getServerID());
 
     // with more than one server-id it should throw, no matter what
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::OPTIONAL),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::OPTIONAL),
                  RFCViolation);
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::OPTIONAL),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::OPTIONAL),
                  RFCViolation);
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::MANDATORY),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::OPTIONAL, Dhcpv6Srv::MANDATORY),
                  RFCViolation);
-    EXPECT_THROW(srv->sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::MANDATORY),
+    EXPECT_THROW(srv.sanityCheck(pkt, Dhcpv6Srv::MANDATORY, Dhcpv6Srv::MANDATORY),
                  RFCViolation);
+}
+
+// This test verifies if selectSubnet() selects proper subnet for a given
+// source address.
+TEST_F(Dhcpv6SrvTest, selectSubnetAddr) {
+    NakedDhcpv6Srv srv(0);
 
+    Subnet6Ptr subnet1(new Subnet6(IOAddress("2001:db8:1::"), 48, 1, 2, 3, 4));
+    Subnet6Ptr subnet2(new Subnet6(IOAddress("2001:db8:2::"), 48, 1, 2, 3, 4));
+    Subnet6Ptr subnet3(new Subnet6(IOAddress("2001:db8:3::"), 48, 1, 2, 3, 4));
+
+    // CASE 1: We have only one subnet defined and we received local traffic.
+    // The only available subnet should be selected
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1); // just a single subnet
+
+    Pkt6Ptr pkt = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
+    pkt->setRemoteAddr(IOAddress("fe80::abcd"));
+
+    Subnet6Ptr selected = srv.selectSubnet(pkt);
+    EXPECT_EQ(selected, subnet1);
+
+    // CASE 2: We have only one subnet defined and we received relayed traffic.
+    // We should NOT select it.
+
+    // Identical steps as in case 1, but repeated for clarity
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1); // just a single subnet
+    pkt->setRemoteAddr(IOAddress("2001:db8:abcd::2345"));
+    selected = srv.selectSubnet(pkt);
+    EXPECT_FALSE(selected);
+
+    // CASE 3: We have three subnets defined and we received local traffic.
+    // Nothing should be selected.
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1);
+    CfgMgr::instance().addSubnet6(subnet2);
+    CfgMgr::instance().addSubnet6(subnet3);
+    pkt->setRemoteAddr(IOAddress("fe80::abcd"));
+    selected = srv.selectSubnet(pkt);
+    EXPECT_FALSE(selected);
+
+    // CASE 4: We have three subnets defined and we received relayed traffic
+    // that came out of subnet 2. We should select subnet2 then
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1);
+    CfgMgr::instance().addSubnet6(subnet2);
+    CfgMgr::instance().addSubnet6(subnet3);
+    pkt->setRemoteAddr(IOAddress("2001:db8:2::baca"));
+    selected = srv.selectSubnet(pkt);
+    EXPECT_EQ(selected, subnet2);
+
+    // CASE 5: We have three subnets defined and we received relayed traffic
+    // that came out of undefined subnet. We should select nothing
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1);
+    CfgMgr::instance().addSubnet6(subnet2);
+    CfgMgr::instance().addSubnet6(subnet3);
+    pkt->setRemoteAddr(IOAddress("2001:db8:4::baca"));
+    selected = srv.selectSubnet(pkt);
+    EXPECT_FALSE(selected);
 
 }
 
+// This test verifies if selectSubnet() selects proper subnet for a given
+// network interface name.
+TEST_F(Dhcpv6SrvTest, selectSubnetIface) {
+    NakedDhcpv6Srv srv(0);
+
+    Subnet6Ptr subnet1(new Subnet6(IOAddress("2001:db8:1::"), 48, 1, 2, 3, 4));
+    Subnet6Ptr subnet2(new Subnet6(IOAddress("2001:db8:2::"), 48, 1, 2, 3, 4));
+    Subnet6Ptr subnet3(new Subnet6(IOAddress("2001:db8:3::"), 48, 1, 2, 3, 4));
+
+    subnet1->setIface("eth0");
+    subnet3->setIface("wifi1");
+
+    // CASE 1: We have only one subnet defined and it is available via eth0.
+    // Packet came from eth0. The only available subnet should be selected
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1); // just a single subnet
+
+    Pkt6Ptr pkt = Pkt6Ptr(new Pkt6(DHCPV6_SOLICIT, 1234));
+    pkt->setIface("eth0");
+
+    Subnet6Ptr selected = srv.selectSubnet(pkt);
+    EXPECT_EQ(selected, subnet1);
+
+    // CASE 2: We have only one subnet defined and it is available via eth0.
+    // Packet came from eth1. We should not select it
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1); // just a single subnet
+
+    pkt->setIface("eth1");
+
+    selected = srv.selectSubnet(pkt);
+    EXPECT_FALSE(selected);
+
+    // CASE 3: We have only 3 subnets defined, one over eth0, one remote and
+    // one over wifi1.
+    // Packet came from eth1. We should not select it
+    CfgMgr::instance().deleteSubnets6();
+    CfgMgr::instance().addSubnet6(subnet1);
+    CfgMgr::instance().addSubnet6(subnet2);
+    CfgMgr::instance().addSubnet6(subnet3);
+
+    pkt->setIface("eth0");
+    EXPECT_EQ(subnet1, srv.selectSubnet(pkt));
+
+    pkt->setIface("eth3"); // no such interface
+    EXPECT_EQ(Subnet6Ptr(), srv.selectSubnet(pkt)); // nothing selected
+
+    pkt->setIface("wifi1");
+    EXPECT_EQ(subnet3, srv.selectSubnet(pkt));
+}
+
+// This test verifies if the server-id disk operations (read, write) are
+// working properly.
+TEST_F(Dhcpv6SrvTest, ServerID) {
+    NakedDhcpv6Srv srv(0);
+
+    string duid1_text = "01:ff:02:03:06:80:90:ab:cd:ef";
+    uint8_t duid1[] = { 0x01, 0xff, 2, 3, 6, 0x80, 0x90, 0xab, 0xcd, 0xef };
+    OptionBuffer expected_duid1(duid1, duid1 + sizeof(duid1));
+
+    fstream file1(DUID_FILE, ios::out | ios::trunc);
+    file1 << duid1_text;
+    file1.close();
+
+    // Test reading from a file
+    EXPECT_TRUE(srv.loadServerID(DUID_FILE));
+    ASSERT_TRUE(srv.getServerID());
+    ASSERT_EQ(sizeof(duid1) + Option::OPTION6_HDR_LEN, srv.getServerID()->len());
+    ASSERT_TRUE(expected_duid1 == srv.getServerID()->getData());
+
+    // Now test writing to a file
+    EXPECT_EQ(0, unlink(DUID_FILE));
+    EXPECT_NO_THROW(srv.writeServerID(DUID_FILE));
+
+    fstream file2(DUID_FILE, ios::in);
+    ASSERT_TRUE(file2.good());
+    string text;
+    file2 >> text;
+    file2.close();
+
+    EXPECT_EQ(duid1_text, text);
+}
+
+/// @todo: Add more negative tests for processX(), e.g. extend sanityCheck() test
+/// to call processX() methods.
+
 }   // end of anonymous namespace

src/bin/loadzone/.gitignore

@@ -1,4 +1,5 @@
 /b10-loadzone
+/b10-loadzone.py
 /loadzone.py
 /run_loadzone.sh
 /b10-loadzone.8

src/bin/loadzone/b10-loadzone.xml

@@ -67,7 +67,7 @@
 
     <para>
     Some control entries (aka directives) are supported.
-    $ORIGIN is followed by a domain name, and sets the the origin
+    $ORIGIN is followed by a domain name, and sets the origin
     that will be used for relative domain names in subsequent records.
     $INCLUDE is followed by a filename to load.
     The previous origin is restored after the file is included.
@@ -224,21 +224,7 @@
   <refsect1>
     <title>BUGS</title>
     <para>
-      As of the initial implementation, the underlying library that
-      this tool uses does not fully validate the loaded zone; for
-      example, loading will succeed even if it doesn't have the SOA or
-      NS record at its origin name.  Such checks will be implemented
-      in a near future version, but until then, the
-      <command>b10-loadzone</command> performs the existence of the
-      SOA and NS records by itself.  However, <command>b10-loadzone</command>
-      only warns about it, and does not cancel the load itself.
-      If this warning message is produced, it's the user's
-      responsibility to fix the errors and reload it.  When the
-      library is updated with the post load checks, it will be more
-      sophisticated and the such zone won't be successfully loaded.
-    </para>
-    <para>
-      There are some other issues noted in the DESCRIPTION section.
+      There are some issues noted in the DESCRIPTION section.
     </para>
   </refsect1>
 </refentry><!--

src/bin/loadzone/loadzone.py.in

@@ -200,29 +200,6 @@ class LoadZoneRunner:
         logger.info(LOADZONE_SQLITE3_USING_DEFAULT_CONFIG, default_db_file)
         return '{"database_file": "' + default_db_file + '"}'
 
-    def __cancel_create(self):
-        '''sqlite3-only hack: delete the zone just created on load failure.
-
-        This should eventually be done via generic datasrc API, but right now
-        we don't have that interface.  Leaving the zone in this situation
-        is too bad, so we handle it with a workaround.
-
-        '''
-        if self._datasrc_type is not 'sqlite3':
-            return
-
-        import sqlite3          # we need the module only here
-        import json
-
-        # If we are here, the following should basically succeed; since
-        # this is considered a temporary workaround we don't bother to catch
-        # and recover rare failure cases.
-        dbfile = json.loads(self._datasrc_config)['database_file']
-        with sqlite3.connect(dbfile) as conn:
-            cur = conn.cursor()
-            cur.execute("DELETE FROM zones WHERE name = ?",
-                        [self._zone_name.to_text()])
-
     def _report_progress(self, loaded_rrs):
         '''Dump the current progress report to stdout.
 
@@ -274,36 +251,14 @@ class LoadZoneRunner:
                 self.__loaded_rrs >= self._report_interval):
                 sys.stdout.write('\n')
         except Exception as ex:
-            # release any remaining lock held in the client/loader
-            loader, datasrc_client = None, None
+            # release any remaining lock held in the loader
+            loader = None
             if created:
-                self.__cancel_create()
+                datasrc_client.delete_zone(self._zone_name)
                 logger.error(LOADZONE_CANCEL_CREATE_ZONE, self._zone_name,
                              self._zone_class)
             raise LoadFailure(str(ex))
 
-    def _post_load_checks(self):
-        '''Perform minimal validity checks on the loaded zone.
-
-        We do this ourselves because the underlying library currently
-        doesn't do any checks.  Once the library support post-load validation
-        this check should be removed.
-
-        '''
-        datasrc_client = DataSourceClient(self._datasrc_type,
-                                          self._datasrc_config)
-        _, finder = datasrc_client.find_zone(self._zone_name) # should succeed
-        result = finder.find(self._zone_name, RRType.SOA())[0]
-        if result is not finder.SUCCESS:
-            self._post_load_warning('zone has no SOA')
-        result = finder.find(self._zone_name, RRType.NS())[0]
-        if result is not finder.SUCCESS:
-            self._post_load_warning('zone has no NS')
-
-    def _post_load_warning(self, msg):
-        logger.warn(LOADZONE_POSTLOAD_ISSUE, self._zone_name,
-                    self._zone_class, msg)
-
     def _set_signal_handlers(self):
         signal.signal(signal.SIGINT, self._interrupt_handler)
         signal.signal(signal.SIGTERM, self._interrupt_handler)
@@ -321,7 +276,6 @@ class LoadZoneRunner:
             total_elapsed_txt = "%.2f" % (time.time() - self.__start_time)
             logger.info(LOADZONE_DONE, self.__loaded_rrs, self._zone_name,
                         self._zone_class, total_elapsed_txt)
-            self._post_load_checks()
             return 0
         except BadArgument as ex:
             logger.error(LOADZONE_ARGUMENT_ERROR, ex)

src/bin/loadzone/loadzone_messages.mes

@@ -46,14 +46,6 @@ in the zone file.  When this happens, the RRs loaded so far are
 effectively deleted from the zone, and the old version (if exists)
 will still remain valid for operations.
 
-% LOADZONE_POSTLOAD_ISSUE New version of zone %1/%2 has an issue: %3
-b10-loadzone detected a problem after a successful load of zone:
-either or both of SOA and NS records are missing at the zone origin.
-In the current implementation the load will not be canceled for such
-problems.  The operator will need to fix the issues and reload the
-zone; otherwise applications (such as b10-auth) that use this data
-source will not work as expected.
-
 % LOADZONE_SQLITE3_USING_DEFAULT_CONFIG Using default configuration with SQLite3 DB file %1
 The SQLite3 data source is specified as the data source type without a
 data source configuration.  b10-loadzone uses the default

src/bin/loadzone/run_loadzone.sh.in

@@ -18,7 +18,7 @@
 PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
 export PYTHON_EXEC
 
-PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs
 export PYTHONPATH
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries

src/bin/loadzone/tests/correct/include.db

@@ -1,8 +1,6 @@
 $ORIGIN include.   ; initialize origin
 $TTL 300
-; this needs #2500
-;@			IN SOA	ns hostmaster (
-@			IN SOA	ns.include. hostmaster.include. (
+@			IN SOA	ns hostmaster (
 				1        ; serial
 				3600
 				1800

src/bin/loadzone/tests/correct/known.test.out

@@ -80,6 +80,6 @@ ns5.example.com.		90	IN	A	4.4.4.4
 comment.example.com.		60	IN	SOA	ns1.example.com. hostmaster.example.com. 1 43200 900 1814400 7200
 comment.example.com.		60	IN	NS	ns1.example.com.
 comment.example.com.		60	IN	TXT	"Simple text"
-comment.example.com.		60	IN	TXT	"; No comment"
+comment.example.com.		60	IN	TXT	"\; No comment"
 comment.example.com.		60	IN	TXT	"Also no comment here"
-comment.example.com.		60	IN	TXT	"A combination ; see?"
+comment.example.com.		60	IN	TXT	"A combination \; see?"

src/bin/loadzone/tests/correct/mix1.db

@@ -1,7 +1,5 @@
 $ORIGIN mix1.
-; this needs #2500
-;@			IN SOA	ns hostmaster (
-@			IN SOA	ns.mix1. hostmaster.mix1. (
+@			IN SOA	ns hostmaster (
 				1        ; serial
 				3600
 				1800

src/bin/loadzone/tests/correct/mix2.db

@@ -1,7 +1,5 @@
 $ORIGIN mix2.
-; this needs #2500
-;@		1	IN SOA	ns hostmaster (
-@		1	IN SOA	ns.mix2. hostmaster.mix2. (
+@		1	IN SOA	ns hostmaster (
 				1        ; serial
 				3600
 				1800

src/bin/loadzone/tests/correct/ttl1.db

@@ -1,7 +1,5 @@
 $ORIGIN ttl1.
-; this needs #2500
-;@			IN SOA	ns hostmaster (
-@			IN SOA	ns.ttl1. hostmaster.ttl1. (
+@			IN SOA	ns hostmaster (
 				1        ; serial
 				3600
 				1800

src/bin/loadzone/tests/correct/ttl2.db

@@ -1,7 +1,5 @@
 $ORIGIN ttl2.
-; this needs #2500
-;@		1	IN SOA	ns hostmaster (
-@		1	IN SOA	ns.ttl2. hostmaster.ttl2 (
+@		1	IN SOA	ns hostmaster (
 				1        ; serial
 				3600
 				1800

src/bin/loadzone/tests/correct/ttlext.db

@@ -1,7 +1,5 @@
 $ORIGIN ttlext.
-; this needs #2500
-;@			IN SOA	ns hostmaster (
-@			IN SOA	ns.ttlext. hostmaster.ttlext. (
+@			IN SOA	ns hostmaster (
 				1        ; serial
 				3600
 				1800

src/bin/loadzone/tests/loadzone_test.py

@@ -237,7 +237,7 @@ class TestLoadZoneRunner(unittest.TestCase):
         self.__check_zone_soa(None, zone_name=Name('example.com'))
 
     def __common_post_load_setup(self, zone_file):
-        '''Common setup procedure for post load tests.'''
+        '''Common setup procedure for post load tests which should fail.'''
         # replace the LoadZoneRunner's original _post_load_warning() for
         # inspection
         self.__warnings = []
@@ -248,26 +248,18 @@ class TestLoadZoneRunner(unittest.TestCase):
         self.__common_load_setup()
         self.__runner._zone_file = zone_file
         self.__check_zone_soa(ORIG_SOA_TXT)
-        self.__runner._do_load()
-        self.__runner._post_load_checks()
+        # It fails because there's problem with the zone data
+        self.assertRaises(LoadFailure, self.__runner._do_load)
 
     def test_load_post_check_fail_soa(self):
         '''Load succeeds but warns about missing SOA, should cause warn'''
-        self.__common_load_setup()
         self.__common_post_load_setup(LOCAL_TESTDATA_PATH +
                                       '/example-nosoa.org.zone')
-        self.__check_zone_soa(False)
-        self.assertEqual(1, len(self.__warnings))
-        self.assertEqual('zone has no SOA', self.__warnings[0])
 
     def test_load_post_check_fail_ns(self):
         '''Load succeeds but warns about missing NS, should cause warn'''
-        self.__common_load_setup()
         self.__common_post_load_setup(LOCAL_TESTDATA_PATH +
                                       '/example-nons.org.zone')
-        self.__check_zone_soa(NEW_SOA_TXT)
-        self.assertEqual(1, len(self.__warnings))
-        self.assertEqual('zone has no NS', self.__warnings[0])
 
     def __interrupt_progress(self, loaded_rrs):
         '''A helper emulating a signal in the middle of loading.

src/bin/msgq/Makefile.am

@@ -4,11 +4,20 @@ pkglibexecdir = $(libexecdir)/@PACKAGE@
 
 pkglibexec_SCRIPTS = b10-msgq
 
+b10_msgqdir = $(pkgdatadir)
+b10_msgq_DATA = msgq.spec
+
 CLEANFILES = b10-msgq msgq.pyc
+CLEANFILES += $(PYTHON_LOGMSGPKG_DIR)/work/msgq_messages.py
+CLEANFILES += $(PYTHON_LOGMSGPKG_DIR)/work/msgq_messages.pyc
 
 man_MANS = b10-msgq.8
 DISTCLEANFILES = $(man_MANS)
-EXTRA_DIST = $(man_MANS) msgq.xml
+EXTRA_DIST = $(man_MANS) msgq.xml msgq_messages.mes msgq.spec
+
+nodist_pylogmessage_PYTHON = $(PYTHON_LOGMSGPKG_DIR)/work/msgq_messages.py
+pylogmessagedir = $(pyexecdir)/isc/log_messages/
+BUILT_SOURCES = $(PYTHON_LOGMSGPKG_DIR)/work/msgq_messages.py
 
 if GENERATE_DOCS
 
@@ -23,6 +32,11 @@ $(man_MANS):
 
 endif
 
+# Define rule to build logging source files from message file
+$(PYTHON_LOGMSGPKG_DIR)/work/msgq_messages.py : msgq_messages.mes
+	$(top_builddir)/src/lib/log/compiler/message \
+	-d $(PYTHON_LOGMSGPKG_DIR)/work -p $(srcdir)/msgq_messages.mes
+
 # this is done here since configure.ac AC_OUTPUT doesn't expand exec_prefix
 b10-msgq: msgq.py
 	$(SED) "s|@@PYTHONPATH@@|@pyexecdir@|" msgq.py >$@

src/bin/msgq/msgq.py.in

@@ -29,34 +29,74 @@ import errno
 import time
 import select
 import random
+import threading
+import isc.config.ccsession
 from optparse import OptionParser, OptionValueError
 import isc.util.process
+import isc.log
+from isc.log_messages.msgq_messages import *
 
 import isc.cc
 
 isc.util.process.rename()
 
+isc.log.init("b10-msgq", buffer=True)
+# Logger that is used in the actual msgq handling - startup, shutdown and the
+# poller thread.
+logger = isc.log.Logger("msgq")
+# A separate copy for the master/config thread when the poller thread runs.
+# We use a separate instance, since the logger itself doesn't have to be
+# thread safe.
+config_logger = isc.log.Logger("msgq")
+TRACE_START = logger.DBGLVL_START_SHUT
+TRACE_BASIC = logger.DBGLVL_TRACE_BASIC
+TRACE_DETAIL = logger.DBGLVL_TRACE_DETAIL
+
 # This is the version that gets displayed to the user.
 # The VERSION string consists of the module name, the module version
 # number, and the overall BIND 10 version number (set in configure.ac).
 VERSION = "b10-msgq 20110127 (BIND 10 @PACKAGE_VERSION@)"
 
+# If B10_FROM_BUILD is set in the environment, we use data files
+# from a directory relative to that, otherwise we use the ones
+# installed on the system
+if "B10_FROM_BUILD" in os.environ:
+    SPECFILE_PATH = os.environ["B10_FROM_BUILD"] + "/src/bin/msgq"
+else:
+    PREFIX = "@prefix@"
+    DATAROOTDIR = "@datarootdir@"
+    SPECFILE_PATH = "@datadir@/@PACKAGE@".replace("${datarootdir}", DATAROOTDIR).replace("${prefix}", PREFIX)
+SPECFILE_LOCATION = SPECFILE_PATH + "/msgq.spec"
+
 class MsgQReceiveError(Exception): pass
 
 class SubscriptionManager:
-    def __init__(self):
+    def __init__(self, cfgmgr_ready):
+        """
+        Initialize the subscription manager.
+        parameters:
+        * cfgmgr_ready: A callable object run once the config manager
+            subscribes. This is a hackish solution, but we can't read
+            the configuration sooner.
+        """
         self.subscriptions = {}
+        self.__cfgmgr_ready = cfgmgr_ready
+        self.__cfgmgr_ready_called = False
 
     def subscribe(self, group, instance, socket):
         """Add a subscription."""
         target = ( group, instance )
         if target in self.subscriptions:
-            print("[b10-msgq] Appending to existing target")
+            logger.debug(TRACE_BASIC, MSGQ_SUBS_APPEND_TARGET, group, instance)
             if socket not in self.subscriptions[target]:
                 self.subscriptions[target].append(socket)
         else:
-            print("[b10-msgq] Creating new target")
+            logger.debug(TRACE_BASIC, MSGQ_SUBS_NEW_TARGET, group, instance)
             self.subscriptions[target] = [ socket ]
+        if group == "ConfigManager" and not self.__cfgmgr_ready_called:
+            logger.debug(TRACE_BASIC, MSGQ_CFGMGR_SUBSCRIBED)
+            self.__cfgmgr_ready_called = True
+            self.__cfgmgr_ready()
 
     def unsubscribe(self, group, instance, socket):
         """Remove the socket from the one specific subscription."""
@@ -124,10 +164,52 @@ class MsgQ:
         self.sockets = {}
         self.connection_counter = random.random()
         self.hostname = socket.gethostname()
-        self.subs = SubscriptionManager()
+        self.subs = SubscriptionManager(self.cfgmgr_ready)
         self.lnames = {}
         self.sendbuffs = {}
         self.running = False
+        self.__cfgmgr_ready = None
+        self.__cfgmgr_ready_cond = threading.Condition()
+        # A lock used when the message queue does anything more complicated.
+        # It is mostly a safety measure, the threads doing so should be mostly
+        # independent, and the one with config session should be read only,
+        # but with threads, one never knows. We use threads for concurrency,
+        # not for performance, so we use wide lock scopes to be on the safe
+        # side.
+        self.__lock = threading.Lock()
+
+    def cfgmgr_ready(self, ready=True):
+        """Notify that the config manager is either subscribed, or
+           that the msgq is shutting down and it won't connect, but
+           anybody waiting for it should stop anyway.
+
+           The ready parameter signifies if the config manager is subscribed.
+
+           This method can be called multiple times, but second and any
+           following call is simply ignored. This means the "abort" version
+           of the call can be used on any stop unconditionally, even when
+           the config manager already connected.
+        """
+        with self.__cfgmgr_ready_cond:
+            if self.__cfgmgr_ready is not None:
+                # This is a second call to this method. In that case it does
+                # nothing.
+                return
+            self.__cfgmgr_ready = ready
+            self.__cfgmgr_ready_cond.notify_all()
+
+    def wait_cfgmgr(self):
+        """Wait for msgq to subscribe.
+
+           When this returns, the config manager is either subscribed, or
+           msgq gave up waiting for it. Success is signified by the return
+           value.
+        """
+        with self.__cfgmgr_ready_cond:
+            # Wait until it either aborts or subscribes
+            while self.__cfgmgr_ready is None:
+                self.__cfgmgr_ready_cond.wait()
+            return self.__cfgmgr_ready
 
     def setup_poller(self):
         """Set up the poll thing.  Internal function."""
@@ -137,7 +219,7 @@ class MsgQ:
             self.poller = select.poll()
 
     def add_kqueue_socket(self, socket, write_filter=False):
-        """Add a kquque filter for a socket.  By default the read
+        """Add a kqueue filter for a socket.  By default the read
         filter is used; if write_filter is set to True, the write
         filter is used.  We use a boolean value instead of a specific
         filter constant, because kqueue filter values do not seem to
@@ -162,9 +244,7 @@ class MsgQ:
 
     def setup_listener(self):
         """Set up the listener socket.  Internal function."""
-        if self.verbose:
-            sys.stdout.write("[b10-msgq] Setting up socket at %s\n" %
-                             self.socket_file)
+        logger.debug(TRACE_BASIC, MSGQ_LISTENER_SETUP, self.socket_file)
 
         self.listen_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
 
@@ -179,8 +259,7 @@ class MsgQ:
             if os.path.exists(self.socket_file):
                 os.remove(self.socket_file)
             self.listen_socket.close()
-            sys.stderr.write("[b10-msgq] failed to setup listener on %s: %s\n"
-                             % (self.socket_file, str(e)))
+            logger.fatal(MSGQ_LISTENER_FAILED, self.socket_file, e)
             raise e
 
         if self.poller:
@@ -188,6 +267,20 @@ class MsgQ:
         else:
             self.add_kqueue_socket(self.listen_socket)
 
+    def setup_signalsock(self):
+        """Create a socket pair used to signal when we want to finish.
+           Using a socket is easy and thread/signal safe way to signal
+           the termination.
+        """
+        # The __poller_sock will be the end in the poller. When it is
+        # closed, we should shut down.
+        (self.__poller_sock, self.__control_sock) = socket.socketpair()
+
+        if self.poller:
+            self.poller.register(self.__poller_sock, select.POLLIN)
+        else:
+            self.add_kqueue_socket(self.__poller_sock)
+
     def setup(self):
         """Configure listener socket, polling, etc.
            Raises a socket.error if the socket_file cannot be
@@ -195,10 +288,10 @@ class MsgQ:
         """
 
         self.setup_poller()
+        self.setup_signalsock()
         self.setup_listener()
 
-        if self.verbose:
-            sys.stdout.write("[b10-msgq] Listening\n")
+        logger.debug(TRACE_START, MSGQ_LISTENER_STARTED);
 
         self.runnable = True
 
@@ -226,10 +319,9 @@ class MsgQ:
     def process_socket(self, fd):
         """Process a read on a socket."""
         if not fd in self.sockets:
-            sys.stderr.write("[b10-msgq] Got read on Strange Socket fd %d\n" % fd)
+            logger.error(MSGQ_READ_UNKNOWN_FD, fd)
             return
         sock = self.sockets[fd]
-#        sys.stderr.write("[b10-msgq] Got read on fd %d\n" %fd)
         self.process_packet(fd, sock)
 
     def kill_socket(self, fd, sock):
@@ -243,7 +335,7 @@ class MsgQ:
         del self.sockets[fd]
         if fd in self.sendbuffs:
             del self.sendbuffs[fd]
-        sys.stderr.write("[b10-msgq] Closing socket fd %d\n" % fd)
+        logger.debug(TRACE_BASIC, MSGQ_SOCK_CLOSE, fd)
 
     def getbytes(self, fd, sock, length):
         """Get exactly the requested bytes, or raise an exception if
@@ -285,15 +377,15 @@ class MsgQ:
         try:
             routing, data = self.read_packet(fd, sock)
         except MsgQReceiveError as err:
+            logger.error(MSGQ_RECV_ERR, fd, err)
             self.kill_socket(fd, sock)
-            sys.stderr.write("[b10-msgq] Receive error: %s\n" % err)
             return
 
         try:
             routingmsg = isc.cc.message.from_wire(routing)
         except DecodeError as err:
             self.kill_socket(fd, sock)
-            sys.stderr.write("[b10-msgq] Routing decode error: %s\n" % err)
+            logger.error(MSGQ_HDR_DECODE_ERR, fd, err)
             return
 
         self.process_command(fd, sock, routingmsg, data)
@@ -301,9 +393,7 @@ class MsgQ:
     def process_command(self, fd, sock, routing, data):
         """Process a single command.  This will split out into one of the
            other functions."""
-        # TODO: A print statement got removed here (one that prints the
-        # routing envelope). When we have logging with multiple levels,
-        # we might want to re-add that on a high debug verbosity.
+        logger.debug(TRACE_DETAIL, MSGQ_RECV_HDR, routing)
         cmd = routing["type"]
         if cmd == 'send':
             self.process_command_send(sock, routing, data)
@@ -319,7 +409,7 @@ class MsgQ:
         elif cmd == 'stop':
             self.stop()
         else:
-            sys.stderr.write("[b10-msgq] Invalid command: %s\n" % cmd)
+            logger.error(MSGQ_INVALID_CMD, cmd)
 
     def preparemsg(self, env, msg = None):
         if type(env) == dict:
@@ -363,8 +453,8 @@ class MsgQ:
             elif e.errno in [ errno.EPIPE,
                               errno.ECONNRESET,
                               errno.ENOBUFS ]:
-                print("[b10-msgq] " + errno.errorcode[e.errno] +
-                      " on send, dropping message and closing connection")
+                logger.error(MSGQ_SEND_ERR, sock.fileno(),
+                             errno.errorcode[e.errno])
                 self.kill_socket(sock.fileno(), sock)
                 return None
             else:
@@ -491,18 +581,23 @@ class MsgQ:
                 if err.args[0] == errno.EINTR:
                     events = []
                 else:
-                    sys.stderr.write("[b10-msgq] Error with poll(): %s\n" % err)
+                    logger.fatal(MSGQ_POLL_ERR, err)
                     break
-            for (fd, event) in events:
-                if fd == self.listen_socket.fileno():
-                    self.process_accept()
-                else:
-                    if event & select.POLLOUT:
-                        self.__process_write(fd)
-                    elif event & select.POLLIN:
-                        self.process_socket(fd)
+            with self.__lock:
+                for (fd, event) in events:
+                    if fd == self.listen_socket.fileno():
+                        self.process_accept()
+                    elif fd == self.__poller_sock.fileno():
+                        # If it's the signal socket, we should terminate now.
+                        self.running = False
+                        break
                     else:
-                        print("[b10-msgq] Error: Unknown even in run_poller()")
+                        if event & select.POLLOUT:
+                            self.__process_write(fd)
+                        elif event & select.POLLIN:
+                            self.process_socket(fd)
+                        else:
+                            logger.error(MSGQ_POLL_UNKNOWN_EVENT, fd, event)
 
     def run_kqueue(self):
         while self.running:
@@ -512,38 +607,95 @@ class MsgQ:
             if not events:
                 raise RuntimeError('serve: kqueue returned no events')
 
-            for event in events:
-                if event.ident == self.listen_socket.fileno():
-                    self.process_accept()
-                else:
-                    if event.filter == select.KQ_FILTER_WRITE:
-                        self.__process_write(event.ident)
-                    if event.filter == select.KQ_FILTER_READ and \
-                            event.data > 0:
-                        self.process_socket(event.ident)
-                    elif event.flags & select.KQ_EV_EOF:
-                        self.kill_socket(event.ident,
-                                         self.sockets[event.ident])
+            with self.__lock:
+                for event in events:
+                    if event.ident == self.listen_socket.fileno():
+                        self.process_accept()
+                    elif event.ident == self.__poller_sock.fileno():
+                        # If it's the signal socket, we should terminate now.
+                        self.running = False
+                        break;
+                    else:
+                        if event.filter == select.KQ_FILTER_WRITE:
+                            self.__process_write(event.ident)
+                        if event.filter == select.KQ_FILTER_READ and \
+                                event.data > 0:
+                            self.process_socket(event.ident)
+                        elif event.flags & select.KQ_EV_EOF:
+                            self.kill_socket(event.ident,
+                                             self.sockets[event.ident])
 
     def stop(self):
-        self.running = False
+        # Signal it should terminate.
+        self.__control_sock.close()
+        self.__control_sock = None
+        # Abort anything waiting on the condition, just to make sure it's not
+        # blocked forever
+        self.cfgmgr_ready(False)
+
+    def cleanup_signalsock(self):
+        """Close the signal sockets. We could do it directly in shutdown,
+           but this part is reused in tests.
+        """
+        if self.__poller_sock:
+            self.__poller_sock.close()
+            self.__poller_sock = None
+        if self.__control_sock:
+            self.__control_sock.close()
+            self.__control_sock = None
 
     def shutdown(self):
         """Stop the MsgQ master."""
-        if self.verbose:
-            sys.stdout.write("[b10-msgq] Stopping the server.\n")
+        logger.debug(TRACE_START, MSGQ_SHUTDOWN)
         self.listen_socket.close()
+        self.cleanup_signalsock()
+        # Close all the sockets too. In real life, there should be none now,
+        # as Msgq should be the last one. But some tests don't adhere to this
+        # and create a new Msgq for each test, which led to huge socket leaks.
+        # Some other threads put some other things in instead of sockets, so
+        # we catch whatever exceptions there we can. This should be safe,
+        # because in real operation, we will terminate now anyway, implicitly
+        # closing anything anyway.
+        for sock in self.sockets.values():
+            try:
+                sock.close()
+            except Exception:
+                pass
         if os.path.exists(self.socket_file):
             os.remove(self.socket_file)
 
-# can signal handling and calling a destructor be done without a
-# global variable?
-msgq = None
+    def config_handler(self, new_config):
+        """The configuration handler (run in a separate thread).
+           Not tested, currently effectively empty.
+        """
+        config_logger.debug(TRACE_DETAIL, MSGQ_CONFIG_DATA, new_config)
+
+        with self.__lock:
+            if not self.running:
+                return
+
+            # TODO: Any config handlig goes here.
+
+            return isc.config.create_answer(0)
 
-def signal_handler(signal, frame):
+    def command_handler(self, command, args):
+        """The command handler (run in a separate thread).
+           Not tested, currently effectively empty.
+        """
+        config_logger.debug(TRACE_DETAIL, MSGQ_COMMAND, command, args)
+
+        with self.__lock:
+            if not self.running:
+                return
+
+            # TODO: Any commands go here
+
+            config_logger.error(MSGQ_COMMAND_UNKNOWN, command)
+            return isc.config.create_answer(1, 'unknown command: ' + command)
+
+def signal_handler(msgq, signal, frame):
     if msgq:
-        msgq.shutdown()
-    sys.exit(0)
+        msgq.stop()
 
 if __name__ == "__main__":
     def check_port(option, opt_str, value, parser):
@@ -556,6 +708,7 @@ if __name__ == "__main__":
 
     # Parse any command-line options.
     parser = OptionParser(version=VERSION)
+    # TODO: Should we remove the option?
     parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
                       help="display more about what is going on")
     parser.add_option("-s", "--socket-file", dest="msgq_socket_file",
@@ -563,22 +716,46 @@ if __name__ == "__main__":
                       help="UNIX domain socket file the msgq daemon will use")
     (options, args) = parser.parse_args()
 
-    signal.signal(signal.SIGTERM, signal_handler)
-
     # Announce startup.
-    if options.verbose:
-        sys.stdout.write("[b10-msgq] %s\n" % VERSION)
+    logger.debug(TRACE_START, MSGQ_START, VERSION)
 
     msgq = MsgQ(options.msgq_socket_file, options.verbose)
 
+    signal.signal(signal.SIGTERM,
+                  lambda signal, frame: signal_handler(msgq, signal, frame))
+
     try:
         msgq.setup()
     except Exception as e:
-        sys.stderr.write("[b10-msgq] Error on startup: %s\n" % str(e))
+        logger.fatal(MSGQ_START_FAIL, e)
         sys.exit(1)
 
+    # We run the processing in a separate thread. This is because we want to
+    # connect to the msgq ourself. But the cc library is unfortunately blocking
+    # in many places and waiting for the processing part to answer, it would
+    # deadlock.
+    poller_thread = threading.Thread(target=msgq.run)
+    poller_thread.daemon = True
     try:
-        msgq.run()
+        poller_thread.start()
+        if msgq.wait_cfgmgr():
+            # Once we get the config manager, we can read our own config.
+            session = isc.config.ModuleCCSession(SPECFILE_LOCATION,
+                                                 msgq.config_handler,
+                                                 msgq.command_handler,
+                                                 None, True,
+                                                 msgq.socket_file)
+            session.start()
+            # And we create a thread that'll just wait for commands and
+            # handle them. We don't terminate the thread, we set it to
+            # daemon. Once the main thread terminates, it'll just die.
+            def run_session():
+                while True:
+                    session.check_command(False)
+            background_thread = threading.Thread(target=run_session)
+            background_thread.daemon = True
+            background_thread.start()
+        poller_thread.join()
     except KeyboardInterrupt:
         pass
 

src/bin/msgq/msgq.spec

@@ -0,0 +1,8 @@
+{
+  "module_spec": {
+    "module_name": "Msgq",
+    "module_description": "The message queue",
+    "config_data": [],
+    "commands": []
+  }
+}

src/bin/msgq/msgq_messages.mes

@@ -0,0 +1,106 @@
+# Copyright (C) 2012  Internet Systems Consortium, Inc. ("ISC")
+#
+# Permission to use, copy, modify, and/or distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+# AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+# PERFORMANCE OF THIS SOFTWARE.
+
+# No namespace declaration - these constants go in the global namespace
+# of the ddns messages python module.
+
+# When you add a message to this file, it is a good idea to run
+# <topsrcdir>/tools/reorder_message_file.py to make sure the
+# messages are in the correct order.
+
+% MSGQ_CFGMGR_SUBSCRIBED The config manager subscribed to message queue
+This is a debug message. The message queue has little bit of special handling
+for the configuration manager. This special handling is happening now.
+
+% MSGQ_COMMAND Running command %1 with arguments %2
+Debug message. The message queue received a command and it is running it.
+
+% MSGQ_COMMAND_UNKNOWN Unknown command '%1'
+The message queue received a command from other module, but it doesn't
+recognize it. This is probably either a coding error or inconsistency between
+the message queue version and version of the module.
+
+% MSGQ_CONFIG_DATA Received configuration update for the msgq: %1
+Debug message. The message queue received a configuration update, handling it.
+
+% MSGQ_HDR_DECODE_ERR Error decoding header received from socket %1: %2
+The socket with mentioned file descriptor sent a packet. However, it was not
+possible to decode the routing header of the packet. The packet is ignored.
+This may be caused by a programmer error (one of the components sending invalid
+data) or possibly by incompatible version of msgq and the component (but that's
+unlikely, as the protocol is not changed often).
+
+% MSGQ_LISTENER_FAILED Failed to initialize listener on socket file '%1': %2
+The message queue daemon tried to listen on a file socket (the path is in the
+message), but it failed. The error from the operating system is logged.
+
+% MSGQ_LISTENER_SETUP Starting to listen on socket file '%1'
+Debug message. The listener is trying to open a listening socket.
+
+% MSGQ_LISTENER_STARTED Successfully started to listen
+Debug message. The message queue successfully opened a listening socket and
+waits for incoming connections.
+
+% MSGQ_POLL_ERR Error while polling for events: %1
+A low-level error happened when waiting for events, the error is logged. The
+reason for this varies, but it usually means the system is short on some
+resources.
+
+% MSGQ_POLL_UNKNOWN_EVENT Got an unknown event from the poller for fd %1: %2
+An unknown event got out from the poll() system call. This should generally not
+happen and it is either a programmer error or OS bug. The event is ignored. The
+number noted as the event is the raw encoded value, which might be useful to
+the authors when figuring the problem out.
+
+% MSGQ_READ_UNKNOWN_FD Got read on strange socket %1
+The OS reported a file descriptor is ready to read. But the daemon doesn't know
+the mentioned file descriptor, which is either a programmer error or OS bug.
+The read event is ignored.
+
+% MSGQ_RECV_ERR Error reading from socket %1: %2
+There was a low-level error when reading from a socket. The error is logged and
+the corresponding socket is dropped.
+
+% MSGQ_RECV_HDR Received header: %1
+Debug message. This message includes the whole routing header of a packet.
+
+% MSGQ_INVALID_CMD Received invalid command: %1
+An unknown command listed in the log has been received. It is ignored. This
+indicates either a programmer error (eg. a typo in the command name) or
+incompatible version of a module and message queue daemon.
+
+% MSGQ_SEND_ERR Error while sending to socket %1: %2
+There was a low-level error when sending data to a socket. The error is logged
+and the corresponding socket is dropped.
+
+% MSGQ_SHUTDOWN Stopping Msgq
+Debug message. The message queue is shutting down.
+
+% MSGQ_SOCK_CLOSE Closing socket fd %1
+Debug message. Closing the mentioned socket.
+
+% MSGQ_START Msgq version %1 starting
+Debug message. The message queue is starting up.
+
+% MSGQ_START_FAIL Error during startup: %1
+There was an error during early startup of the daemon. More concrete error is
+in the log. The daemon terminates as a result.
+
+% MSGQ_SUBS_APPEND_TARGET Appending to existing target for subscription to group '%1' for instance '%2'
+Debug message. Creating a new subscription by appending it to already existing
+data structure.
+
+% MSGQ_SUBS_NEW_TARGET Creating new target for subscription to group '%1' for instance '%2'
+Debug message. Creating a new subscription. Also creating a new data structure
+to hold it.

src/bin/msgq/run_msgq.sh.in

@@ -20,9 +20,25 @@ export PYTHON_EXEC
 
 MYPATH_PATH=@abs_top_builddir@/src/bin/msgq
 
-PYTHONPATH=@abs_top_srcdir@/src/lib/python
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/log/.libs
 export PYTHONPATH
 
+# If necessary (rare cases), explicitly specify paths to dynamic libraries
+# required by loadable python modules.
+SET_ENV_LIBRARY_PATH=@SET_ENV_LIBRARY_PATH@
+if test $SET_ENV_LIBRARY_PATH = yes; then
+    @ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/acl/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:@abs_top_builddir@/src/lib/datasrc/.libs:$@ENV_LIBRARY_PATH@
+    export @ENV_LIBRARY_PATH@
+fi
+
+B10_FROM_SOURCE=@abs_top_srcdir@
+export B10_FROM_SOURCE
+# TODO: We need to do this feature based (ie. no general from_source)
+# But right now we need a second one because some spec files are
+# generated and hence end up under builddir
+B10_FROM_BUILD=@abs_top_builddir@
+export B10_FROM_BUILD
+
 BIND10_MSGQ_SOCKET_FILE=@abs_top_builddir@/msgq_socket
 export BIND10_MSGQ_SOCKET_FILE
 

src/bin/msgq/tests/Makefile.am

@@ -21,6 +21,8 @@ endif
 	$(LIBRARY_PATH_PLACEHOLDER) \
 	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/msgq \
 	BIND10_TEST_SOCKET_FILE=$(builddir)/test_msgq_socket.sock \
+	B10_FROM_SOURCE=$(abs_top_srcdir) \
+	B10_FROM_BUILD=$(abs_top_builddir) \
 	$(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \
 	done
 

src/bin/msgq/tests/msgq_test.in

@@ -1,28 +0,0 @@
-#! /bin/sh
-
-# Copyright (C) 2010  Internet Systems Consortium.
-#
-# Permission to use, copy, modify, and distribute this software for any
-# purpose with or without fee is hereby granted, provided that the above
-# copyright notice and this permission notice appear in all copies.
-#
-# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
-# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
-# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
-# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
-# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
-# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
-# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-
-PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
-export PYTHON_EXEC
-
-MYPATH_PATH=@abs_top_srcdir@/src/bin/msgq/tests
-
-PYTHONPATH=@abs_top_srcdir@/src/bin/msgq:@abs_top_srcdir@/src/lib/python
-
-export PYTHONPATH
-
-cd ${MYPATH_PATH}
-exec ${PYTHON_EXEC} -O msgq_test.py $*

src/bin/msgq/tests/msgq_test.py

@@ -10,6 +10,7 @@ import errno
 import threading
 import isc.cc
 import collections
+import isc.log
 
 #
 # Currently only the subscription part and some sending is implemented...
@@ -18,7 +19,12 @@ import collections
 
 class TestSubscriptionManager(unittest.TestCase):
     def setUp(self):
-        self.sm = SubscriptionManager()
+        self.__cfgmgr_ready_called = 0
+        self.sm = SubscriptionManager(self.cfgmgr_ready)
+
+    def cfgmgr_ready(self):
+        # Called one more time
+        self.__cfgmgr_ready_called += 1
 
     def test_subscription_add_delete_manager(self):
         self.sm.subscribe("a", "*", 'sock1')
@@ -100,7 +106,7 @@ class TestSubscriptionManager(unittest.TestCase):
         try:
             msgq.setup()
             self.assertTrue(os.path.exists(socket_file))
-            msgq.shutdown();
+            msgq.shutdown()
             self.assertFalse(os.path.exists(socket_file))
         except socket.error:
             # ok, the install path doesn't exist at all,
@@ -114,6 +120,25 @@ class TestSubscriptionManager(unittest.TestCase):
     def test_open_socket_bad(self):
         msgq = MsgQ("/does/not/exist")
         self.assertRaises(socket.error, msgq.setup)
+        # But we can clean up after that.
+        msgq.shutdown()
+
+    def test_subscribe_cfgmgr(self):
+        """Test special handling of the config manager. Once it subscribes,
+           the message queue needs to connect and read the config. But not
+           before and only once.
+        """
+        self.assertEqual(0, self.__cfgmgr_ready_called)
+        # Not called when something else subscribes
+        self.sm.subscribe('SomethingElse', '*', 's1')
+        self.assertEqual(0, self.__cfgmgr_ready_called)
+        # Called whenever the config manager subscribes
+        self.sm.subscribe('ConfigManager', '*', 's2')
+        self.assertEqual(1, self.__cfgmgr_ready_called)
+        # But not called again when it subscribes again (should not
+        # happen in practice, but we make sure anyway)
+        self.sm.subscribe('ConfigManager', '*', 's3')
+        self.assertEqual(1, self.__cfgmgr_ready_called)
 
 class DummySocket:
     """
@@ -193,7 +218,6 @@ class MsgQThread(threading.Thread):
     def stop(self):
         self.msgq_.stop()
 
-
 class SendNonblock(unittest.TestCase):
     """
     Tests that the whole thing will not get blocked if someone does not read.
@@ -281,8 +305,10 @@ class SendNonblock(unittest.TestCase):
             if queue_pid == 0:
                 signal.alarm(120)
                 msgq.setup_poller()
+                msgq.setup_signalsock()
                 msgq.register_socket(queue)
                 msgq.run()
+                msgq.cleanup_signalsock()
             else:
                 try:
                     def killall(signum, frame):
@@ -356,6 +382,7 @@ class SendNonblock(unittest.TestCase):
         # Don't need a listen_socket
         msgq.listen_socket = DummySocket
         msgq.setup_poller()
+        msgq.setup_signalsock()
         msgq.register_socket(write)
         msgq.register_socket(control_write)
         # Queue the message for sending
@@ -383,6 +410,10 @@ class SendNonblock(unittest.TestCase):
         # Fail the test if it didn't stop
         self.assertFalse(msgq_thread.isAlive(), "Thread did not stop")
 
+        # Clean up some internals of msgq (usually called as part of
+        # shutdown, but we skip that one here)
+        msgq.cleanup_signalsock()
+
         # Check the exception from the thread, if any
         # First, if we didn't expect it; reraise it (to make test fail and
         # show the stacktrace for debugging)
@@ -455,6 +486,81 @@ class SendNonblock(unittest.TestCase):
         self.do_send_with_send_error(3, sockerr, False, sockerr)
         self.do_send_with_send_error(23, sockerr, False, sockerr)
 
+class ThreadTests(unittest.TestCase):
+    """Test various things around thread synchronization."""
+
+    def setUp(self):
+        self.__msgq = MsgQ()
+        self.__abort_wait = False
+        self.__result = None
+        self.__notify_thread = threading.Thread(target=self.__notify)
+        self.__wait_thread = threading.Thread(target=self.__wait)
+        # Make sure the threads are killed if left behind by the test.
+        self.__notify_thread.daemon = True
+        self.__wait_thread.daemon = True
+
+    def __notify(self):
+        """Call the cfgmgr_ready."""
+        if self.__abort_wait:
+            self.__msgq.cfgmgr_ready(False)
+        else:
+            self.__msgq.cfgmgr_ready()
+
+    def __wait(self):
+        """Wait for config manager and store the result."""
+        self.__result = self.__msgq.wait_cfgmgr()
+
+    def test_wait_cfgmgr(self):
+        """One thread signals the config manager subscribed, the other
+           waits for it. We then check it terminated correctly.
+        """
+        self.__notify_thread.start()
+        self.__wait_thread.start()
+        # Timeout to ensure the test terminates even on failure
+        self.__wait_thread.join(60)
+        self.assertTrue(self.__result)
+
+    def test_wait_cfgmgr_2(self):
+        """Same as test_wait_cfgmgr, but starting the threads in reverse order
+           (the result should be the same).
+        """
+        self.__wait_thread.start()
+        self.__notify_thread.start()
+        # Timeout to ensure the test terminates even on failure
+        self.__wait_thread.join(60)
+        self.assertTrue(self.__result)
+
+    def test_wait_abort(self):
+        """Similar to test_wait_cfgmgr, but the config manager is never
+           subscribed and it is aborted.
+        """
+        self.__abort_wait = True
+        self.__wait_thread.start()
+        self.__notify_thread.start()
+        # Timeout to ensure the test terminates even on failure
+        self.__wait_thread.join(60)
+        self.assertIsNotNone(self.__result)
+        self.assertFalse(self.__result)
+
+    def __check_ready_and_abort(self):
+        """Check that when we first say the config manager is ready and then
+           try to abort, it uses the first result.
+        """
+        self.__msgq.cfgmgr_ready()
+        self.__msgq.cfgmgr_ready(False)
+        self.__result = self.__msgq.wait_cfgmgr()
+
+    def test_ready_and_abort(self):
+        """Perform the __check_ready_and_abort test, but in a separate thread,
+           so in case something goes wrong with the synchronisation and it
+           deadlocks, the test will terminate anyway.
+        """
+        test_thread = threading.Thread(target=self.__check_ready_and_abort)
+        test_thread.daemon = True
+        test_thread.start()
+        test_thread.join(60)
+        self.assertTrue(self.__result)
 
 if __name__ == '__main__':
+    isc.log.resetUnitTestRootLogger()
     unittest.main()

src/bin/resolver/resolver.h

@@ -200,14 +200,14 @@ public:
     /**
      * \short Get info about timeouts.
      *
-     * \returns Timeout and retries (as described in setTimeouts).
+     * \return Timeout and retries (as described in setTimeouts).
      */
     std::pair<int, unsigned> getTimeouts() const;
 
     /**
      * \brief Get the timeout for outgoing queries
      *
-     * \returns Timeout for outgoing queries
+     * \return Timeout for outgoing queries
      */
     int getQueryTimeout() const;
 
@@ -218,7 +218,7 @@ public:
      * (internal resolving on the query will continue, see
      * \c getLookupTimeout())
      * 
-     * \returns Timeout for outgoing queries
+     * \return Timeout for outgoing queries
      */
     int getClientTimeout() const;
 

src/bin/stats/tests/b10-stats-httpd_test.py

@@ -42,6 +42,7 @@ except ImportError:
     lxml_etree = None
 
 import isc
+import isc.log
 import stats_httpd
 import stats
 from test_utils import BaseModules, ThreadingServerManager, MyStats,\
@@ -1066,4 +1067,5 @@ class TestStatsHttpd(unittest.TestCase):
             imp.reload(stats_httpd)
 
 if __name__ == "__main__":
+    isc.log.resetUnitTestRootLogger()
     unittest.main()

src/bin/stats/tests/b10-stats_test.py

@@ -29,6 +29,7 @@ import time
 import imp
 
 import stats
+import isc.log
 import isc.cc.session
 from test_utils import BaseModules, ThreadingServerManager, MyStats, SignalHandler, send_command, send_shutdown
 from isc.testutils.ccsession_mock import MockModuleCCSession
@@ -1254,8 +1255,6 @@ class TestOSEnv(unittest.TestCase):
         os.environ["B10_FROM_SOURCE"] = path
         imp.reload(stats)
 
-def test_main():
-    unittest.main()
-
 if __name__ == "__main__":
-    test_main()
+    isc.log.resetUnitTestRootLogger()
+    unittest.main()

src/bin/stats/tests/test_utils.py

@@ -103,20 +103,9 @@ class ThreadingServerManager:
         else:
             self.server._thread.join(0) # timeout is 0
 
-def do_nothing(*args, **kwargs): pass
-
-class dummy_sys:
-    """Dummy for sys"""
-    class dummy_io:
-        write = do_nothing
-    stdout = stderr = dummy_io()
-
 class MockMsgq:
     def __init__(self):
         self._started = threading.Event()
-        # suppress output to stdout and stderr
-        msgq.sys = dummy_sys()
-        msgq.print = do_nothing
         self.msgq = msgq.MsgQ(verbose=False)
         result = self.msgq.setup()
         if result:
@@ -124,10 +113,15 @@ class MockMsgq:
 
     def run(self):
         self._started.set()
-        self.msgq.run()
+        try:
+            self.msgq.run()
+        finally:
+            # Make sure all the sockets, etc, are removed once it stops.
+            self.msgq.shutdown()
 
     def shutdown(self):
-        self.msgq.shutdown()
+        # Ask it to terminate nicely
+        self.msgq.stop()
 
 class MockCfgmgr:
     def __init__(self):
@@ -554,15 +548,20 @@ class BaseModules:
 
 
     def shutdown(self):
+        # MockMsgq. We need to wait (blocking) for it, otherwise it'll wipe out
+        # a socket for another test during its shutdown.
+        self.msgq.shutdown(True)
+
+        # We also wait for the others, but these are just so we don't create
+        # too many threads in parallel.
+
         # MockAuth
-        self.auth2.shutdown()
-        self.auth.shutdown()
+        self.auth2.shutdown(True)
+        self.auth.shutdown(True)
         # MockBoss
-        self.boss.shutdown()
+        self.boss.shutdown(True)
         # MockCfgmgr
-        self.cfgmgr.shutdown()
-        # MockMsgq
-        self.msgq.shutdown()
+        self.cfgmgr.shutdown(True)
         # remove the unused socket file
         socket_file = self.msgq.server.msgq.socket_file
         try:

src/bin/sysinfo/run_sysinfo.sh.in

@@ -20,10 +20,11 @@ export PYTHON_EXEC
 
 SYSINFO_PATH=@abs_top_builddir@/src/bin/sysinfo
 
-# Note: we shouldn't need log_messages except for the seemingly necessary
-# dependency due to the automatic import in the isc package (its __init__.py
-# imports some other modules)
-PYTHONPATH=@abs_top_builddir@/src/lib/python:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python/isc/log_messages
+# Note: we shouldn't need log_messages and lib/dns except for the seemingly
+# necessary dependency due to the automatic import in the isc package (its
+# __init__.py imports some other modules)
+# #2145 should eliminate the need for them.
+PYTHONPATH=@abs_top_builddir@/src/lib/python:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/dns/python/.libs
 export PYTHONPATH
 
 # Likewise, we need only because isc.log requires some loadable modules.

src/bin/xfrin/tests/xfrin_test.py

@@ -60,7 +60,7 @@ TSIG_KEY = TSIGKey("example.com:SFuWd/q99SzF8Yzd1QbB9g==")
 
 # SOA intended to be used for the new SOA as a result of transfer.
 soa_rdata = Rdata(RRType.SOA(), TEST_RRCLASS,
-                  'master.example.com. admin.example.com ' +
+                  'master.example.com. admin.example.com. ' +
                   '1234 3600 1800 2419200 7200')
 soa_rrset = RRset(TEST_ZONE_NAME, TEST_RRCLASS, RRType.SOA(), RRTTL(3600))
 soa_rrset.add_rdata(soa_rdata)
@@ -68,7 +68,7 @@ soa_rrset.add_rdata(soa_rdata)
 # SOA intended to be used for the current SOA at the secondary side.
 # Note that its serial is smaller than that of soa_rdata.
 begin_soa_rdata = Rdata(RRType.SOA(), TEST_RRCLASS,
-                        'master.example.com. admin.example.com ' +
+                        'master.example.com. admin.example.com. ' +
                         '1230 3600 1800 2419200 7200')
 begin_soa_rrset = RRset(TEST_ZONE_NAME, TEST_RRCLASS, RRType.SOA(), RRTTL(3600))
 begin_soa_rrset.add_rdata(begin_soa_rdata)

src/bin/xfrin/xfrin_messages.mes

@@ -24,7 +24,7 @@ The serial fields of the first and last SOAs of AXFR (including AXFR-style
 IXFR) are not the same.  According to RFC 5936 these two SOAs must be the
 "same" (not only for the serial), but it is still not clear what the
 receiver should do if this condition does not hold.  There was a discussion
-about this at the IETF dnsext wg:
+about this at the IETF dnsext working group:
 http://www.ietf.org/mail-archive/web/dnsext/current/msg07908.html
 and the general feeling seems that it would be better to reject the
 transfer if a mismatch is detected.  On the other hand, also as noted
@@ -61,10 +61,10 @@ There was an error opening a connection to the master. The error is
 shown in the log message.
 
 % XFRIN_GOT_INCREMENTAL_RESP got incremental response for %1
-In an attempt of IXFR processing, the begenning SOA of the first difference
+In an attempt of IXFR processing, the beginning SOA of the first difference
 (following the initial SOA that specified the final SOA for all the
 differences) was found.  This means a connection for xfrin tried IXFR
-and really aot a response for incremental updates.
+and really got a response for incremental updates.
 
 % XFRIN_GOT_NONINCREMENTAL_RESP got nonincremental response for %1
 Non incremental transfer was detected at the "first data" of a transfer,
@@ -149,16 +149,16 @@ daemon will now shut down.
 The AXFR transfer of the given zone was successful.
 The provided information contains the following values:
 
-messages: Number of overhead DNS messages in the transfer
+messages: Number of overhead DNS messages in the transfer.
 
 records: Number of Resource Records in the full transfer, excluding the
 final SOA record that marks the end of the AXFR.
 
 bytes: Full size of the transfer data on the wire.
 
-run time: Time (in seconds) the complete axfr took
+run time: Time (in seconds) the complete axfr took.
 
-bytes/second: Transfer speed
+bytes/second: Transfer speed.
 
 % XFRIN_TSIG_KEY_NOT_FOUND TSIG key not found in key ring: %1
 An attempt to start a transfer with TSIG was made, but the configured TSIG

src/bin/xfrout/tests/xfrout_test.py.in

@@ -249,7 +249,7 @@ class TestXfroutSessionBase(unittest.TestCase):
             # In the RDATA only the serial matters.
             for i in range(0, num_soa):
                 soa.add_rdata(Rdata(RRType.SOA(), soa_class,
-                                    'm r ' + str(ixfr) + ' 1 1 1 1'))
+                                    'm. r. ' + str(ixfr) + ' 1 1 1 1'))
             msg.add_rrset(Message.SECTION_AUTHORITY, soa)
 
         renderer = MessageRenderer()

src/bin/xfrout/xfrout_messages.mes

@@ -70,7 +70,7 @@ AXFR-style IXFR.
 
 % XFROUT_IXFR_NO_ZONE IXFR client %1, %2: zone not found with journal
 The requested zone in IXFR was not found in the data source
-even though the xfrout daemon sucessfully found the SOA RR of the zone
+even though the xfrout daemon successfully found the SOA RR of the zone
 in the data source.  This can happen if the administrator removed the
 zone from the data source within the small duration between these
 operations, but it's more likely to be a bug or broken data source.
@@ -84,9 +84,6 @@ NOTAUTH.
 An IXFR request was received, but the client's SOA version is the same as
 or newer than that of the server.  The xfrout server responds to the
 request with the answer section being just one SOA of that version.
-Note: as of this wrting the 'newer version' cannot be identified due to
-the lack of support for the serial number arithmetic.  This will soon
-be implemented.
 
 % XFROUT_MODULECC_SESSION_ERROR error encountered by configuration/command module: %1
 There was a problem in the lower level module handling configuration and
@@ -206,7 +203,7 @@ xfrout daemon process is still running. This xfrout daemon (the one
 printing this message) will not start.
 
 % XFROUT_XFR_TRANSFER_CHECK_ERROR %1 client %2: check for transfer of %3 failed: %4
-Pre-response check for an incomding XFR request failed unexpectedly.
+Pre-response check for an incoming XFR request failed unexpectedly.
 The most likely cause of this is that some low level error in the data
 source, but it may also be other general (more unlikely) errors such
 as memory shortage.  Some detail of the error is also included in the

src/lib/cc/data.h

@@ -109,8 +109,7 @@ public:
 
     /// \name pure virtuals, every derived class must implement these
 
-    /// \returns true if the other ElementPtr has the same type and
-    ///          value
+    /// \return true if the other ElementPtr has the same type and value
     virtual bool equals(const Element& other) const = 0;
 
     /// Converts the Element to JSON format and appends it to

src/lib/config/config_data.cc

@@ -235,7 +235,7 @@ ConfigData::getItemList(const std::string& identifier, bool recurse) const {
 ConstElementPtr
 ConfigData::getFullConfig() const {
     ElementPtr result = Element::createMap();
-    ConstElementPtr items = getItemList("", true);
+    ConstElementPtr items = getItemList("", false);
     BOOST_FOREACH(ConstElementPtr item, items->listValue()) {
         result->set(item->stringValue(), getValue(item->stringValue()));
     }

src/lib/config/config_data.h

@@ -93,8 +93,8 @@ public:
     void setLocalConfig(isc::data::ElementPtr config) { _config = config; }
 
     /// Returns the local (i.e. non-default) configuration.
-    /// \returns An ElementPtr pointing to a MapElement containing all
-    ///          non-default configuration options.
+    /// \return An ElementPtr pointing to a MapElement containing all
+    ///         non-default configuration options.
     isc::data::ElementPtr getLocalConfig() { return (_config); }
 
     /// Returns a list of all possible configuration options as specified
@@ -110,11 +110,11 @@ public:
     isc::data::ConstElementPtr getItemList(const std::string& identifier = "",
                                            bool recurse = false) const;
 
-    /// Returns all current configuration settings (both non-default and default).
+    /// Returns a map of the top-level configuration items, as currently
+    /// set or their defaults
+    ///
     /// \return An ElementPtr pointing to a MapElement containing
-    ///         string->value elements, where the string is the
-    ///         full identifier of the configuration option and the
-    ///         value is an ElementPtr with the value.
+    ///         the top-level configuration items
     isc::data::ConstElementPtr getFullConfig() const;
 
 private:
@@ -126,6 +126,6 @@ private:
 }
 #endif
 
-// Local Variables: 
+// Local Variables:
 // mode: c++
-// End: 
+// End:

src/lib/config/tests/config_data_unittests.cc

@@ -118,7 +118,7 @@ TEST(ConfigData, getLocalConfig) {
     ModuleSpec spec2 = moduleSpecFromFile(std::string(TEST_DATA_PATH) + "/spec2.spec");
     ConfigData cd = ConfigData(spec2);
     EXPECT_EQ("{  }", cd.getLocalConfig()->str());
-    
+
     ElementPtr my_config = Element::fromJSON("{ \"item1\": 2 }");
     cd.setLocalConfig(my_config);
     EXPECT_EQ("{ \"item1\": 2 }", cd.getLocalConfig()->str());
@@ -141,12 +141,15 @@ TEST(ConfigData, getFullConfig) {
     ModuleSpec spec2 = moduleSpecFromFile(std::string(TEST_DATA_PATH) + "/spec2.spec");
     ConfigData cd = ConfigData(spec2);
 
-    EXPECT_EQ("{ \"item1\": 1, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6/value1\": \"default\", \"item6/value2\": None }", cd.getFullConfig()->str());
+    EXPECT_EQ("{ \"item1\": 1, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6\": {  } }", cd.getFullConfig()->str());
     ElementPtr my_config = Element::fromJSON("{ \"item1\": 2 }");
     cd.setLocalConfig(my_config);
-    EXPECT_EQ("{ \"item1\": 2, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6/value1\": \"default\", \"item6/value2\": None }", cd.getFullConfig()->str());
+    EXPECT_EQ("{ \"item1\": 2, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6\": {  } }", cd.getFullConfig()->str());
     ElementPtr my_config2 = Element::fromJSON("{ \"item6\": { \"value1\": \"a\" } }");
     cd.setLocalConfig(my_config2);
-    EXPECT_EQ("{ \"item1\": 1, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6/value1\": \"a\", \"item6/value2\": None }", cd.getFullConfig()->str());
+    EXPECT_EQ("{ \"item1\": 1, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6\": { \"value1\": \"a\" } }", cd.getFullConfig()->str());
+    ElementPtr my_config3 = Element::fromJSON("{ \"item6\": { \"value2\": 123 } }");
+    cd.setLocalConfig(my_config3);
+    EXPECT_EQ("{ \"item1\": 1, \"item2\": 1.1, \"item3\": true, \"item4\": \"test\", \"item5\": [ \"a\", \"b\" ], \"item6\": { \"value2\": 123 } }", cd.getFullConfig()->str());
 }
 

src/lib/datasrc/Makefile.am

@@ -38,6 +38,7 @@ libb10_datasrc_la_SOURCES += client_list.h client_list.cc
 libb10_datasrc_la_SOURCES += memory_datasrc.h memory_datasrc.cc
 libb10_datasrc_la_SOURCES += master_loader_callbacks.h
 libb10_datasrc_la_SOURCES += master_loader_callbacks.cc
+libb10_datasrc_la_SOURCES += rrset_collection_base.h rrset_collection_base.cc
 libb10_datasrc_la_SOURCES += zone_loader.h zone_loader.cc
 nodist_libb10_datasrc_la_SOURCES = datasrc_messages.h datasrc_messages.cc
 libb10_datasrc_la_LDFLAGS = -no-undefined -version-info 1:0:1

src/lib/datasrc/client.cc

@@ -28,20 +28,22 @@ namespace datasrc {
 
 ZoneIteratorPtr
 DataSourceClient::getIterator(const isc::dns::Name&, bool) const {
-    isc_throw(isc::NotImplemented,
-              "Data source doesn't support iteration");
+    isc_throw(isc::NotImplemented, "Data source doesn't support iteration");
 }
 
 unsigned int
 DataSourceClient::getZoneCount() const {
-    isc_throw(isc::NotImplemented,
-              "Data source doesn't support getZoneCount");
+    isc_throw(isc::NotImplemented, "Data source doesn't support getZoneCount");
 }
 
 bool
 DataSourceClient::createZone(const dns::Name&) {
-    isc_throw(isc::NotImplemented,
-              "Data source doesn't support addZone");
+    isc_throw(isc::NotImplemented, "Data source doesn't support createZone");
+}
+
+bool
+DataSourceClient::deleteZone(const dns::Name&) {
+    isc_throw(isc::NotImplemented, "Data source doesn't support deleteZone");
 }
 
 } // end namespace datasrc

src/lib/datasrc/client.h

@@ -385,9 +385,41 @@ public:
     ///                       direct zone creation.
     /// \throw DataSourceError If something goes wrong in the data source
     ///                        while creating the zone.
-    /// \param name The (fully qualified) name of the zone to create
+    /// \param zone_name The (fully qualified) name of the zone to create
     /// \return True if the zone was added, false if it already existed
-    virtual bool createZone(const dns::Name& name);
+    virtual bool createZone(const dns::Name& zone_name);
+
+    /// \brief Delete a zone from the data source
+    ///
+    /// This method also checks if the specified zone exists in the data
+    /// source, and returns true/false depending on whether the zone
+    /// existed/not existed, respectively.  In either case, on successful
+    /// return it ensures the data source does not contain the specified
+    /// name of the zone.
+    ///
+    /// \note This is a tentative API, and this method is likely to change
+    /// or be removed in the near future. For that reason, it currently
+    /// provides a default implementation that throws NotImplemented.
+    /// Note also that this method does not delete other database records
+    /// related to the zone, such as zone's resource records or differences
+    /// corresponding to updates made in the zone.  This is primarily for
+    /// implementation simplicity (in the currently intended usage there
+    /// wouldn't be such other data at the time of this call anyway) and due
+    /// to the fact that details of managing zones is still in flux.  Once
+    /// the design in this area is fixed we may revisit the behavior.
+    ///
+    /// Apart from the two exceptions mentioned below, in theory this
+    /// call can throw anything, depending on the implementation of
+    /// the datasource backend.
+    ///
+    /// \throw NotImplemented If the datasource backend does not support
+    ///                       direct zone deletion.
+    /// \throw DataSourceError If something goes wrong in the data source
+    ///                        while deleting the zone.
+    /// \param zone_name The (fully qualified) name of the zone to be deleted
+    /// \return true if the zone previously existed and has been deleted by
+    /// this method; false if the zone didn't exist.
+    virtual bool deleteZone(const dns::Name& zone_name);
 };
 }
 }

src/lib/datasrc/database.cc

@@ -19,6 +19,7 @@
 #include <datasrc/database.h>
 #include <datasrc/data_source.h>
 #include <datasrc/iterator.h>
+#include <datasrc/rrset_collection_base.h>
 
 #include <exceptions/exceptions.h>
 #include <dns/name.h>
@@ -117,13 +118,25 @@ DatabaseClient::findZone(const Name& name) const {
 }
 
 bool
-DatabaseClient::createZone(const Name& name) {
+DatabaseClient::createZone(const Name& zone_name) {
     TransactionHolder transaction(*accessor_);
-    std::pair<bool, int> zone(accessor_->getZone(name.toText()));
+    const std::pair<bool, int> zone(accessor_->getZone(zone_name.toText()));
     if (zone.first) {
         return (false);
     }
-    accessor_->addZone(name.toText());
+    accessor_->addZone(zone_name.toText());
+    transaction.commit();
+    return (true);
+}
+
+bool
+DatabaseClient::deleteZone(const Name& zone_name) {
+    TransactionHolder transaction(*accessor_);
+    const std::pair<bool, int> zinfo(accessor_->getZone(zone_name.toText()));
+    if (!zinfo.first) {         // if it doesn't exist just return false
+        return (false);
+    }
+    accessor_->deleteZone(zinfo.second);
     transaction.commit();
     return (true);
 }
@@ -1372,6 +1385,36 @@ DatabaseClient::getIterator(const isc::dns::Name& name,
     return (iterator);
 }
 
+/// \brief datasrc implementation of RRsetCollectionBase.
+class RRsetCollection : public isc::datasrc::RRsetCollectionBase {
+public:
+    /// \brief Constructor.
+    RRsetCollection(ZoneUpdater& updater, const isc::dns::RRClass& rrclass) :
+        isc::datasrc::RRsetCollectionBase(updater, rrclass)
+    {}
+
+    /// \brief Destructor
+    virtual ~RRsetCollection() {}
+
+    /// \brief A wrapper around \c disable() so that it can be used as a
+    /// public method. \c disable() is protected.
+    void disableWrapper() {
+        disable();
+    }
+
+protected:
+    // TODO: RRsetCollectionBase::Iter is not implemented and the
+    // following two methods just throw.
+
+    virtual RRsetCollectionBase::IterPtr getBeginning() {
+        isc_throw(NotImplemented, "This method is not implemented.");
+    }
+
+    virtual RRsetCollectionBase::IterPtr getEnd() {
+        isc_throw(NotImplemented, "This method is not implemented.");
+    }
+};
+
 //
 // Zone updater using some database system as the underlying data source.
 //
@@ -1411,6 +1454,15 @@ public:
 
     virtual ZoneFinder& getFinder() { return (*finder_); }
 
+    virtual isc::datasrc::RRsetCollectionBase& getRRsetCollection() {
+        if (!rrset_collection_) {
+            // This is only assigned the first time and remains for the
+            // lifetime of the DatabaseUpdater.
+            rrset_collection_.reset(new RRsetCollection(*this, zone_class_));
+        }
+        return (*rrset_collection_);
+    }
+
     virtual void addRRset(const AbstractRRset& rrset);
     virtual void deleteRRset(const AbstractRRset& rrset);
     virtual void commit();
@@ -1435,6 +1487,7 @@ private:
     DiffPhase diff_phase_;
     Serial serial_;
     boost::scoped_ptr<DatabaseClient::Finder> finder_;
+    boost::shared_ptr<isc::datasrc::RRsetCollection> rrset_collection_;
 
     // This is a set of validation checks commonly used for addRRset() and
     // deleteRRset to minimize duplicate code logic and to make the main
@@ -1565,6 +1618,14 @@ isNSEC3KindType(RRType rrtype, const Rdata& rdata) {
 
 void
 DatabaseUpdater::addRRset(const AbstractRRset& rrset) {
+    if (rrset_collection_) {
+        isc_throw(InvalidOperation,
+                  "Cannot add RRset after an RRsetCollection has been "
+                  "requested for ZoneUpdater for "
+                  << zone_name_ << "/" << zone_class_ << " on "
+                  << db_name_);
+    }
+
     validateAddOrDelete("add", rrset, DELETE, ADD);
 
     // It's guaranteed rrset has at least one RDATA at this point.
@@ -1615,6 +1676,14 @@ DatabaseUpdater::addRRset(const AbstractRRset& rrset) {
 
 void
 DatabaseUpdater::deleteRRset(const AbstractRRset& rrset) {
+    if (rrset_collection_) {
+        isc_throw(InvalidOperation,
+                  "Cannot delete RRset after an RRsetCollection has been "
+                  "requested for ZoneUpdater for "
+                  << zone_name_ << "/" << zone_class_ << " on "
+                  << db_name_);
+    }
+
     // If this is the first operation, pretend we are starting a new delete
     // sequence after adds.  This will simplify the validation below.
     if (diff_phase_ == NOT_STARTED) {
@@ -1669,6 +1738,11 @@ DatabaseUpdater::commit() {
     accessor_->commit();
     committed_ = true; // make sure the destructor won't trigger rollback
 
+    // Disable the RRsetCollection if it exists.
+    if (rrset_collection_) {
+        rrset_collection_->disableWrapper();
+    }
+
     // We release the accessor immediately after commit is completed so that
     // we don't hold the possible internal resource any longer.
     accessor_.reset();
@@ -1745,7 +1819,7 @@ public:
                 arg(zone_).arg(rrclass_).arg(accessor_->getDBName());
             return (rrset);
         } catch (const Exception& ex) {
-            LOG_ERROR(logger, DATASRC_DATABASE_JOURNALREADR_BADDATA).
+            LOG_ERROR(logger, DATASRC_DATABASE_JOURNALREADER_BADDATA).
                 arg(zone_).arg(rrclass_).arg(accessor_->getDBName()).
                 arg(begin_).arg(end_).arg(ex.what());
             isc_throw(DataSourceError, "Failed to create RRset from diff on "

src/lib/datasrc/database.h

@@ -155,7 +155,7 @@ public:
     ///
     /// It is empty, but needs a virtual one, since we will use the derived
     /// classes in polymorphic way.
-    virtual ~DatabaseAccessor() { }
+    virtual ~DatabaseAccessor() {}
 
     /// \brief Retrieve a zone identifier
     ///
@@ -164,8 +164,8 @@ public:
     /// apex), as the DatabaseClient will loop trough the labels itself and
     /// find the most suitable zone.
     ///
-    /// It is not specified if and what implementation of this method may throw,
-    /// so code should expect anything.
+    /// It is not specified if and what implementation of this method may
+    /// throw, so code should expect anything.
     ///
     /// \param name The (fully qualified) domain name of the zone's apex to be
     ///             looked up.
@@ -195,6 +195,23 @@ public:
     ///         or was created by this call).
     virtual int addZone(const std::string& name) = 0;
 
+    /// \brief Delete a zone from the database
+    ///
+    /// Like for deleteRecordToZone, implementations are not required to
+    /// check for the existence of the given zone name, it is the
+    /// responsibility of the caller to do so.
+    ///
+    /// Callers must also start a transaction before calling this method.
+    /// Implementations should throw InvalidOperation if this has not been
+    /// done. Callers should also expect DataSourceError for other potential
+    /// problems specific to the database.
+    ///
+    /// \note This method does not delete other database records related to
+    /// the zone.  See \c DataSourceClient::deleteZone for the rationale.
+    ///
+    /// \param zone_id The ID of the zone, that would be returned by getZone().
+    virtual void deleteZone(int zone_id) = 0;
+
     /// \brief This holds the internal context of ZoneIterator for databases
     ///
     /// While the ZoneIterator implementation from DatabaseClient does all the
@@ -212,15 +229,15 @@ public:
         /// \brief Destructor
         ///
         /// Virtual destructor, so any descendand class is destroyed correctly.
-        virtual ~IteratorContext() { }
+        virtual ~IteratorContext() {}
 
         /// \brief Function to provide next resource record
         ///
         /// This function should provide data about the next resource record
         /// from the data that is searched. The data is not converted yet.
         ///
-        /// Depending on how the iterator was constructed, there is a difference
-        /// in behaviour; for a 'full zone iterator', created with
+        /// Depending on how the iterator was constructed, there is a
+        /// difference in behaviour; for a 'full zone iterator', created with
         /// getAllRecords(), all COLUMN_COUNT elements of the array are
         /// overwritten.
         /// For a 'name iterator', created with getRecords(), the column
@@ -1399,7 +1416,9 @@ public:
     /// does not, creates it, commits, and returns true. If the zone
     /// does exist already, it does nothing (except abort the transaction)
     /// and returns false.
-    virtual bool createZone(const isc::dns::Name& name);
+    virtual bool createZone(const isc::dns::Name& zone_name);
+
+    virtual bool deleteZone(const isc::dns::Name& zone_name);
 
     /// \brief Get the zone iterator
     ///

src/lib/datasrc/datasrc_messages.mes

@@ -70,6 +70,19 @@ The maximum allowed number of items of the hotspot cache is set to the given
 number. If there are too many, some of them will be dropped. The size of 0
 means no limit.
 
+% DATASRC_CHECK_ERROR post-load check of zone %1/%2 failed: %3
+The zone was loaded into the data source successfully, but the content fails
+basic sanity checks. See the message if you want to know what exactly is wrong
+with the data. The data can not be used and previous version, if any, will be
+preserved.
+
+% DATASRC_CHECK_WARNING %1/%2: %3
+The zone was loaded into the data source successfully, but there's some problem
+with the content. The problem does not stop the new version from being used
+(though there may be other problems that do, see DATASRC_CHECK_ERROR),
+but it should still be checked and fixed. See the message to know what exactly
+is wrong with the data.
+
 % DATASRC_DATABASE_COVER_NSEC_UNSUPPORTED %1 doesn't support DNSSEC when asked for NSEC data covering %2
 The datasource tried to provide an NSEC proof that the named domain does not
 exist, but the database backend doesn't support DNSSEC. No proof is included
@@ -144,7 +157,7 @@ instead.
 % DATASRC_DATABASE_FOUND_EMPTY_NONTERMINAL empty non-terminal %2 in %1
 The domain name does not have any RRs associated with it, so it doesn't
 exist in the database.  However, it has a subdomain, so it does exist
-in the DNS address space. This type of domain is known an an "empty
+in the DNS address space. This type of domain is known as an "empty
 non-terminal" and so we return NXRRSET instead of NXDOMAIN.
 
 % DATASRC_DATABASE_FOUND_NXDOMAIN search in datasource %1 resulted in NXDOMAIN for %2/%3/%4
@@ -202,7 +215,7 @@ a zone's difference sequences from a database-based data source.  The
 zone's name and class, database name, and the start and end serials
 are shown in the message.
 
-% DATASRC_DATABASE_JOURNALREADR_BADDATA failed to convert a diff to RRset in %1/%2 on %3 between %4 and %5: %6
+% DATASRC_DATABASE_JOURNALREADER_BADDATA failed to convert a diff to RRset in %1/%2 on %3 between %4 and %5: %6
 This is an error message indicating that a zone's diff is broken and
 the data source library failed to convert it to a valid RRset.  The
 most likely cause of this is that someone has manually modified the
@@ -287,7 +300,7 @@ matching the name.  This is returned as the result of the search.
 The given wildcard matches the name being sough but it as an empty
 nonterminal (e.g. there's nothing at *.example.com but something like
 subdomain.*.example.org, do exist: so *.example.org exists in the
-namespace but has no RRs assopciated with it). This will produce NXRRSET.
+namespace but has no RRs associated with it). This will produce NXRRSET.
 
 % DATASRC_DATABASE_WILDCARD_MATCH search in datasource %1 resulted in wildcard match at %2 with RRset %3
 The database doesn't contain directly matching name.  When searching
@@ -490,7 +503,7 @@ destroyed.
 % DATASRC_MEM_WILDCARD_CANCEL wildcard match canceled for '%1'
 Debug information. A domain above wildcard was reached, but there's something
 below the requested domain. Therefore the wildcard doesn't apply here.  This
-behaviour is specified by RFC 1034, section 4.3.3
+behaviour is specified by RFC 1034, section 4.3.3.
 
 % DATASRC_MEM_WILDCARD_DNAME DNAME record in wildcard domain '%1'
 The software refuses to load DNAME records into a wildcard domain.  It isn't

src/lib/datasrc/rrset_collection_base.cc

@@ -0,0 +1,71 @@
+// Copyright (C) 2013  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#include <datasrc/rrset_collection_base.h>
+#include <datasrc/zone_loader.h>
+#include <exceptions/exceptions.h>
+
+using namespace isc;
+using namespace isc::dns;
+
+namespace isc {
+namespace datasrc {
+
+ConstRRsetPtr
+isc::datasrc::RRsetCollectionBase::find(const isc::dns::Name& name,
+                                        const isc::dns::RRClass& rrclass,
+                                        const isc::dns::RRType& rrtype) const
+{
+    if (isDisabled()) {
+        isc_throw(RRsetCollectionError, "This RRsetCollection is disabled.");
+    }
+
+    if (rrclass != rrclass_) {
+        // We could throw an exception here, but RRsetCollection is
+        // expected to support an arbitrary collection of RRsets, and it
+        // can be queried just as arbitrarily. So we just return nothing
+        // here.
+        return (ConstRRsetPtr());
+    }
+
+    ZoneFinder& finder = updater_.getFinder();
+    try {
+        ZoneFinderContextPtr result =
+            finder.find(name, rrtype,
+                        ZoneFinder::NO_WILDCARD | ZoneFinder::FIND_GLUE_OK);
+        // We return the result rrset only if the result code is
+        // SUCCESS. We return empty if CNAME, DNAME, DELEGATION,
+        // etc. are returned by the ZoneFinder.
+        //
+        // Note that in the case that the queried type itself is CNAME
+        // or DNAME, then the finder will return SUCCESS.
+        if (result->code == ZoneFinder::SUCCESS) {
+            return (result->rrset);
+        } else {
+            return (ConstRRsetPtr());
+        }
+    } catch (const OutOfZone&) {
+        // As RRsetCollection is an arbitrary set of RRsets, in case the
+        // searched name is out of zone, we return nothing instead of
+        // propagating the exception.
+        return (ConstRRsetPtr());
+    } catch (const DataSourceError& e) {
+        isc_throw(RRsetCollectionError,
+                  "ZoneFinder threw a DataSourceError: "
+                      << e.getMessage().c_str());
+    }
+}
+
+} // end of namespace datasrc
+} // end of namespace isc

src/lib/datasrc/rrset_collection_base.h

@@ -0,0 +1,126 @@
+// Copyright (C) 2013  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#ifndef RRSET_COLLECTION_DATASRC_H
+#define RRSET_COLLECTION_DATASRC_H 1
+
+#include <dns/rrset_collection_base.h>
+#include <dns/rrclass.h>
+#include <datasrc/zone.h>
+
+namespace isc {
+namespace datasrc {
+
+/// \brief A forward declaration
+class ZoneUpdater;
+
+/// \brief datasrc derivation of \c isc::dns::RRsetCollectionBase.
+///
+/// This is an abstract class that adds datasrc related detail to
+/// \c isc::dns::RRsetCollectionBase. Derived classes need to complete
+/// the implementation (add iterator support, etc.) before using it.
+class RRsetCollectionBase : public isc::dns::RRsetCollectionBase {
+public:
+    /// \brief Constructor.
+    ///
+    /// No reference (count via \c shared_ptr) to the \c ZoneUpdater is
+    /// acquired. The RRsetCollection must not be used after its
+    /// \c ZoneUpdater has been destroyed.
+    ///
+    /// \param updater The ZoneUpdater to wrap around.
+    /// \param rrclass The RRClass of the records in the zone.
+    RRsetCollectionBase(ZoneUpdater& updater,
+                        const isc::dns::RRClass& rrclass) :
+        updater_(updater),
+        rrclass_(rrclass),
+        disabled_(false)
+    {}
+
+    /// \brief Destructor
+    virtual ~RRsetCollectionBase() {}
+
+    /// \brief Find a matching RRset in the collection.
+    ///
+    /// Returns the RRset in the collection that exactly matches the
+    /// given \c name, \c rrclass and \c rrtype.  If no matching RRset
+    /// is found, \c NULL is returned.
+    ///
+    /// Note that not all records added through the updater may
+    /// necessarily be found by this method, such as RRs subject to
+    /// DNAME substitution.
+    ///
+    /// \throw isc::dns::RRsetCollectionError if \c find() results in
+    /// some underlying datasrc error, or if \c disable() was called.
+    ///
+    /// \param name The name of the RRset to search for.
+    /// \param rrclass The class of the RRset to search for.
+    /// \param rrtype The type of the RRset to search for.
+    /// \returns The RRset if found, \c NULL otherwise.
+    virtual isc::dns::ConstRRsetPtr find(const isc::dns::Name& name,
+                                         const isc::dns::RRClass& rrclass,
+                                         const isc::dns::RRType& rrtype) const;
+
+protected:
+    /// \brief Disable the RRsetCollection.
+    ///
+    /// After calling this method, calling operations such as find() or
+    /// using the iterator would result in an \c
+    /// isc::dns::RRsetCollectionError. This method is typically called
+    /// in the \c commit() implementations of some \c ZoneUpdaters.
+    void disable() {
+        disabled_ = true;
+    }
+
+    /// \brief Return if the RRsetCollection is disabled.
+    bool isDisabled() const {
+        return (disabled_);
+    }
+
+    /// \brief See \c isc::dns::RRsetCollectionBase::getBeginning() for
+    /// documentation.
+    ///
+    /// \throw isc::dns::RRsetCollectionError if using the iterator
+    /// results in some underlying datasrc error, or if \c disable() was
+    /// called.
+    virtual IterPtr getBeginning() = 0;
+
+    /// \brief See \c isc::dns::RRsetCollectionBase::getEnd() for
+    /// documentation.
+    ///
+    /// \throw isc::dns::RRsetCollectionError if using the iterator
+    /// results in some underlying datasrc error, or if \c disable() was
+    /// called.
+    virtual IterPtr getEnd() = 0;
+
+private:
+    ZoneUpdater& updater_;
+    isc::dns::RRClass rrclass_;
+    bool disabled_;
+};
+
+/// \brief A pointer-like type pointing to an
+/// \c isc::datasrc::RRsetCollectionBase object.
+///
+/// This type is used to handle RRsetCollections in a polymorphic manner
+/// in libdatasrc.
+typedef boost::shared_ptr<isc::datasrc::RRsetCollectionBase> RRsetCollectionPtr;
+
+} // end of namespace datasrc
+} // end of namespace isc
+
+#endif  // RRSET_COLLECTION_DATASRC_H
+
+// Local Variables:
+// mode: c++
+// End:

src/lib/datasrc/sqlite3_accessor.cc

@@ -79,7 +79,8 @@ enum StatementID {
     DEL_ZONE_NSEC3_RECORDS = 20,
     DEL_NSEC3_RECORD = 21,
     ADD_ZONE = 22,
-    NUM_STATEMENTS = 23
+    DELETE_ZONE = 23,
+    NUM_STATEMENTS = 24
 };
 
 const char* const text_statements[NUM_STATEMENTS] = {
@@ -165,7 +166,9 @@ const char* const text_statements[NUM_STATEMENTS] = {
     "AND rdtype=?3 AND rdata=?4",
 
     // ADD_ZONE: add a zone to the zones table
-    "INSERT INTO zones (name, rdclass) VALUES (?1, ?2)" // ADD_ZONE
+    "INSERT INTO zones (name, rdclass) VALUES (?1, ?2)", // ADD_ZONE
+    // DELETE_ZONE: delete a zone from the zones table
+    "DELETE FROM zones WHERE id=?1" // DELETE_ZONE
 };
 
 struct SQLite3Parameters {
@@ -643,6 +646,19 @@ SQLite3Accessor::addZone(const std::string& name) {
     return (getzone_result.second);
 }
 
+void
+SQLite3Accessor::deleteZone(int zone_id) {
+    // Transaction should have been started by the caller
+    if (!dbparameters_->in_transaction) {
+        isc_throw(InvalidOperation, "performing deleteZone on SQLite3 "
+                  "data source without transaction");
+    }
+
+    StatementProcessor proc(*dbparameters_, DELETE_ZONE, "delete zone");
+    proc.bindInt(1, zone_id);
+    proc.exec();
+}
+
 namespace {
 
 // Conversion to plain char

src/lib/datasrc/sqlite3_accessor.h

@@ -139,6 +139,10 @@ public:
     /// \return the id of the zone that has been added
     virtual int addZone(const std::string& name);
 
+    // Nothing special to add for this implementation (the base class
+    // description is sufficient).
+    virtual void deleteZone(int zone_id);
+
     /// \brief Look up all resource records for a name
     ///
     /// This implements the getRecords() method from DatabaseAccessor

src/lib/datasrc/tests/Makefile.am

@@ -118,3 +118,5 @@ EXTRA_DIST += testdata/new_minor_schema.sqlite3
 EXTRA_DIST += testdata/newschema.sqlite3
 EXTRA_DIST += testdata/oldschema.sqlite3
 EXTRA_DIST += testdata/static.zone
+EXTRA_DIST += testdata/novalidate.zone
+EXTRA_DIST += testdata/checkwarn.zone

src/lib/datasrc/tests/database_unittest.cc

@@ -55,6 +55,7 @@ namespace {
 
 // Imaginary zone IDs used in the mock accessor below.
 const int READONLY_ZONE_ID = 42;
+const int NEW_ZONE_ID = 420;
 const int WRITABLE_ZONE_ID = 4200;
 
 // Commonly used test data
@@ -257,26 +258,43 @@ const char* TEST_NSEC3_RECORDS[][5] = {
  */
 class NopAccessor : public DatabaseAccessor {
 public:
-    NopAccessor() : database_name_("mock_database")
-    { }
+    NopAccessor() : database_name_("mock_database") {
+        zones_["example.org."] = READONLY_ZONE_ID;
+        zones_["null.example.org."] = 13;
+        zones_["empty.example.org."] = 0;
+        zones_["bad.example.org."] = -1;
+    }
 
     virtual std::pair<bool, int> getZone(const std::string& name) const {
-        if (name == "example.org.") {
-            return (std::pair<bool, int>(true, READONLY_ZONE_ID));
-        } else if (name == "null.example.org.") {
-            return (std::pair<bool, int>(true, 13));
-        } else if (name == "empty.example.org.") {
-            return (std::pair<bool, int>(true, 0));
-        } else if (name == "bad.example.org.") {
-            return (std::pair<bool, int>(true, -1));
+        std::map<std::string, int>::const_iterator found = zones_.find(name);
+        if (found != zones_.end()) {
+            return (std::pair<bool, int>(true, found->second));
         } else {
             return (std::pair<bool, int>(false, 0));
         }
     }
 
-    virtual int addZone(const std::string&) {
-        isc_throw(isc::NotImplemented,
-                  "This database datasource can't add zones");
+    // A simple implementation of addZone.
+    virtual int addZone(const std::string& zone_name) {
+        if (zone_name == "example.com.") {
+            zones_[zone_name] = NEW_ZONE_ID;
+        }
+
+        // for simplicity we assume zone_name is in zones_ at this point
+        return (zones_[zone_name]);
+    }
+
+    // A simple implementation of deleteZone.
+    virtual void deleteZone(int zone_id) {
+        std::map<std::string, int>::iterator it = zones_.begin();
+        std::map<std::string, int>::iterator end = zones_.end();
+        while (it != end) {
+            if (it->second == zone_id) {
+                zones_.erase(it);
+                return;
+            }
+            ++it;
+        }
     }
 
     virtual boost::shared_ptr<DatabaseAccessor> clone() {
@@ -337,7 +355,7 @@ public:
 
 private:
     const std::string database_name_;
-
+    std::map<std::string, int> zones_;
 };
 
 /**
@@ -438,7 +456,7 @@ public:
 
         // Check any attempt of multiple transactions
         if (did_transaction_) {
-            isc_throw(isc::Unexpected, "MockAccessor::startTransaction() "
+            isc_throw(DataSourceError, "MockAccessor::startTransaction() "
                       "called multiple times - likely a bug in the test");
         }
 
@@ -447,6 +465,14 @@ public:
         did_transaction_ = true;
     }
 
+    // If the test needs multiple calls to startTransaction() and knows it's
+    // safe, it can use this method to disable the safeguard check in
+    // startTransaction(); the test can also use this method by emulating a
+    // lock conflict by setting is_allowed to false.
+    void allowMoreTransaction(bool is_allowed) {
+        did_transaction_ = !is_allowed;
+    }
+
 private:
     class DomainIterator : public IteratorContext {
     public:
@@ -1293,6 +1319,17 @@ public:
         }
     }
 
+    // Mock-only; control whether to allow subsequent transaction.
+    void allowMoreTransaction(bool is_allowed) {
+        if (is_mock_) {
+            // Use a separate variable for MockAccessor&; some compilers
+            // would be confused otherwise.
+            MockAccessor& mock_accessor =
+                dynamic_cast<MockAccessor&>(*current_accessor_);
+            mock_accessor.allowMoreTransaction(is_allowed);
+        }
+    }
+
     // Some tests only work for MockAccessor.  We remember whether our accessor
     // is of that type.
     bool is_mock_;
@@ -2184,6 +2221,12 @@ TYPED_TEST(DatabaseClientTest, findDelegation) {
                this->rrttl_, ZoneFinder::DNAME, this->expected_rdatas_,
                this->expected_sig_rdatas_, ZoneFinder::RESULT_DEFAULT,
                isc::dns::Name("dname.example.org."));
+    // below.dname.example.org. has an A record
+    doFindTest(*finder, isc::dns::Name("below.dname.example.org."),
+               isc::dns::RRType::A(), isc::dns::RRType::DNAME(),
+               this->rrttl_, ZoneFinder::DNAME, this->expected_rdatas_,
+               this->expected_sig_rdatas_, ZoneFinder::RESULT_DEFAULT,
+               isc::dns::Name("dname.example.org."));
     doFindTest(*finder, isc::dns::Name("really.deep.below.dname.example.org."),
                isc::dns::RRType::AAAA(), isc::dns::RRType::DNAME(),
                this->rrttl_, ZoneFinder::DNAME, this->expected_rdatas_,
@@ -4104,51 +4147,289 @@ TYPED_TEST(DatabaseClientTest, createZone) {
         zone(this->client_->findZone(new_name));
     ASSERT_EQ(result::NOTFOUND, zone.code);
 
-    // The mock implementation does not do createZone,
-    // in which case it should throw NotImplemented (from
-    // the base class)
-    if (this->is_mock_) {
-        ASSERT_THROW(this->client_->createZone(new_name), isc::NotImplemented);
-    } else {
-        // But in the real case, it should work and return true
-        ASSERT_TRUE(this->client_->createZone(new_name));
-        const DataSourceClient::FindResult
-            zone2(this->client_->findZone(new_name));
-        ASSERT_EQ(result::SUCCESS, zone2.code);
-        // And the second call should return false since
-        // it already exists
-        ASSERT_FALSE(this->client_->createZone(new_name));
-    }
+    // Adding a new zone; it should work and return true
+    ASSERT_TRUE(this->client_->createZone(new_name));
+    const DataSourceClient::FindResult
+        zone2(this->client_->findZone(new_name));
+    ASSERT_EQ(result::SUCCESS, zone2.code);
+    // And the second call should return false since
+    // it already exists
+    this->allowMoreTransaction(true);
+    ASSERT_FALSE(this->client_->createZone(new_name));
 }
 
 TYPED_TEST(DatabaseClientTest, createZoneRollbackOnLocked) {
-    // skip test for mock
-    if (this->is_mock_) {
-        return;
-    }
-
     const Name new_name("example.com");
     isc::datasrc::ZoneUpdaterPtr updater =
         this->client_->getUpdater(this->zname_, true);
+    this->allowMoreTransaction(false);
     ASSERT_THROW(this->client_->createZone(new_name), DataSourceError);
     // createZone started a transaction as well, but since it failed,
     // it should have been rolled back. Roll back the other one as
     // well, and the next attempt should succeed
     updater.reset();
+    this->allowMoreTransaction(true);
     ASSERT_TRUE(this->client_->createZone(new_name));
 }
 
 TYPED_TEST(DatabaseClientTest, createZoneRollbackOnExists) {
-    // skip test for mock
-    if (this->is_mock_) {
-        return;
-    }
-
     const Name new_name("example.com");
     ASSERT_FALSE(this->client_->createZone(this->zname_));
-    // createZone started a transaction, but since it failed,
-    // it should have been rolled back, and the next attempt should succeed
+
+    // deleteZone started a transaction, but since the zone didn't even exist
+    // the transaction was not committed but should have been rolled back.
+    // The first transaction shouldn't leave any state, lock, etc, that
+    // would hinder the second attempt.
+    this->allowMoreTransaction(true);
     ASSERT_TRUE(this->client_->createZone(new_name));
 }
 
+TYPED_TEST(DatabaseClientTest, deleteZone) {
+    // Check the zone currently exists.
+    EXPECT_EQ(result::SUCCESS, this->client_->findZone(this->zname_).code);
+
+    // Deleting an existing zone; it should work and return true (previously
+    // existed and is now deleted)
+    EXPECT_TRUE(this->client_->deleteZone(this->zname_));
+
+    // Now it's not found by findZone
+    EXPECT_EQ(result::NOTFOUND, this->client_->findZone(this->zname_).code);
+
+    // And the second call should return false since it doesn't exist any more
+    this->allowMoreTransaction(true);
+    EXPECT_FALSE(this->client_->deleteZone(this->zname_));
+}
+
+TYPED_TEST(DatabaseClientTest, deleteZoneRollbackOnLocked) {
+    isc::datasrc::ZoneUpdaterPtr updater =
+        this->client_->getUpdater(this->zname_, true);
+
+    // updater locks the DB so deleteZone() will fail.
+    this->allowMoreTransaction(false);
+    EXPECT_THROW(this->client_->deleteZone(this->zname_), DataSourceError);
+
+    // deleteZone started a transaction as well, but since it failed,
+    // it should have been rolled back. Roll back the other one as
+    // well, and the next attempt should succeed
+    updater.reset();
+    this->allowMoreTransaction(true);
+    EXPECT_TRUE(this->client_->deleteZone(this->zname_));
+}
+
+TYPED_TEST(DatabaseClientTest, deleteZoneRollbackOnNotFind) {
+    // attempt of deleting non-existent zone.  result in false
+    const Name new_name("example.com");
+    EXPECT_FALSE(this->client_->deleteZone(new_name));
+
+    // deleteZone started a transaction, but since the zone didn't even exist
+    // the transaction was not committed but should have been rolled back.
+    // The first transaction shouldn't leave any state, lock, etc, that
+    // would hinder the second attempt.
+    this->allowMoreTransaction(true);
+    EXPECT_TRUE(this->client_->deleteZone(this->zname_));
+}
+
+TYPED_TEST_CASE(RRsetCollectionTest, TestAccessorTypes);
+
+// This test fixture is templated so that we can share (most of) the test
+// cases with different types of data sources.  Note that in test cases
+// we need to use 'this' to refer to member variables of the test class.
+template <typename ACCESSOR_TYPE>
+class RRsetCollectionTest : public DatabaseClientTest<ACCESSOR_TYPE> {
+public:
+    RRsetCollectionTest() :
+        DatabaseClientTest<ACCESSOR_TYPE>(),
+        updater(this->client_->getUpdater(this->zname_, false)),
+        collection(updater->getRRsetCollection())
+    {}
+
+    ZoneUpdaterPtr updater;
+    isc::datasrc::RRsetCollectionBase& collection;
+};
+
+TYPED_TEST(RRsetCollectionTest, find) {
+    // Test the find() that returns ConstRRsetPtr
+    ConstRRsetPtr rrset = this->collection.find(Name("www.example.org."),
+                                                RRClass::IN(), RRType::A());
+    ASSERT_TRUE(rrset);
+    EXPECT_EQ(RRType::A(), rrset->getType());
+    EXPECT_EQ(RRTTL(3600), rrset->getTTL());
+    EXPECT_EQ(RRClass("IN"), rrset->getClass());
+    EXPECT_EQ(Name("www.example.org"), rrset->getName());
+
+    // foo.example.org doesn't exist
+    rrset = this->collection.find(Name("foo.example.org"), this->qclass_,
+                                  RRType::A());
+    EXPECT_FALSE(rrset);
+
+    // www.example.org exists, but not with MX
+    rrset = this->collection.find(Name("www.example.org"), this->qclass_,
+                                  RRType::MX());
+    EXPECT_FALSE(rrset);
+
+    // www.example.org exists, with AAAA
+    rrset = this->collection.find(Name("www.example.org"), this->qclass_,
+                                  RRType::AAAA());
+    EXPECT_TRUE(rrset);
+
+    // www.example.org with AAAA does not exist in RRClass::CH()
+    rrset = this->collection.find(Name("www.example.org"), RRClass::CH(),
+                                  RRType::AAAA());
+    EXPECT_FALSE(rrset);
+
+    // Out-of-zone find()s must not throw.
+    rrset = this->collection.find(Name("www.example.com"), this->qclass_,
+                                  RRType::A());
+    EXPECT_FALSE(rrset);
+
+    // "cname.example.org." with type CNAME should return the CNAME RRset
+    rrset = this->collection.find(Name("cname.example.org"), this->qclass_,
+                                  RRType::CNAME());
+    ASSERT_TRUE(rrset);
+    EXPECT_EQ(RRType::CNAME(), rrset->getType());
+    EXPECT_EQ(Name("cname.example.org"), rrset->getName());
+
+    // "cname.example.org." with type A should return nothing
+    rrset = this->collection.find(Name("cname.example.org"), this->qclass_,
+                                  RRType::A());
+    EXPECT_FALSE(rrset);
+
+    // "dname.example.org." with type DNAME should return the DNAME RRset
+    rrset = this->collection.find(Name("dname.example.org"), this->qclass_,
+                                  RRType::DNAME());
+    ASSERT_TRUE(rrset);
+    EXPECT_EQ(RRType::DNAME(), rrset->getType());
+    EXPECT_EQ(Name("dname.example.org"), rrset->getName());
+
+    // "below.dname.example.org." with type AAAA should return nothing
+    rrset = this->collection.find(Name("below.dname.example.org"),
+                                  this->qclass_, RRType::AAAA());
+    EXPECT_FALSE(rrset);
+
+    // "below.dname.example.org." with type A does not return the record
+    // (see top of file). See \c isc::datasrc::RRsetCollectionBase::find()
+    // documentation for details.
+    rrset = this->collection.find(Name("below.dname.example.org"),
+                                  this->qclass_, RRType::A());
+    EXPECT_FALSE(rrset);
+
+    // With the FIND_GLUE_OK option passed to ZoneFinder's find(),
+    // searching for "delegation.example.org." with type NS should
+    // return the NS record. Without FIND_GLUE_OK, ZoneFinder's find()
+    // would return DELEGATION and the find() below would return
+    // nothing.
+    rrset = this->collection.find(Name("delegation.example.org"),
+                                  this->qclass_, RRType::NS());
+    ASSERT_TRUE(rrset);
+    EXPECT_EQ(RRType::NS(), rrset->getType());
+    EXPECT_EQ(Name("delegation.example.org"), rrset->getName());
+
+    // With the NO_WILDCARD option passed to ZoneFinder's find(),
+    // searching for some "foo.wildcard.example.org." would make
+    // ZoneFinder's find() return NXDOMAIN, and the find() below should
+    // return nothing.
+    rrset = this->collection.find(Name("foo.wild.example.org"),
+                                  this->qclass_, RRType::A());
+    EXPECT_FALSE(rrset);
+
+    // Searching directly for "*.wild.example.org." should return the
+    // record.
+    rrset = this->collection.find(Name("*.wild.example.org"),
+                                  this->qclass_, RRType::A());
+    ASSERT_TRUE(rrset);
+    EXPECT_EQ(RRType::A(), rrset->getType());
+    EXPECT_EQ(Name("*.wild.example.org"), rrset->getName());
+}
+
+TYPED_TEST(RRsetCollectionTest, iteratorTest) {
+    // Iterators are currently not implemented.
+    EXPECT_THROW(this->collection.begin(), isc::NotImplemented);
+    EXPECT_THROW(this->collection.end(), isc::NotImplemented);
+}
+
+typedef RRsetCollectionTest<MockAccessor> MockRRsetCollectionTest;
+
+TEST_F(MockRRsetCollectionTest, findError) {
+    // A test using the MockAccessor for checking that FindError is
+    // thrown properly if a find attempt using ZoneFinder results in a
+    // DataSourceError.
+    //
+    // The "dsexception.example.org." name is rigged by the MockAccessor
+    // to throw a DataSourceError.
+    EXPECT_THROW({
+        this->collection.find(Name("dsexception.example.org"), this->qclass_,
+                              RRType::A());
+    }, RRsetCollectionError);
+}
+
+TYPED_TEST_CASE(RRsetCollectionAndUpdaterTest, TestAccessorTypes);
+
+// This test fixture is templated so that we can share (most of) the test
+// cases with different types of data sources.  Note that in test cases
+// we need to use 'this' to refer to member variables of the test class.
+template <typename ACCESSOR_TYPE>
+class RRsetCollectionAndUpdaterTest : public DatabaseClientTest<ACCESSOR_TYPE> {
+public:
+    RRsetCollectionAndUpdaterTest() :
+        DatabaseClientTest<ACCESSOR_TYPE>(),
+        updater_(this->client_->getUpdater(this->zname_, false))
+    {}
+
+    ZoneUpdaterPtr updater_;
+};
+
+// Test that using addRRset() or deleteRRset() on the ZoneUpdater throws
+// after an RRsetCollection is created.
+TYPED_TEST(RRsetCollectionAndUpdaterTest, updateThrows) {
+    // 1. Addition test
+
+    // addRRset() must not throw.
+    this->updater_->addRRset(*this->rrset_);
+
+    // Now setup a new updater and call getRRsetCollection() on it.
+    this->updater_.reset();
+    this->updater_ = this->client_->getUpdater(this->zname_, false);
+    (void) this->updater_->getRRsetCollection();
+
+    // addRRset() must throw isc::InvalidOperation here.
+    EXPECT_THROW(this->updater_->addRRset(*this->rrset_),
+                 isc::InvalidOperation);
+
+    // 2. Deletion test
+
+    // deleteRRset() must not throw.
+    this->updater_.reset();
+    this->updater_ = this->client_->getUpdater(this->zname_, false);
+    this->updater_->addRRset(*this->rrset_);
+    this->updater_->deleteRRset(*this->rrset_);
+
+    // Now setup a new updater and call getRRsetCollection() on it.
+    this->updater_.reset();
+    this->updater_ = this->client_->getUpdater(this->zname_, false);
+    this->updater_->addRRset(*this->rrset_);
+    (void) this->updater_->getRRsetCollection();
+
+    // deleteRRset() must throw isc::InvalidOperation here.
+    EXPECT_THROW(this->updater_->deleteRRset(*this->rrset_),
+                 isc::InvalidOperation);
+}
+
+// Test that using an RRsetCollection after calling commit() on the
+// ZoneUpdater throws, as the RRsetCollection is disabled.
+TYPED_TEST(RRsetCollectionAndUpdaterTest, useAfterCommitThrows) {
+     isc::datasrc::RRsetCollectionBase& collection =
+         this->updater_->getRRsetCollection();
+
+     // find() must not throw here.
+     collection.find(Name("foo.wild.example.org"), this->qclass_, RRType::A());
+
+     this->updater_->commit();
+
+     // find() must throw RRsetCollectionError here, as the
+     // RRsetCollection is disabled.
+     EXPECT_THROW(collection.find(Name("foo.wild.example.org"),
+                                  this->qclass_, RRType::A()),
+                  RRsetCollectionError);
+}
+
 }

src/lib/datasrc/tests/master_loader_callbacks_test.cc

@@ -65,6 +65,9 @@ public:
     virtual ZoneFinder& getFinder() {
         isc_throw(isc::NotImplemented, "Not to be called in this test");
     }
+    virtual isc::datasrc::RRsetCollectionBase& getRRsetCollection() {
+        isc_throw(isc::NotImplemented, "Not to be called in this test");
+    }
     virtual void deleteRRset(const isc::dns::AbstractRRset&) {
         isc_throw(isc::NotImplemented, "Not to be called in this test");
     }

src/lib/datasrc/tests/memory/rdata_serialization_unittest.cc

@@ -71,20 +71,20 @@ struct TestRdata {
 // unusual and corner cases).
 const TestRdata test_rdata_list[] = {
     {"IN", "A", "192.0.2.1", 0},
-    {"IN", "NS", "ns.example.com", 0},
-    {"IN", "CNAME", "cname.example.com", 0},
-    {"IN", "SOA", "ns.example.com root.example.com 0 0 0 0 0", 0},
-    {"IN", "PTR", "reverse.example.com", 0},
+    {"IN", "NS", "ns.example.com.", 0},
+    {"IN", "CNAME", "cname.example.com.", 0},
+    {"IN", "SOA", "ns.example.com. root.example.com. 0 0 0 0 0", 0},
+    {"IN", "PTR", "reverse.example.com.", 0},
     {"IN", "HINFO", "\"cpu-info\" \"OS-info\"", 1},
-    {"IN", "MINFO", "root.example.com mbox.example.com", 0},
-    {"IN", "MX", "10 mx.example.com", 0},
+    {"IN", "MINFO", "root.example.com. mbox.example.com.", 0},
+    {"IN", "MX", "10 mx.example.com.", 0},
     {"IN", "TXT", "\"test1\" \"test 2\"", 1},
-    {"IN", "RP", "root.example.com. rp-text.example.com", 0},
-    {"IN", "AFSDB", "1 afsdb.example.com", 0},
+    {"IN", "RP", "root.example.com. rp-text.example.com.", 0},
+    {"IN", "AFSDB", "1 afsdb.example.com.", 0},
     {"IN", "AAAA", "2001:db8::1", 0},
-    {"IN", "SRV", "1 0 10 target.example.com", 0},
-    {"IN", "NAPTR", "100 50 \"s\" \"http\" \"\" _http._tcp.example.com", 1},
-    {"IN", "DNAME", "dname.example.com", 0},
+    {"IN", "SRV", "1 0 10 target.example.com.", 0},
+    {"IN", "NAPTR", "100 50 \"s\" \"http\" \"\" _http._tcp.example.com.", 1},
+    {"IN", "DNAME", "dname.example.com.", 0},
     {"IN", "DS", "12892 5 2 5F0EB5C777586DE18DA6B5", 1},
     {"IN", "SSHFP", "1 1 dd465c09cfa51fb45020cc83316fff", 1},
     // We handle RRSIG separately, so it's excluded from the list
@@ -98,7 +98,7 @@ const TestRdata test_rdata_list[] = {
     {"IN", "TYPE65000", "\\# 3 010203", 1}, // some "custom" type
     {"IN", "TYPE65535", "\\# 0", 1},        // max RR type, 0-length RDATA
     {"CH", "A", "\\# 2 0102", 1}, // A RR for non-IN class; varlen data
-    {"CH", "NS", "ns.example.com", 0}, // class CH, generic data
+    {"CH", "NS", "ns.example.com.", 0}, // class CH, generic data
     {"CH", "TXT", "BIND10", 1},        // ditto
     {"HS", "A", "\\# 5 0102030405", 1}, // A RR for non-IN class; varlen data
     {NULL, NULL, NULL, 0}

src/lib/datasrc/tests/sqlite3_accessor_unittest.cc

@@ -20,6 +20,8 @@
 
 #include <dns/rrclass.h>
 
+#include <exceptions/exceptions.h>
+
 #include <sqlite3.h>
 
 #include <gtest/gtest.h>
@@ -1615,4 +1617,50 @@ TEST_F(SQLite3Update, addZoneWhileLocked) {
     EXPECT_FALSE(accessor->getZone(new_zone).first);
 }
 
+//
+// Tests for deleteZone() follow.
+//
+TEST_F(SQLite3Update, deleteZone) {
+    const std::pair<bool, int> zone_info(accessor->getZone("example.com."));
+    ASSERT_TRUE(zone_info.first);
+    zone_id = zone_info.second;
+
+    // Calling deleteZone without transaction should fail
+    EXPECT_THROW(accessor->deleteZone(zone_info.first), isc::InvalidOperation);
+
+    // Delete the zone.  Then confirm it, both before and after commit.
+    accessor->startTransaction();
+    accessor->deleteZone(zone_info.second);
+    EXPECT_FALSE(accessor->getZone("example.com.").first);
+    accessor->commit();
+    EXPECT_FALSE(accessor->getZone("example.com.").first);
+
+    // Records are not deleted.
+    std::string data[DatabaseAccessor::COLUMN_COUNT];
+    EXPECT_TRUE(accessor->getRecords("example.com.", zone_id, false)
+                ->getNext(data));
+}
+
+TEST_F(SQLite3Update, deleteZoneWhileLocked) {
+    const std::pair<bool, int> zone_info(accessor->getZone("example.com."));
+    ASSERT_TRUE(zone_info.first);
+    zone_id = zone_info.second;
+
+    // Adding another (not commit yet), it should lock the db
+    const std::string new_zone = "new.example.com.";
+    accessor->startTransaction();
+    zone_id = accessor->addZone(new_zone);
+
+    // deleteZone should throw an exception that it is locked
+    another_accessor->startTransaction();
+    EXPECT_THROW(another_accessor->deleteZone(zone_id), DataSourceError);
+    // Commit should do nothing, but not fail
+    another_accessor->commit();
+
+    accessor->rollback();
+
+    // The zone should still exist.
+    EXPECT_TRUE(accessor->getZone("example.com.").first);
+}
+
 } // end anonymous namespace