sync w/ trunk

git-svn-id: svn://bind10.isc.org/svn/bind10/branches/trac241@2663 e5f2f494-b856-4b98-b285-d166d9295462

ChangeLog

@@ -1,16 +1,189 @@
-  54.   [bug]      zhanglikun
+  80.	[bug]		jelte
+	bindctl no longer accepts configuration changes for unknown or
+	non-running modules (for the latter, this is until we have a
+	way to verify those options, at which point it'll be allowed
+	again).
+	(Trac #99, r2657)
+
+  79.	[func]		feng, jinmei
+	Refactored the ASIO link interfaces to move incoming XFR and
+	NOTIFY processing to the auth server class.  Wrapper classes for
+	ASIO specific concepts were also provided, so that other BIND 10
+	modules can (eventually) use the interface without including the
+	ASIO header file directly.  On top of these changes, AXFR and
+	NOTIFY processing was massively improved in terms of message
+	validation and protocol conformance.  Detailed tests were provided
+	to confirm the behavior.
+	Note: Right now, NOTIFY doesn't actually trigger subsequent zone
+	transfer due to security reasons. (Trac #221, r2565)
+
+  78.	[bug]		jinmei
+	lib/dns: Fixed miscellaneous bugs in the base32 (hex) and hex
+	(base16) implementation, including incorrect padding handling,
+	parser failure in decoding with a SunStudio build, missing
+	validation on the length of encoded hex string.  Test cases were
+	more detailed to identify these bugs and confirm the fix.  Also
+	renamed the incorrect term of "base32" to "base32hex".  This
+	changed the API, but they are not intended to be used outside
+	libdns++, so we don't consider it a backward incompatible change.
+	(Trac #256, r2549)
+
+  77.	[func]		zhanglikun
+	Make error message be more friendly when running cmdctl and it's 
+	already running(listening on same port)(Trac #277, r2540)
+
+  76.	[bug]		jelte
+	Fixed a bug in the handling of 'remote' config modules (i.e.
+	modules that peek at the configuration of other modules), where
+	they answered 'unknown command' to commands for those other
+	modules. (Trac #278, r2506)
+
+  75.	[bug]		jinmei
+	Fixed a bug in the sqlite3 data source where temporary strings
+	could be referenced after destruction.  It caused various lookup
+	failures with SunStudio build. (Trac #288, r2494)
+
+  74.	[func]*		jinmei
+	Refactored the cc::Session class by introducing an abstract base
+	class.  Test code can use their own derived mock class so that
+	tests can be done without establishing a real CC session.  This
+	change also modified some public APIs, mainly in the config
+	module. (Trac #275, r2459)
+
+  73.	[bug]		jelte
+  	Fixed a bug where in bindctl, locally changed settings were
+	reset when the list of running modules is updated. (Trac #285,
+	r2452)
+
+  72.	[build]		jinmei
+	Added -R when linking python wrapper modules to libpython when
+	possible.  This helps build BIND 10 on platforms that install
+	libpython whose path is unknown to run-time loader.  NetBSD is a
+	known such platform. (Trac #148, r2427)
+
+  71.  [func]		each
+  	Add "-a" (address) option to bind10 to specify an address for
+	the auth server to listen on.
+
+  70.  [func]		each
+  	Added a hot-spot cache to libdatasrc to speed up access to
+	repeatedly-queried data and reduce the number of queries to
+	the underlying database; this should substantially improve
+	performance.  Also added a "-n" ("no cache") option to
+	bind10 and b10-auth to disable the cache if needed.
+	(Trac #192, svn r2383)
+
+bind10-devel-20100701 released on July 1, 2010
+
+  69.  [func]*		jelte
+	Added python wrappers for libdns++ (isc::dns), and libxfr. This
+	removes the dependency on Boost.Python. The wrappers don't
+	completely implement all functionality, but the high-level API
+	is wrapped, and current modules use it now.
+	(Trac #181, svn r2361)
+
+  68.  [func]		zhanglikun
+	Add options -c(--certificate-chain) to bindctl. Override class
+	HTTPSConnection to support server certificate validation.
+	Add support to cmdctl.spec file, now there are three configurable 
+	items for cmdctl: 'key_file', 'cert_file' and 'accounts_file', 
+	all of them can be changed in runtime.
+	(Trac #127, svn r2357)
+
+  67.  [func]		zhanglikun
+	Make bindctl's command parser only do minimal check.
+	Parameter value can be a sequence of non-space characters,
+	or a string surrounded by quotation marks (these marks can
+	be a part of the value string in escaped form). Make error
+	message be more friendly. (If there is some error in
+	parameter's value, the parameter name will be provided).
+	Refactor function login_to_cmdctl() in class BindCmdInterpreter:
+	avoid using Exception to catch all exceptions.
+	(Trac #220, svn r2356)
+
+  66.  [bug]		each
+	Check for duplicate RRsets before inserting data into a message
+	section; this, among other things, will prevent multiple copies
+	of the same CNAME from showing up when there's a loop.  (Trac #69,
+	svn r2350)
+    
+  65.  [func]		shentingting
+	Various loadzone improvements: allow optional comment for
+	$TTL, allow optional origin and comment for $INCLUDE, allow
+	optional comment for $ORIGIN, support BIND9 extension of
+	time units for TTLs, and fix bug to not use class as part
+	of label name when records don't have a label but do have
+	a class.  Added verbose options to exactly what is happening
+	with loadzone.  Added loadzone test suite of different file
+	formats to load.
+	(Trac #197, #199, #244, #161, #198, #174, #175, svn r2340)
+
+  64.  [func]		jerry
+	Added python logging framework. It is for testing and
+	experimenting with logging ideas. Currently, it supports
+	three channels (file, syslog and stderr) and five levels
+	(debug, info, warning, error and critical).
+	(Trac #176, svn r2338)
+
+  63.  [func]		shane
+	Added initial support for setuid(), using the "-u" flag. This will
+	be replaced in the future, but for now provides a reasonable 
+	starting point.
+	(Trac #180, svn r2330)
+
+  62.  [func]		jelte
+	bin/xfrin: Use the database_file as configured in Auth to transfers
+	bin/xfrout: Use the database_file as configured in Auth to transfers
+
+  61.  [bug]		jelte
+	bin/auth: Enable b10-auth to be launched in source tree
+	(i.e. use a zone database file relative to that)
+
+  60.	[build]		jinmei
+	Supported SunStudio C++ compiler.  Note: gtest still doesn't work.
+	(Trac #251, svn r2310)
+
+  59.	[bug]		jinmei
+	lib/datasrc,bin/auth: The authoritative server could return a
+	SERVFAIL with a partial answer if it finds a data source broken
+	while looking for an answer.  This can happen, for example, if a
+	zone that doesn't have an NS RR is configured and loaded as a
+	sqlite3 data source. (Trac #249, r2286)
+
+  58.	[bug]		jinmei
+	Worked around an interaction issue between ASIO and standard C++
+	library headers.  Without this ASIO didn't work: sometimes the
+	application crashes, sometimes it blocked in the ASIO module.
+	(Trac #248, svn r2187, r2190)
+
+  57.	[func]		jinmei
+	lib/datasrc: used a simpler version of Name::split (change 31) for
+	better readability.  No behavior change. (Trac #200, svn r2159)
+
+  56.	[func]*		jinmei
+	lib/dns: renamed the library name to libdns++ to avoid confusion
+	with the same name of library of BIND 9.
+	(Trac #190, svn r2153)
+
+  55.	[bug]		shane
+	bin/xfrout: xfrout exception on Ctrl-C now no longer generates
+	exception for 'Interrupted system call'
+	(Track #136, svn r2147)
+
+  54.	[bug]		zhanglikun
+>>>>>>> .merge-right.r2662
 	bin/xfrout: Enable b10-xfrout can be launched in source
 	code tree.
 	(Trac #224, svn r2103)
 
-  53.   [bug]      zhanglikun
+  53.	[bug]		zhanglikun
 	bin/bindctl: Generate a unique session ID by using 
 	socket.gethostname() instead of socket.gethostbyname(), 
 	since the latter one could make bindctl	stall if its own 
 	host name can't be resolved.
 	(Trac #228, svn r2096)
 
-  52.   [func]      zhanglikun
+  52.	[func]		zhanglikun
 	bin/xfrout: When xfrout is launched, check whether the
 	socket file is being used by one running xfrout process, 
 	if it is, exit from python.	If the file isn't a socket file 
@@ -20,7 +193,7 @@
 
 bind10-devel-20100602 released on June 2, 2010
 
-  51.   [build]         jelte
+  51.   [build]		jelte
 	lib/python: Add bind10_config.py module for paths and
 	possibly other configure-time variables. Allow some components
 	to find spec files in build tree when ran from source.
@@ -72,6 +245,10 @@ bind10-devel-20100602 released on June 2, 2010
 	Renamed libauth to libdatasrc.
 
   38.   [bug]           zhanglikun
+	Send command 'shutdown' to Xfrin and Xfrout when boss receive SIGINT.
+	Remove unused socket file when Xfrout process exits. Make sure Xfrout
+	exit by itself when it receives SIGINT, instead of being killed by the
+	signal SIGTERM or SIGKILL sent from boss.
 	(Trac #135, #151, #134, svn r1797)
 
   37.   [build]         jinmei
@@ -132,7 +309,7 @@ bind10-devel-20100421 released on April 21, 2010
 
   24.	[func]
 	Support case-sensitive name compression in MessageRenderer.
-	(svn r1704)
+	(Trac #142, svn r1704)
 
   23.	[func]
 	Support a simple name with possible compression. (svn r1701)

Makefile.am

@@ -41,8 +41,8 @@ report-coverage:
 coverage: clean-coverage perform-coverage report-coverage
 
 #### include external sources in the distributed tarball:
-# EXTRA_DIST = ext/asio/README
-EXTRA_DIST = ext/asio/asio/local/stream_protocol.hpp
+EXTRA_DIST = ext/asio/README
+EXTRA_DIST += ext/asio/asio/local/stream_protocol.hpp
 EXTRA_DIST += ext/asio/asio/local/basic_endpoint.hpp
 EXTRA_DIST += ext/asio/asio/local/datagram_protocol.hpp
 EXTRA_DIST += ext/asio/asio/local/connect_pair.hpp

README

@@ -3,21 +3,21 @@ This is the source for the development version of BIND 10.
 BIND is the popular implementation of a DNS server, developer
 interfaces, and DNS tools. BIND 10 is a rewrite of BIND 9. BIND 10
 is written in C++ and Python and provides a modular environment
-for serving and maintaining DNS.
+for serving, maintaining, and developing DNS.
 
 BIND10-devel is new development leading up to the production
 BIND 10 release. It contains prototype code and experimental
 interfaces. Nevertheless it is ready to use now for testing the
-new BIND 10 infrastructure ideas. The Year 1 (Y1) deliverable of
-the five year plan is described here:
+new BIND 10 infrastructure ideas. The Year 2 milestones of the
+five year plan are described here:
 
-        http://bind10.isc.org/wiki/Year1Deliverable
+	https://bind10.isc.org/wiki/Year2Milestones
 
 This release includes the bind10 master process, b10-msgq message
 bus, b10-auth authoritative DNS server (with SQLite3 backend),
 b10-cmdctl remote control daemon, b10-cfgmgr configuration manager,
 b10-xfrin AXFR inbound service, b10-xfrout outgoing AXFR service,
-and a new libdns library.
+and a new libdns++ library for C++ with a python wrapper.
 
 Documentation is included and also available via the BIND 10
 website at http://bind10.isc.org/

configure.ac

@@ -2,18 +2,44 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.59])
-AC_INIT(bind10-devel, 20100602, bind10-dev@isc.org)
+AC_INIT(bind10-devel, 20100701, bind10-dev@isc.org)
 AC_CONFIG_SRCDIR(README)
 AM_INIT_AUTOMAKE
 AC_CONFIG_HEADERS([config.h])
 
 # Checks for programs.
 AC_PROG_CXX
-AC_PROG_CC
 AC_PROG_LIBTOOL
 
 # Use C++ language
-AC_LANG_CPLUSPLUS
+AC_LANG([C++])
+
+# Identify the compiler: this check must be after AC_PROG_CXX and AC_LANG.
+AM_CONDITIONAL(USE_GXX, test "X${GXX}" = "Xyes")
+AC_CHECK_DECL([__SUNPRO_CC], [SUNCXX="yes"], [SUNCXX="no"])
+
+# Linker options
+
+# check -R rather than gcc specific -rpath to be as portable as possible.
+AC_MSG_CHECKING([whether -R flag is available in linker])
+LDFLAGS_SAVED="$LDFLAGS"
+LDFLAGS="$LDFLAGS -R/usr/lib"
+AC_TRY_LINK([],[],
+	[ AC_MSG_RESULT(yes)
+		rpath_available=yes
+	],[ AC_MSG_RESULT(no)
+	rpath_available=no
+	])
+LDFLAGS=$LDFLAGS_SAVED
+
+# OS dependent compiler flags
+case "$host" in
+*-solaris*)
+	# Solaris requires special definitions to get some standard libraries
+	# (e.g. getopt(3)) available with common used header files.
+	CPPFLAGS="$CPPFLAGS -D_XPG4_2 -D__EXTENSIONS__"
+	;;
+esac
 
 m4_define([_AM_PYTHON_INTERPRETER_LIST], [python python3 python3.1])
 AC_ARG_WITH([pythonpath],
@@ -71,11 +97,29 @@ else
 		AC_MSG_WARN([${PYTHON}-config does not exist or is not executable, so we could not detect python development environment.  Your system may require an additional package (e.g. "python3-dev").  Alternatively, if you are sure you have python headers and libraries, define PYTHON_INCLUDES and PYTHON_LDFLAGS and run this script.])
 	fi
 fi
+
+# Some OSes including NetBSD don't install libpython.so in a well known path.
+# To avoid requiring dynamic library path with our python wrapper loadable
+# modules, we embed the path to the modules when possible.  We do this even
+# when the path is known in the common operational environment (e.g. when
+# it's stored in a common "hint" file) for simplicity.
+if test $rpath_available = yes; then
+	python_rpath=
+	for flag in ${PYTHON_LDFLAGS}; do
+		python_rpath="${python_rpath} `echo $flag | sed -ne 's/^\(\-L\)/-R/p'`"
+	done
+	PYTHON_LDFLAGS="${PYTHON_LDFLAGS} ${python_rpath}"
+fi
+
 AC_SUBST(PYTHON_INCLUDES)
 AC_SUBST(PYTHON_LDFLAGS)
 
-# Check for python library (not absolutely mandatory, but needed for
-# Boost.Python when we use it.  See below.)
+CPPFLAGS_SAVED="$CPPFLAGS"
+CPPFLAGS="$CPPFLAGS ${PYTHON_INCLUDES}"
+AC_CHECK_HEADERS([Python.h],, AC_MSG_ERROR([Missing Python.h]))
+CPPFLAGS="$CPPFLAGS_SAVED"
+
+# Check for python library.  Needed for Python-wrapper libraries.
 LDFLAGS_SAVED="$LDFLAGS"
 LDFLAGS="$LDFLAGS $PYTHON_LDFLAGS"
 python_bin="python${PYTHON_VERSION}"
@@ -84,11 +128,12 @@ if test $python_lib != "no"; then
 	PYTHON_LIB="-l$python_lib"
 fi
 AC_SUBST(PYTHON_LIB)
+LDFLAGS=$LDFLAGS_SAVED
 
 # TODO: check for _sqlite3.py module
 
-#
-# B10_CXXFLAGS is the default C++ compiler flags.  This will (and should) be
+# Compiler dependent settings: define some mandatory CXXFLAGS here.
+# We also use a separate variable B10_CXXFLAGS.  This will (and should) be
 # used as the default value for each specifc AM_CXXFLAGS:
 # AM_CXXFLAGS = $(B10_CXXFLAGS)
 # AM_CXXFLAGS += ... # add module specific flags
@@ -97,17 +142,24 @@ AC_SUBST(PYTHON_LIB)
 # gcc's -Wno-XXX option must be specified after -Wall or -Wextra, we cannot
 # specify the default warning flags in CXXFLAGS and let specific modules
 # "override" the default.
-#
-B10_CXXFLAGS=
 
-if test "X$GCC" = "Xyes"; then
-B10_CXXFLAGS="-g -Wall -Wextra -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare"
+CXXFLAGS=-g
+werror_ok=0
+
+# SunStudio compiler requires special compiler options for boost
+# (http://blogs.sun.com/sga/entry/boost_mini_howto)
+if test "$SUNCXX" = "yes"; then
+CXXFLAGS="$CXXFLAGS -library=stlport4 -features=tmplife -features=tmplrefstatic"
+fi
+
+# gcc specific settings:
+if test "X$GXX" = "Xyes"; then
+B10_CXXFLAGS="-Wall -Wextra -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare"
 UNUSED_PARAM_ATTRIBUTE='__attribute__((unused))'
 
 # Certain versions of gcc (g++) have a bug that incorrectly warns about
 # the use of anonymous name spaces even if they're closed in a single
 # translation unit.  For these versions we have to disable -Werror.
-werror_ok=0
 CXXFLAGS_SAVED="$CXXFLAGS"
 CXXFLAGS="$CXXFLAGS $B10_CXXFLAGS -Werror"
 AC_MSG_CHECKING(for in-TU anonymous namespace breakage)
@@ -118,13 +170,14 @@ namespace isc {class Bar {Foo foo_;};} ],,
 	 B10_CXXFLAGS="$B10_CXXFLAGS -Werror"],
 	[AC_MSG_RESULT(yes)])
 CXXFLAGS="$CXXFLAGS_SAVED"
-fi				dnl GCC = yes
+
+fi				dnl GXX = yes
 
 AM_CONDITIONAL(GCC_WERROR_OK, test $werror_ok = 1)
 AC_DEFINE_UNQUOTED(UNUSED_PARAM, $UNUSED_PARAM_ATTRIBUTE, Define to compiler keyword indicating a function argument is intentionally unused)
 
 # produce PIC unless we disable shared libraries. need this for python bindings.
-if test $enable_shared != "no" -a "X$GCC" = "Xyes"; then
+if test $enable_shared != "no" -a "X$GXX" = "Xyes"; then
    B10_CXXFLAGS="$B10_CXXFLAGS -fPIC"
 fi
 
@@ -211,54 +264,6 @@ AC_HELP_STRING([--with-boost-lib=PATH],
    fi])
 AC_SUBST(BOOST_LDFLAGS)
 
-# Check availability of the Boost Python library
-
-AC_MSG_CHECKING([for boost::python library])
-AC_ARG_WITH([boost-python],
-AC_HELP_STRING([--with-boost-python],
-  [specify whether to use the boost python library]),
-  [with_boost_python="$withval"], [with_boost_python="auto"])
-if test "$with_boost_python" != "no"; then
-	if test "$with_boost_python" != "auto" -a "X$PYTHON_LIB" = X; then
-		AC_MSG_ERROR([Boost.Python requested but python library is not available])
-	fi
-	LDFLAGS_SAVED="$LDFLAGS"
-	LIBS_SAVED="$LIBS"
-	CPPFLAGS_SAVED="$CPPFLAGS"
-	CPPFLAGS="$CPPFLAGS $PYTHON_INCLUDES"
-
-	for BOOST_TRY_LIB in boost_python boost_python-mt; do
-		LDFLAGS="$LDFLAGS_SAVED ${BOOST_LDFLAGS} ${PYTHON_LDFLAGS}"
-		LIBS="$LIBS_SAVED -l${BOOST_TRY_LIB} ${PYTHON_LIB}"
-		AC_TRY_LINK([#include <boost/python/module.hpp>
-	      using namespace boost::python;
-	      BOOST_PYTHON_MODULE(test) { throw "Boost::Python test."; }],
-			[ return 0; ],
-			[ AC_MSG_RESULT(yes)
-			BOOST_PYTHON_LIB="-l${BOOST_TRY_LIB}"
-			],[])
-		if test "X${BOOST_PYTHON_LIB}" != X; then
-        		break
-		fi
-	done
-
-	LDFLAGS="$LDFLAGS_SAVED"
-	CPPFLAGS="$CPPFLAGS_SAVED"
-	LIBS="$LIBS_SAVED"
-fi
-
-if test "X${BOOST_PYTHON_LIB}" = X; then
-	AC_MSG_RESULT(no)
-	if test "$with_boost_python" = "yes"; then
-	   AC_MSG_ERROR([boost python library is requested but not found])
-	fi
-else
-	AC_DEFINE(HAVE_BOOST_PYTHON, 1, Define to 1 if boost python library is available)
-fi
-
-AM_CONDITIONAL(HAVE_BOOST_PYTHON, test "X${BOOST_PYTHON_LIB}" != X)
-AC_SUBST(BOOST_PYTHON_LIB)
-
 #
 # Check availability of gtest, which will be used for unit tests.
 #
@@ -364,7 +369,7 @@ fi
 # run time performance.  Hpefully we can find a better solution or the ASIO
 # code will be updated by the time we really need it.
 AC_CHECK_HEADERS(sys/devpoll.h, ac_cv_have_devpoll=yes, ac_cv_have_devpoll=no)
-if test "X$ac_cv_have_devpoll" = "Xyes" -a "X$GCC" = "Xyes"; then
+if test "X$ac_cv_have_devpoll" = "Xyes" -a "X$GXX" = "Xyes"; then
 	CPPFLAGS="$CPPFLAGS -DASIO_DISABLE_DEV_POLL=1"
 fi
 
@@ -393,8 +398,11 @@ AC_CONFIG_FILES([Makefile
                  src/bin/bindctl/Makefile
                  src/bin/bindctl/tests/Makefile
                  src/bin/cfgmgr/Makefile
+                 src/bin/cfgmgr/tests/Makefile
                  src/bin/host/Makefile
                  src/bin/loadzone/Makefile
+                 src/bin/loadzone/tests/correct/Makefile
+                 src/bin/loadzone/tests/error/Makefile
                  src/bin/msgq/Makefile
                  src/bin/msgq/tests/Makefile
                  src/bin/auth/Makefile
@@ -416,19 +424,25 @@ AC_CONFIG_FILES([Makefile
                  src/lib/python/isc/cc/tests/Makefile
                  src/lib/python/isc/config/Makefile
                  src/lib/python/isc/config/tests/Makefile
+                 src/lib/python/isc/log/Makefile
+                 src/lib/python/isc/log/tests/Makefile
                  src/lib/config/Makefile
                  src/lib/config/tests/Makefile
                  src/lib/dns/Makefile
                  src/lib/dns/tests/Makefile
+                 src/lib/dns/python/Makefile
+                 src/lib/dns/python/tests/Makefile
                  src/lib/exceptions/Makefile
                  src/lib/datasrc/Makefile
                  src/lib/datasrc/tests/Makefile
                  src/lib/xfr/Makefile
                ])
 AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
+           src/bin/cfgmgr/tests/b10-cfgmgr_test.py
            src/bin/cmdctl/cmdctl.py
            src/bin/cmdctl/run_b10-cmdctl.sh
            src/bin/cmdctl/tests/cmdctl_test
+           src/bin/cmdctl/cmdctl.spec.pre
            src/bin/xfrin/tests/xfrin_test
            src/bin/xfrin/xfrin.py
            src/bin/xfrin/xfrin.spec.pre
@@ -444,6 +458,8 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            src/bin/bindctl/bindctl-source.py
            src/bin/bindctl/tests/bindctl_test
            src/bin/loadzone/run_loadzone.sh
+           src/bin/loadzone/tests/correct/correct_test.sh
+           src/bin/loadzone/tests/error/error_test.sh
            src/bin/loadzone/b10-loadzone.py
            src/bin/usermgr/run_b10-cmdctl-usermgr.sh
            src/bin/usermgr/b10-cmdctl-usermgr.py
@@ -455,6 +471,7 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            src/lib/config/tests/data_def_unittests_config.h
            src/lib/python/isc/config/tests/config_test
            src/lib/python/isc/cc/tests/cc_test
+           src/lib/python/isc/log/tests/log_test
            src/lib/dns/gen-rdatacode.py
            src/lib/python/bind10_config.py
            src/lib/dns/tests/testdata/gen-wiredata.py
@@ -470,6 +487,8 @@ AC_OUTPUT([src/bin/cfgmgr/b10-cfgmgr.py
            chmod +x src/bin/bindctl/tests/bindctl_test
            chmod +x src/bin/bindctl/run_bindctl.sh
            chmod +x src/bin/loadzone/run_loadzone.sh
+           chmod +x src/bin/loadzone/tests/correct/correct_test.sh
+           chmod +x src/bin/loadzone/tests/error/error_test.sh
            chmod +x src/bin/usermgr/run_b10-cmdctl-usermgr.sh
            chmod +x src/bin/msgq/run_msgq.sh
            chmod +x src/bin/msgq/tests/msgq_test
@@ -497,8 +516,10 @@ Flags:
   CXXFLAGS:      $CXXFLAGS
   B10_CXXFLAGS:  $B10_CXXFLAGS
 dnl includes too
-  Boost Python:  $BOOST_PYTHON_LIB
-  SQLite:	 $SQLITE_CFLAGS
+  Python:        ${PYTHON_INCLUDES}
+                 ${PYTHON_LDFLAGS}
+                 ${PYTHON_LIB}
+  SQLite:        $SQLITE_CFLAGS
                  $SQLITE_LIBS
 
 Features:

doc/Doxyfile

@@ -568,7 +568,7 @@ WARN_LOGFILE           =
 # directories like "/usr/src/myproject". Separate the files or directories
 # with spaces.
 
-INPUT                  = ../src/lib/cc ../src/lib/config ../src/lib/dns ../src/lib/exceptions ../src/lib/bench
+INPUT                  = ../src/lib/cc ../src/lib/config ../src/lib/dns ../src/lib/exceptions ../src/lib/datasrc ../src/bin/auth ../src/lib/bench
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is

doc/guide/bind10-guide.xml

@@ -76,7 +76,7 @@
 	requires SQLite 3.3.9 or newer.
         The <command>b10-xfrin</command> and <command>b10-xfrout</command>
 	modules require the libboost library,
-        Boost Python library, libpython3 library,
+        libpython3 library,
 	and the Python _sqlite3.so module.
       </para></note>
 <!-- TODO: this will change ... -->
@@ -272,7 +272,7 @@ var/
         </para>
 
         <para>
-	  The Boost Library, Boost Python Library, Python Library,
+	  The Boost Library, Python Library,
 	  and Python _sqlite3 module are required to enable the
 	  Xfrout and Xfrin support.
         </para>
@@ -284,7 +284,7 @@ var/
 
         <para>
           Building BIND 10 also requires a C++ compiler and
-          standard development headers.
+          standard development headers, make, and pkg-config.
           BIND 10 builds have been tested with GCC g++ 3.4.3, 4.1.2,
           4.1.3, 4.2.1, 4.3.2, and 4.4.1.
         </para>
@@ -495,14 +495,6 @@ var/
           </varlistentry>
 
           <varlistentry>
-            <term>--with-boost-python</term>
-            <listitem> 
-              <simpara>Define to use the Boost Python library.
-              </simpara>
-            </listitem> 
-          </varlistentry>
-
-          <varlistentry>
             <term>--with-pythonpath</term>
             <listitem> 
               <simpara>Define the path to Python 3.1 if it is not in the
@@ -527,14 +519,12 @@ var/
   <!-- TODO: lcov -->
 
         <para>
-          For example, the following configures it
-    build with Boost Python support (for Python DNS library),
+          For example, the following configures it to
     find the Boost headers and library, find the
     Python interpreter, and sets the installation location:
 
           <screen>$ <userinput>./configure --with-boost-lib=/usr/pkg/lib \
       --with-boost-include=/usr/pkg/include \
-      --with-boost-python \
       --with-pythonpath=/usr/pkg/bin/python3.1 \
       --prefix=/opt/bind10</userinput></screen>
         </para>

ext/asio/README

@@ -0,0 +1,5 @@
+ASIO library header files
+Version 1.4.5 (2010-05-12)
+Downloaded from http://sourceforge.net/projects/asio/files
+Project page: http://think-async.com/Asio
+No local modifications.

src/bin/auth/Makefile.am

@@ -1,8 +1,10 @@
 SUBDIRS = . tests
 
 AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
+AM_CPPFLAGS += -I$(top_srcdir)/src/bin -I$(top_builddir)/src/bin
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/dns -I$(top_builddir)/src/lib/dns
 AM_CPPFLAGS += -I$(top_builddir)/src/lib/cc
+AM_CPPFLAGS += -I$(top_srcdir)/src/bin -I$(top_builddir)/src/bin
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 
@@ -36,7 +38,10 @@ lib_LIBRARIES = libasio_link.a
 libasio_link_a_SOURCES = asio_link.cc asio_link.h
 # Note: the ordering matters: -Wno-... must follow -Wextra (defined in
 # B10_CXXFLAGS)
-libasio_link_a_CXXFLAGS = $(AM_CXXFLAGS) -Wno-unused-parameter
+libasio_link_a_CXXFLAGS = $(AM_CXXFLAGS)
+if USE_GXX
+libasio_link_a_CXXFLAGS += -Wno-unused-parameter
+endif
 libasio_link_a_CPPFLAGS = $(AM_CPPFLAGS)
 
 BUILT_SOURCES = spec_config.h 
@@ -45,15 +50,13 @@ b10_auth_SOURCES = auth_srv.cc auth_srv.h
 b10_auth_SOURCES += common.h
 b10_auth_SOURCES += main.cc
 b10_auth_LDADD =  $(top_builddir)/src/lib/datasrc/.libs/libdatasrc.a
-b10_auth_LDADD += $(top_builddir)/src/lib/dns/.libs/libdns.a
+b10_auth_LDADD += $(top_builddir)/src/lib/dns/.libs/libdns++.a
 b10_auth_LDADD += $(top_builddir)/src/lib/config/.libs/libcfgclient.a
-b10_auth_LDADD += $(top_builddir)/src/lib/cc/libcc.a
+b10_auth_LDADD += $(top_builddir)/src/lib/cc/.libs/libcc.a
 b10_auth_LDADD += $(top_builddir)/src/lib/exceptions/.libs/libexceptions.a
 b10_auth_LDADD += $(top_builddir)/src/bin/auth/libasio_link.a
-b10_auth_LDADD += $(SQLITE_LIBS)
-if HAVE_BOOST_PYTHON
 b10_auth_LDADD += $(top_builddir)/src/lib/xfr/.libs/libxfr.a
-endif
+b10_auth_LDADD += $(SQLITE_LIBS)
 
 # TODO: config.h.in is wrong because doesn't honor pkgdatadir
 # and can't use @datadir@ because doesn't expand default ${prefix}

src/bin/auth/asio_link.cc

@@ -16,81 +16,168 @@
 
 #include <config.h>
 
+#include <unistd.h>             // for some IPC/network system calls
+#include <sys/socket.h>
+#include <netinet/in.h>
+
 #include <asio.hpp>
+#include <boost/lexical_cast.hpp>
 #include <boost/bind.hpp>
 
+#include <boost/shared_ptr.hpp>
+
 #include <dns/buffer.h>
 #include <dns/message.h>
 #include <dns/messagerenderer.h>
 
-#if defined(HAVE_BOOST_PYTHON)
-#define USE_XFROUT
-#include <xfr/xfrout_client.h>
-#endif
-
 #include <asio_link.h>
 
-#include "spec_config.h"        // for XFROUT.  should not be here.
-#include "auth_srv.h"
+#include <auth/auth_srv.h>
+#include <auth/common.h>
 
 using namespace asio;
-using ip::udp;
-using ip::tcp;
+using asio::ip::udp;
+using asio::ip::tcp;
 
 using namespace std;
 using namespace isc::dns;
-#ifdef USE_XFROUT
-using namespace isc::xfr;
-#endif
-
-namespace {
-// As a short term workaround, we have XFROUT specific code.  We should soon
-// refactor the code with some abstraction so that we can separate this level
-// details from the (AS)IO module.
-#ifdef USE_XFROUT
-//TODO. The sample way for checking axfr query, the code should be merged to auth server class
-bool
-check_axfr_query(char* const msg_data, const uint16_t msg_len) {
-    if (msg_len < 15) {
-        return false;
-    }
 
-    const uint16_t query_type = *(uint16_t *)(msg_data + (msg_len - 4));
-    if ( query_type == 0xFC00) {
-        return true;
+namespace asio_link {
+IOAddress::IOAddress(const string& address_str)
+    // XXX: we cannot simply construct the address in the initialization list
+    // because we'd like to throw our own exception on failure.
+{
+    error_code err;
+    asio_address_ = ip::address::from_string(address_str, err);
+    if (err) {
+        isc_throw(IOError, "Failed to convert string to address '"
+                  << address_str << "': " << err.message());
     }
-    
-    return false;
 }
 
-//TODO. Send the xfr query to xfrout module, the code should be merged to auth server class
-void
-dispatch_axfr_query(const int tcp_sock, char const axfr_query[],
-                    const uint16_t query_len)
-{
-    string path;
-    if (getenv("B10_FROM_BUILD")) {
-        path = string(getenv("B10_FROM_BUILD")) + "/auth_xfrout_conn";
-    } else {
-        path = UNIX_SOCKET_FILE;
+IOAddress::IOAddress(const ip::address& asio_address) :
+    asio_address_(asio_address)
+{}
+
+string
+IOAddress::toText() const {
+    return (asio_address_.to_string());
+}
+
+// Note: this implementation is optimized for the case where this object
+// is created from an ASIO endpoint object in a receiving code path
+// by avoiding to make a copy of the base endpoint.  For TCP it may not be
+// a bug deal, but when we receive UDP packets at a high rate, the copy
+// overhead might be significant.
+class TCPEndpoint : public IOEndpoint {
+public:
+    TCPEndpoint(const IOAddress& address, const unsigned short port) :
+        asio_endpoint_placeholder_(
+            new tcp::endpoint(ip::address::from_string(address.toText()),
+                              port)),
+        asio_endpoint_(*asio_endpoint_placeholder_)
+    {}
+    TCPEndpoint(const tcp::endpoint& asio_endpoint) :
+        asio_endpoint_placeholder_(NULL), asio_endpoint_(asio_endpoint)
+    {}
+        
+    ~TCPEndpoint() { delete asio_endpoint_placeholder_; }
+    virtual IOAddress getAddress() const {
+        return (asio_endpoint_.address());
     }
-    
-    XfroutClient xfr_client(path);
-    try {
-        xfr_client.connect();
-        xfr_client.sendXfroutRequestInfo(tcp_sock, (uint8_t *)axfr_query,
-                                         query_len);
-        xfr_client.disconnect();
+private:
+    const tcp::endpoint* asio_endpoint_placeholder_;
+    const tcp::endpoint& asio_endpoint_;
+};
+
+class UDPEndpoint : public IOEndpoint {
+public:
+    UDPEndpoint(const IOAddress& address, const unsigned short port) :
+        asio_endpoint_placeholder_(
+            new udp::endpoint(ip::address::from_string(address.toText()),
+                              port)),
+        asio_endpoint_(*asio_endpoint_placeholder_)
+    {}
+    UDPEndpoint(const udp::endpoint& asio_endpoint) :
+        asio_endpoint_placeholder_(NULL), asio_endpoint_(asio_endpoint)
+    {}
+    ~UDPEndpoint() { delete asio_endpoint_placeholder_; }
+    virtual IOAddress getAddress() const {
+        return (asio_endpoint_.address());
     }
-    catch (const exception & err) {
-        //if (verbose_mode)
-        cerr << "error handle xfr query:" << err.what() << endl;
+private:
+    const udp::endpoint* asio_endpoint_placeholder_;
+    const udp::endpoint& asio_endpoint_;
+};
+
+const IOEndpoint*
+IOEndpoint::create(const int protocol, const IOAddress& address,
+                   const unsigned short port)
+{
+    if (protocol == IPPROTO_UDP) {
+        return (new UDPEndpoint(address, port));
+    } else if (protocol == IPPROTO_TCP) {
+        return (new TCPEndpoint(address, port));
     }
+    isc_throw(IOError,
+              "IOEndpoint creation attempt for unsupported protocol: " <<
+              protocol);
 }
-#endif
+
+class TCPSocket : public IOSocket {
+private:
+    TCPSocket(const TCPSocket& source);
+    TCPSocket& operator=(const TCPSocket& source);
+public:
+    TCPSocket(tcp::socket& socket) : socket_(socket) {}
+    virtual int getNative() const { return (socket_.native()); }
+    virtual int getProtocol() const { return (IPPROTO_TCP); }
+private:
+    tcp::socket& socket_;
+};
+
+class UDPSocket : public IOSocket {
+private:
+    UDPSocket(const UDPSocket& source);
+    UDPSocket& operator=(const UDPSocket& source);
+public:
+    UDPSocket(udp::socket& socket) : socket_(socket) {}
+    virtual int getNative() const { return (socket_.native()); }
+    virtual int getProtocol() const { return (IPPROTO_UDP); }
+private:
+    udp::socket& socket_;
+};
+
+class DummySocket : public IOSocket {
+private:
+    DummySocket(const DummySocket& source);
+    DummySocket& operator=(const DummySocket& source);
+public:
+    DummySocket(const int protocol) : protocol_(protocol) {}
+    virtual int getNative() const { return (-1); }
+    virtual int getProtocol() const { return (protocol_); }
+private:
+    const int protocol_;
+};
+
+IOSocket&
+IOSocket::getDummyUDPSocket() {
+    static DummySocket socket(IPPROTO_UDP);
+    return (socket);
 }
 
-namespace asio_link {
+IOSocket&
+IOSocket::getDummyTCPSocket() {
+    static DummySocket socket(IPPROTO_TCP);
+    return (socket);
+}
+
+IOMessage::IOMessage(const void* data, const size_t data_size,
+                     IOSocket& io_socket, const IOEndpoint& remote_endpoint) :
+    data_(data), data_size_(data_size), io_socket_(io_socket),
+    remote_endpoint_(remote_endpoint)
+{}
+
 //
 // Helper classes for asynchronous I/O using asio
 //
@@ -99,15 +186,18 @@ public:
     TCPClient(AuthSrv* auth_server, io_service& io_service) :
         auth_server_(auth_server),
         socket_(io_service),
+        io_socket_(socket_),
         response_buffer_(0),
         responselen_buffer_(TCP_MESSAGE_LENGTHSIZE),
         response_renderer_(response_buffer_),
-        dns_message_(Message::PARSE)
+        dns_message_(Message::PARSE),
+        custom_callback_(NULL)
     {}
 
     void start() {
         // Check for queued configuration commands
-        if (auth_server_->configSession()->hasQueuedMsgs()) {
+        if (auth_server_ != NULL &&
+            auth_server_->configSession()->hasQueuedMsgs()) {
             auth_server_->configSession()->checkCommand();
         }
         async_read(socket_, asio::buffer(data_, TCP_MESSAGE_LENGTHSIZE),
@@ -126,7 +216,6 @@ public:
 
             uint16_t msglen = dnsbuffer.readUint16();
             async_read(socket_, asio::buffer(data_, msglen),
-
                        boost::bind(&TCPClient::requestRead, this,
                                    placeholders::error,
                                    placeholders::bytes_transferred));
@@ -139,30 +228,29 @@ public:
                      size_t bytes_transferred)
     {
         if (!error) {
-            InputBuffer dnsbuffer(data_, bytes_transferred);
-#ifdef USE_XFROUT
-            if (check_axfr_query(data_, bytes_transferred)) {
-                dispatch_axfr_query(socket_.native(), data_, bytes_transferred); 
-                // start to get new query ?
+            const TCPEndpoint remote_endpoint(socket_.remote_endpoint());
+            const IOMessage io_message(data_, bytes_transferred, io_socket_,
+                                       remote_endpoint);
+            // currently, for testing purpose only
+            if (custom_callback_ != NULL) {
+                (*custom_callback_)(io_message);
                 start();
+                return;
+            }
+
+            if (auth_server_->processMessage(io_message, dns_message_,
+                                             response_renderer_)) {
+                responselen_buffer_.writeUint16(
+                    response_buffer_.getLength());
+                async_write(socket_,
+                            asio::buffer(
+                                responselen_buffer_.getData(),
+                                responselen_buffer_.getLength()),
+                            boost::bind(&TCPClient::responseWrite, this,
+                                        placeholders::error));
             } else {
-#endif          
-                if (auth_server_->processMessage(dnsbuffer, dns_message_,
-                                                response_renderer_, false)) {
-                    responselen_buffer_.writeUint16(
-                        response_buffer_.getLength());
-                    async_write(socket_,
-                                asio::buffer(
-                                    responselen_buffer_.getData(),
-                                    responselen_buffer_.getLength()),
-                                boost::bind(&TCPClient::responseWrite, this,
-                                            placeholders::error));
-                } else {
-                    delete this;
-                }
-#ifdef USE_XFROUT
+                delete this;
             }
-#endif
         } else {
             delete this;
         }
@@ -172,9 +260,9 @@ public:
         if (!error) {
                 async_write(socket_,
                             asio::buffer(response_buffer_.getData(),
-                                                response_buffer_.getLength()),
-                        boost::bind(&TCPClient::handleWrite, this,
-                                    placeholders::error));
+                                         response_buffer_.getLength()),
+                            boost::bind(&TCPClient::handleWrite, this,
+                                        placeholders::error));
         } else {
             delete this;
         }
@@ -188,9 +276,15 @@ public:
       }
     }
 
+    // Currently this is for tests only
+    void setCallBack(const IOService::IOCallBack* callback) {
+        custom_callback_ = callback;
+    }
+
 private:
     AuthSrv* auth_server_;
     tcp::socket socket_;
+    TCPSocket io_socket_;
     OutputBuffer response_buffer_;
     OutputBuffer responselen_buffer_;
     MessageRenderer response_renderer_;
@@ -198,21 +292,25 @@ private:
     enum { MAX_LENGTH = 65535 };
     static const size_t TCP_MESSAGE_LENGTHSIZE = 2;
     char data_[MAX_LENGTH];
+
+    // currently, for testing purpose only.
+    const IOService::IOCallBack* custom_callback_;
 };
 
 class TCPServer {
 public:
     TCPServer(AuthSrv* auth_server, io_service& io_service,
-              int af, short port) :
+              const ip::address& addr, const uint16_t port) :
         auth_server_(auth_server), io_service_(io_service),
         acceptor_(io_service_), listening_(new TCPClient(auth_server_,
-                                                         io_service_))
+                                                         io_service_)),
+        custom_callback_(NULL)
     {
-        tcp::endpoint endpoint(af == AF_INET6 ? tcp::v6() : tcp::v4(), port);
+        tcp::endpoint endpoint(addr, port);
         acceptor_.open(endpoint.protocol());
         // Set v6-only (we use a different instantiation for v4,
         // otherwise asio will bind to both v4 and v6
-        if (af == AF_INET6) {
+        if (addr.is_v6()) {
             acceptor_.set_option(ip::v6_only(true));
         }
         acceptor_.set_option(tcp::acceptor::reuse_address(true));
@@ -230,6 +328,7 @@ public:
     {
         if (!error) {
             assert(new_client == listening_);
+            new_client->setCallBack(custom_callback_);
             new_client->start();
             listening_ = new TCPClient(auth_server_, io_service_);
             acceptor_.async_accept(listening_->getSocket(),
@@ -241,31 +340,41 @@ public:
         }
     }
 
+    // Currently this is for tests only
+    void setCallBack(const IOService::IOCallBack* callback) {
+        custom_callback_ = callback;
+    }
+
 private:
     AuthSrv* auth_server_;
     io_service& io_service_;
     tcp::acceptor acceptor_;
     TCPClient* listening_;
+
+    // currently, for testing purpose only.
+    const IOService::IOCallBack* custom_callback_;
 };
 
 class UDPServer {
 public:
     UDPServer(AuthSrv* auth_server, io_service& io_service,
-              int af, short port) :
+              const ip::address& addr, const uint16_t port) :
         auth_server_(auth_server),
         io_service_(io_service),
-        socket_(io_service, af == AF_INET6 ? udp::v6() : udp::v4()),
+        socket_(io_service, addr.is_v6() ? udp::v6() : udp::v4()),
+        io_socket_(socket_),
         response_buffer_(0),
         response_renderer_(response_buffer_),
-        dns_message_(Message::PARSE)
+        dns_message_(Message::PARSE),
+        custom_callback_(NULL)
     {
         // Set v6-only (we use a different instantiation for v4,
         // otherwise asio will bind to both v4 and v6
-        if (af == AF_INET6) {
+        if (addr.is_v6()) {
             socket_.set_option(asio::ip::v6_only(true));
-            socket_.bind(udp::endpoint(udp::v6(), port));
+            socket_.bind(udp::endpoint(addr, port));
         } else {
-            socket_.bind(udp::endpoint(udp::v4(), port));
+            socket_.bind(udp::endpoint(addr, port));
         }
         startReceive();
     }
@@ -274,16 +383,25 @@ public:
                        size_t bytes_recvd)
     {
         // Check for queued configuration commands
-        if (auth_server_->configSession()->hasQueuedMsgs()) {
+        if (auth_server_ != NULL &&
+            auth_server_->configSession()->hasQueuedMsgs()) {
             auth_server_->configSession()->checkCommand();
         }
         if (!error && bytes_recvd > 0) {
-            InputBuffer request_buffer(data_, bytes_recvd);
+            const UDPEndpoint remote_endpoint(sender_endpoint_);
+            const IOMessage io_message(data_, bytes_recvd, io_socket_,
+                                       remote_endpoint);
+            // currently, for testing purpose only
+            if (custom_callback_ != NULL) {
+                (*custom_callback_)(io_message);
+                startReceive();
+                return;
+            }
 
             dns_message_.clear(Message::PARSE);
             response_renderer_.clear();
-            if (auth_server_->processMessage(request_buffer, dns_message_,
-                                            response_renderer_, true)) {
+            if (auth_server_->processMessage(io_message, dns_message_,
+                                             response_renderer_)) {
                 socket_.async_send_to(
                     asio::buffer(response_buffer_.getData(),
                                         response_buffer_.getLength()),
@@ -307,6 +425,11 @@ public:
         // the next request.
         startReceive();
     }
+
+    // Currently this is for tests only
+    void setCallBack(const IOService::IOCallBack* callback) {
+        custom_callback_ = callback;
+    }
 private:
     void startReceive() {
         socket_.async_receive_from(
@@ -320,86 +443,107 @@ private:
     AuthSrv* auth_server_;
     io_service& io_service_;
     udp::socket socket_;
+    UDPSocket io_socket_;
     OutputBuffer response_buffer_;
     MessageRenderer response_renderer_;
     Message dns_message_;
     udp::endpoint sender_endpoint_;
     enum { MAX_LENGTH = 4096 };
     char data_[MAX_LENGTH];
-};
 
-// This is a helper structure just to make the construction of IOServiceImpl
-// exception safe.  If the constructor of {UDP/TCP}Server throws an exception,
-// the destructor of this class will automatically perform the necessary
-// cleanup.
-struct ServerSet {
-    ServerSet() : udp4_server(NULL), udp6_server(NULL),
-                  tcp4_server(NULL), tcp6_server(NULL)
-    {}
-    ~ServerSet() {
-        delete udp4_server;
-        delete udp6_server;
-        delete tcp4_server;
-        delete tcp6_server;
-    }
-    UDPServer* udp4_server;
-    UDPServer* udp6_server;
-    TCPServer* tcp4_server;
-    TCPServer* tcp6_server;
+    // currently, for testing purpose only.
+    const IOService::IOCallBack* custom_callback_;
 };
 
 class IOServiceImpl {
 public:
-    IOServiceImpl(AuthSrv* auth_server, const char* port,
-                  const bool use_ipv4, const bool use_ipv6);
-    ~IOServiceImpl();
+    IOServiceImpl(AuthSrv* auth_server, const char& port,
+                  const ip::address* v4addr, const ip::address* v6addr);
     asio::io_service io_service_;
     AuthSrv* auth_server_;
-    UDPServer* udp4_server_;
-    UDPServer* udp6_server_;
-    TCPServer* tcp4_server_;
-    TCPServer* tcp6_server_;
+
+    typedef boost::shared_ptr<UDPServer> UDPServerPtr;
+    typedef boost::shared_ptr<TCPServer> TCPServerPtr;
+    UDPServerPtr udp4_server_;
+    UDPServerPtr udp6_server_;
+    TCPServerPtr tcp4_server_;
+    TCPServerPtr tcp6_server_;
+
+    // This member is used only for testing at the moment.
+    IOService::IOCallBack callback_;
 };
 
-IOServiceImpl::IOServiceImpl(AuthSrv* auth_server, const char* const port,
-                             const bool use_ipv4, const bool use_ipv6) :
-    auth_server_(auth_server), udp4_server_(NULL), udp6_server_(NULL),
-    tcp4_server_(NULL), tcp6_server_(NULL)
+IOServiceImpl::IOServiceImpl(AuthSrv* auth_server, const char& port,
+                             const ip::address* const v4addr,
+                             const ip::address* const v6addr) :
+    auth_server_(auth_server),
+    udp4_server_(UDPServerPtr()), udp6_server_(UDPServerPtr()),
+    tcp4_server_(TCPServerPtr()), tcp6_server_(TCPServerPtr())
 {
-    ServerSet servers;
-    short portnum = atoi(port);
-
-    if (use_ipv4) {
-        servers.udp4_server = new UDPServer(auth_server, io_service_,
-                                            AF_INET, portnum);
-        servers.tcp4_server = new TCPServer(auth_server, io_service_,
-                                            AF_INET, portnum);
-    }
-    if (use_ipv6) {
-        servers.udp6_server = new UDPServer(auth_server, io_service_,
-                                            AF_INET6, portnum);
-        servers.tcp6_server = new TCPServer(auth_server, io_service_,
-                                            AF_INET6, portnum);
+    uint16_t portnum;
+
+    try {
+        // XXX: SunStudio with stlport4 doesn't reject some invalid
+        // representation such as "-1" by lexical_cast<uint16_t>, so
+        // we convert it into a signed integer of a larger size and perform
+        // range check ourselves.
+        const int32_t portnum32 = boost::lexical_cast<int32_t>(&port);
+        if (portnum32 < 0 || portnum32 > 65535) {
+            isc_throw(IOError, "Invalid port number '" << &port);
+        }
+        portnum = portnum32;
+    } catch (const boost::bad_lexical_cast& ex) {
+        isc_throw(IOError, "Invalid port number '" << &port << "': " <<
+                  ex.what());
     }
 
-    // Now we don't have to worry about exception, and need to make sure that
-    // the server objects won't be accidentally cleaned up.
-    servers.udp4_server = NULL;
-    servers.udp6_server = NULL;
-    servers.tcp4_server = NULL;
-    servers.tcp6_server = NULL;
+    try {
+        if (v4addr != NULL) {
+            udp4_server_ = UDPServerPtr(new UDPServer(auth_server, io_service_,
+                                                      *v4addr, portnum));
+            tcp4_server_ = TCPServerPtr(new TCPServer(auth_server, io_service_,
+                                                      *v4addr, portnum));
+        }
+        if (v6addr != NULL) {
+            udp6_server_ = UDPServerPtr(new UDPServer(auth_server, io_service_,
+                                                      *v6addr, portnum));
+            tcp6_server_ = TCPServerPtr(new TCPServer(auth_server, io_service_,
+                                                      *v6addr, portnum));
+        }
+    } catch (const asio::system_error& err) {
+        // We need to catch and convert any ASIO level exceptions.
+        // This can happen for unavailable address, binding a privilege port
+        // without the privilege, etc.
+        isc_throw(IOError, "Failed to initialize network servers: " <<
+                  err.what());
+    }
 }
 
-IOServiceImpl::~IOServiceImpl() {
-    delete udp4_server_;
-    delete udp6_server_;
-    delete tcp4_server_;
-    delete tcp6_server_;
+IOService::IOService(AuthSrv* auth_server, const char& port,
+                     const char& address) :
+    impl_(NULL)
+{
+    error_code err;
+    const ip::address addr = ip::address::from_string(&address, err);
+    if (err) {
+        isc_throw(IOError, "Invalid IP address '" << &address << "': "
+                  << err.message());
+    }
+
+    impl_ = new IOServiceImpl(auth_server, port,
+                              addr.is_v4() ? &addr : NULL,
+                              addr.is_v6() ? &addr : NULL);
 }
 
-IOService::IOService(AuthSrv* auth_server, const char* const port,
-                     const bool use_ipv4, const bool use_ipv6) {
-    impl_ = new IOServiceImpl(auth_server, port, use_ipv4, use_ipv6);
+IOService::IOService(AuthSrv* auth_server, const char& port,
+                     const bool use_ipv4, const bool use_ipv6) :
+    impl_(NULL)
+{
+    const ip::address v4addr_any = ip::address(ip::address_v4::any());
+    const ip::address* const v4addrp = use_ipv4 ? &v4addr_any : NULL; 
+    const ip::address v6addr_any = ip::address(ip::address_v6::any());
+    const ip::address* const v6addrp = use_ipv6 ? &v6addr_any : NULL;
+    impl_ = new IOServiceImpl(auth_server, port, v4addrp, v6addrp);
 }
 
 IOService::~IOService() {
@@ -420,4 +564,21 @@ asio::io_service&
 IOService::get_io_service() {
     return impl_->io_service_;
 }
+
+void
+IOService::setCallBack(const IOCallBack callback) {
+    impl_->callback_ = callback;
+    if (impl_->udp4_server_ != NULL) {
+        impl_->udp4_server_->setCallBack(&impl_->callback_);
+    }
+    if (impl_->udp6_server_ != NULL) {
+        impl_->udp6_server_->setCallBack(&impl_->callback_);
+    }
+    if (impl_->tcp4_server_ != NULL) {
+        impl_->tcp4_server_->setCallBack(&impl_->callback_);
+    }
+    if (impl_->tcp6_server_ != NULL) {
+        impl_->tcp6_server_->setCallBack(&impl_->callback_);
+    }
+}
 }

src/bin/auth/asio_link.h

@@ -17,19 +17,430 @@
 #ifndef __ASIO_LINK_H
 #define __ASIO_LINK_H 1
 
+// IMPORTANT NOTE: only very few ASIO headers files can be included in
+// this file.  In particular, asio.hpp should never be included here.
+// See the description of the namespace below.
+#include <unistd.h>             // for some network system calls
+#include <asio/ip/address.hpp>
+
+#include <functional>
+#include <string>
+
+#include <boost/function.hpp>
+
+#include <exceptions/exceptions.h>
+
+namespace asio {
+// forward declaration for IOService::get_io_service() below
+class io_service;
+}
+
 class AuthSrv;
 
+/// \namespace asio_link
+/// \brief A wrapper interface for the ASIO library.
+///
+/// The \c asio_link namespace is used to define a set of wrapper interfaces
+/// for the ASIO library.
+///
+/// BIND 10 uses the non-Boost version of ASIO because it's header-only,
+/// i.e., does not require a separate library object to be linked, and thus
+/// lowers the bar for introduction.
+///
+/// But the advantage comes with its own costs: since the header-only version
+/// includes more definitions in public header files, it tends to trigger
+/// more compiler warnings for our own sources, and, depending on the
+/// compiler options, may make the build fail.
+///
+/// We also found it may be tricky to use ASIO and standard C++ libraries
+/// in a single translation unit, i.e., a .cc file: depending on the order
+/// of including header files, ASIO may or may not work on some platforms.
+///
+/// This wrapper interface is intended to centralize these
+/// problematic issues in a single sub module.  Other BIND 10 modules should
+/// simply include \c asio_link.h and use the wrapper API instead of
+/// including ASIO header files and using ASIO-specific classes directly.
+///
+/// This wrapper may be used for other IO libraries if and when we want to
+/// switch, but generality for that purpose is not the primary goal of
+/// this module.  The resulting interfaces are thus straightforward mapping
+/// to the ASIO counterparts.
+///
+/// Notes to developers:
+/// Currently the wrapper interface is specific to the authoritative
+/// server implementation.  But the plan is to generalize it and have
+/// other modules use it.
+///
+/// One obvious drawback of this approach is performance overhead
+/// due to the additional layer.  We should eventually evaluate the cost
+/// of the wrapper abstraction in benchmark tests. Another drawback is
+/// that the wrapper interfaces don't provide all features of ASIO
+/// (at least for the moment).  We should also re-evaluate the
+/// maintenance overhead of providing necessary wrappers as we develop
+/// more.
+///
+/// On the other hand, we may be able to exploit the wrapper approach to
+/// simplify the interfaces (by limiting the usage) and unify performance
+/// optimization points.
+///
+/// As for optimization, we may want to provide a custom allocator for
+/// the placeholder of callback handlers:
+/// http://think-async.com/Asio/asio-1.3.1/doc/asio/reference/asio_handler_allocate.html
+
 namespace asio_link {
 struct IOServiceImpl;
 
+/// \brief An exception that is thrown if an error occurs within the IO
+/// module.  This is mainly intended to be a wrapper exception class for
+/// ASIO specific exceptions.
+class IOError : public isc::Exception {
+public:
+    IOError(const char* file, size_t line, const char* what) :
+        isc::Exception(file, line, what) {}
+};
+
+/// \brief The \c IOAddress class represents an IP addresses (version
+/// agnostic)
+///
+/// This class is a wrapper for the ASIO \c ip::address class.
+class IOAddress {
+public:
+    ///
+    /// \name Constructors and Destructor
+    ///
+    /// This class is copyable.  We use default versions of copy constructor
+    /// and the assignment operator.
+    /// We use the default destructor.
+    //@{
+    /// \brief Constructor from string.
+    ///
+    /// This constructor converts a textual representation of IPv4 and IPv6
+    /// addresses into an IOAddress object.
+    /// If \c address_str is not a valid representation of any type of
+    /// address, an exception of class \c IOError will be thrown.
+    /// This constructor allocates memory for the object, and if that fails
+    /// a corresponding standard exception will be thrown.
+    ///
+    /// \param address_str Textual representation of address.
+    IOAddress(const std::string& address_str);
+
+    /// \brief Constructor from an ASIO \c ip::address object.
+    ///
+    /// This constructor is intended to be used within the wrapper
+    /// implementation; user applications of the wrapper API won't use it.
+    ///
+    /// This constructor never throws an exception.
+    ///
+    /// \param asio_address The ASIO \c ip::address to be converted.
+    IOAddress(const asio::ip::address& asio_adress);
+    //@}
+
+    /// \brief Convert the address to a string.
+    ///
+    /// This method is basically expected to be exception free, but
+    /// generating the string will involve resource allocation,
+    /// and if it fails the corresponding standard exception will be thrown.
+    ///
+    /// \return A string representation of the address.
+    std::string toText() const;
+private:
+    asio::ip::address asio_address_;
+};
+
+/// \brief The \c IOEndpoint class is an abstract base class to represent
+/// a communication endpoint.
+///
+/// This class is a wrapper for the ASIO endpoint classes such as
+/// \c ip::tcp::endpoint and \c ip::udp::endpoint.
+///
+/// Derived class implementations are completely hidden within the
+/// implementation.  User applications only get access to concrete
+/// \c IOEndpoint objects via the abstract interfaces.
+class IOEndpoint {
+    ///
+    /// \name Constructors and Destructor
+    ///
+    /// Note: The copy constructor and the assignment operator are
+    /// intentionally defined as private, making this class non-copyable.
+    //@{
+private:
+    IOEndpoint(const IOEndpoint& source);
+    IOEndpoint& operator=(const IOEndpoint& source);
+protected:
+    /// \brief The default constructor.
+    ///
+    /// This is intentionally defined as \c protected as this base class
+    /// should never be instantiated (except as part of a derived class).
+    IOEndpoint() {}
+public:
+    /// The destructor.
+    virtual ~IOEndpoint() {}
+    //@}
+
+    /// \brief Returns the address of the endpoint.
+    ///
+    /// This method returns an IOAddress object corresponding to \c this
+    /// endpoint.
+    /// Note that the return value is a real object, not a reference or
+    /// a pointer.
+    /// This is aligned with the interface of the ASIO counterpart:
+    /// the \c address() method of \c ip::xxx::endpoint classes returns
+    /// an \c ip::address object.
+    /// This also means handling the address of an endpoint using this method
+    /// can be expensive.  If the address information is necessary in a
+    /// performance sensitive context and there's a more efficient interface
+    /// for that purpose, it's probably better to avoid using this method.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return A copy of \c IOAddress object corresponding to the endpoint.
+    virtual IOAddress getAddress() const = 0;
+
+    /// \brief A polymorphic factory of endpoint from address and port.
+    ///
+    /// This method creates a new instance of (a derived class of)
+    /// \c IOEndpoint object that identifies the pair of given address
+    /// and port.
+    /// The appropriate derived class is chosen based on the specified
+    /// transport protocol.  If the \c protocol doesn't specify a protocol
+    /// supported in this implementation, an exception of class \c IOError
+    /// will be thrown.
+    ///
+    /// Memory for the created object will be dynamically allocated.  It's
+    /// caller's responsibility to \c delete it later.
+    /// If resource allocation for the new object fails, a corresponding
+    /// standard exception will be thrown.
+    ///
+    /// \param protocol The transport protocol used for the endpoint.
+    /// Currently, only \c IPPROTO_UDP and \c IPPROTO_TCP can be specified.
+    /// \param address The (IP) address of the endpoint.
+    /// \param port The transport port number of the endpoint
+    /// \return A pointer to a newly created \c IOEndpoint object.
+    static const IOEndpoint* create(const int protocol,
+                                    const IOAddress& address,
+                                    const unsigned short port);
+};
+
+/// \brief The \c IOSocket class is an abstract base class to represent
+/// various types of network sockets.
+///
+/// This class is a wrapper for the ASIO socket classes such as
+/// \c ip::tcp::socket and \c ip::udp::socket.
+///
+/// Derived class implementations are completely hidden within the
+/// implementation.  User applications only get access to concrete
+/// \c IOSocket objects via the abstract interfaces.
+/// We may revisit this decision when we generalize the wrapper and more
+/// modules use it.  Also, at that point we may define a separate (visible)
+/// derived class for testing purposes rather than providing factory methods
+/// (i.e., getDummy variants below).
+class IOSocket {
+    ///
+    /// \name Constructors and Destructor
+    ///
+    /// Note: The copy constructor and the assignment operator are
+    /// intentionally defined as private, making this class non-copyable.
+    //@{
+private:
+    IOSocket(const IOSocket& source);
+    IOSocket& operator=(const IOSocket& source);
+protected:
+    /// \brief The default constructor.
+    ///
+    /// This is intentionally defined as \c protected as this base class
+    /// should never be instantiated (except as part of a derived class).
+    IOSocket() {}
+public:
+    /// The destructor.
+    virtual ~IOSocket() {}
+    //@}
+
+    /// \brief Return the "native" representation of the socket.
+    ///
+    /// In practice, this is the file descriptor of the socket for
+    /// UNIX-like systems so the current implementation simply uses
+    /// \c int as the type of the return value.
+    /// We may have to need revisit this decision later.
+    ///
+    /// In general, the application should avoid using this method;
+    /// it essentially discloses an implementation specific "handle" that
+    /// can change the internal state of the socket (consider the
+    /// application closes it, for example).
+    /// But we sometimes need to perform very low-level operations that
+    /// requires the native representation.  Passing the file descriptor
+    /// to a different process is one example.
+    /// This method is provided as a necessary evil for such limited purposes.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return The native representation of the socket.  This is the socket
+    /// file descriptor for UNIX-like systems.
+    virtual int getNative() const = 0;
+
+    /// \brief Return the transport protocol of the socket.
+    ///
+    /// Currently, it returns \c IPPROTO_UDP for UDP sockets, and
+    /// \c IPPROTO_TCP for TCP sockets.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return IPPROTO_UDP for UDP sockets
+    /// \return IPPROTO_TCP for TCP sockets
+    virtual int getProtocol() const = 0;
+
+    /// \brief Return a non-usable "dummy" UDP socket for testing.
+    ///
+    /// This is a class method that returns a "mock" of UDP socket.
+    /// This is not associated with any actual socket, and its only
+    /// responsibility is to return \c IPPROTO_UDP from \c getProtocol().
+    /// The only feasible usage of this socket is for testing so that
+    /// the test code can prepare some "UDP data" even without opening any
+    /// actual socket.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \return A reference to an \c IOSocket object whose \c getProtocol()
+    /// returns \c IPPROTO_UDP.
+    static IOSocket& getDummyUDPSocket();
+
+    /// \brief Return a non-usable "dummy" TCP socket for testing.
+    ///
+    /// See \c getDummyUDPSocket().  This method is its TCP version.
+    ///
+    /// \return A reference to an \c IOSocket object whose \c getProtocol()
+    /// returns \c IPPROTO_TCP.
+    static IOSocket& getDummyTCPSocket();
+};
+
+/// \brief The \c IOMessage class encapsulates an incoming message received
+/// on a socket.
+///
+/// An \c IOMessage object represents a tuple of a chunk of data
+/// (a UDP packet or some segment of TCP stream), the socket over which the
+/// data is passed, the information about the other end point of the
+/// communication, and perhaps more.
+///
+/// The current design and interfaces of this class is tentative.
+/// It only provides a minimal level of support that is necessary for
+/// the current implementation of the authoritative server.
+/// A future version of this class will definitely support more.
+class IOMessage {
+    ///
+    /// \name Constructors and Destructor
+    ///
+    /// Note: The copy constructor and the assignment operator are
+    /// intentionally defined as private, making this class non-copyable.
+    //@{
+private:
+    IOMessage(const IOMessage& source);
+    IOMessage& operator=(const IOMessage& source);
+public:
+    /// \brief Constructor from message information.
+    ///
+    /// This constructor needs to handle the ASIO \c ip::address class,
+    /// and is intended to be used within this wrapper implementation.
+    /// Once the \c IOMessage object is created, the application can
+    /// get access to the information via the wrapper interface such as
+    /// \c getRemoteAddress().
+    ///
+    /// This constructor never throws an exception.
+    ///
+    /// \param data A pointer to the message data.
+    /// \param data_size The size of the message data in bytes.
+    /// \param io_socket The socket over which the data is given.
+    /// \param remote_endpoint The other endpoint of the socket, that is,
+    /// the sender of the message.
+    IOMessage(const void* data, const size_t data_size, IOSocket& io_socket,
+              const IOEndpoint& remote_endpoint);
+    //@}
+
+    /// \brief Returns a pointer to the received data.
+    const void* getData() const { return (data_); }
+
+    /// \brief Returns the size of the received data in bytes.
+    size_t getDataSize() const { return (data_size_); }
+
+    /// \brief Returns the socket on which the message arrives.
+    const IOSocket& getSocket() const { return (io_socket_); }
+
+    /// \brief Returns the endpoint that sends the message.
+    const IOEndpoint& getRemoteEndpoint() const { return (remote_endpoint_); }
+private:
+    const void* data_;
+    const size_t data_size_;
+    IOSocket& io_socket_;
+    const IOEndpoint& remote_endpoint_;
+};
+
+/// \brief The \c IOService class is a wrapper for the ASIO \c io_service
+/// class.
+///
+/// Currently, the interface of this class is very specific to the
+/// authoritative server implementation as indicated in the signature of
+/// the constructor, but the plan is to generalize it so that other BIND 10
+/// modules can use this interface, too.
 class IOService {
+    ///
+    /// \name Constructors and Destructor
+    ///
+    /// These are currently very specific to the authoritative server
+    /// implementation.
+    ///
+    /// Note: The copy constructor and the assignment operator are
+    /// intentionally defined as private, making this class non-copyable.
+    //@{
+private:
+    IOService(const IOService& source);
+    IOService& operator=(const IOService& source);
 public:
-    IOService(AuthSrv* auth_server, const char* port,
+    /// \brief The constructor with a specific IP address and port on which
+    /// the services listen on.
+    IOService(AuthSrv* auth_server, const char& port, const char& address);
+    /// \brief The constructor with a specific port on which the services
+    /// listen on.
+    ///
+    /// It effectively listens on "any" IPv4 and/or IPv6 addresses.
+    /// IPv4/IPv6 services will be available if and only if \c use_ipv4
+    /// or \c use_ipv6 is \c true, respectively.
+    IOService(AuthSrv* auth_server, const char& port,
               const bool use_ipv4, const bool use_ipv6);
+    /// \brief The destructor.
     ~IOService();
+    //@}
+
+    /// \brief Start the underlying event loop.
+    ///
+    /// This method does not return control to the caller until
+    /// the \c stop() method is called via some handler.
     void run();
+
+    /// \brief Stop the underlying event loop.
+    ///
+    /// This will return the control to the caller of the \c run() method.
     void stop();
+
+    /// \brief Return the native \c io_service object used in this wrapper.
+    ///
+    /// This is a short term work around to support other BIND 10 modules
+    /// that share the same \c io_service with the authoritative server.
+    /// It will eventually be removed once the wrapper interface is
+    /// generalized.
     asio::io_service& get_io_service();
+
+    /// \brief A functor(-like) class that specifies a custom call back
+    /// invoked from the event loop instead of the embedded authoritative
+    /// server callbacks.
+    ///
+    /// Currently, the callback is intended to be used only for testing
+    /// purposes.  But we'll need a generic callback type like this to
+    /// generalize the wrapper interface.
+    typedef boost::function<void(const IOMessage& io_message)> IOCallBack;
+
+    /// \brief Set the custom call back invoked from the event loop.
+    ///
+    /// Right now this method is only for testing, but will eventually be
+    /// generalized.
+    void setCallBack(IOCallBack callback);
 private:
     IOServiceImpl* impl_;
 };

src/bin/auth/auth.spec.pre.in

@@ -5,7 +5,7 @@
     "config_data": [
       { "item_name": "database_file",
         "item_type": "string",
-        "item_optional": True,
+        "item_optional": true,
         "item_default": "@@LOCALSTATEDIR@@/@PACKAGE@/zone.sqlite3"
       }
     ],

src/bin/auth/auth_srv.cc

@@ -14,6 +14,8 @@
 
 // $Id$
 
+#include <netinet/in.h>
+
 #include <algorithm>
 #include <cassert>
 #include <iostream>
@@ -40,19 +42,23 @@
 
 #include <cc/data.h>
 
-#include "common.h"
-#include "auth_srv.h"
+#include <xfr/xfrout_client.h>
 
-#include <boost/lexical_cast.hpp>
+#include <auth/common.h>
+#include <auth/auth_srv.h>
+#include <auth/asio_link.h>
 
 using namespace std;
 
 using namespace isc;
+using namespace isc::cc;
 using namespace isc::datasrc;
 using namespace isc::dns;
 using namespace isc::dns::rdata;
 using namespace isc::data;
 using namespace isc::config;
+using namespace isc::xfr;
+using namespace asio_link;
 
 class AuthSrvImpl {
 private:
@@ -60,12 +66,18 @@ private:
     AuthSrvImpl(const AuthSrvImpl& source);
     AuthSrvImpl& operator=(const AuthSrvImpl& source);
 public:
-    AuthSrvImpl();
-
+    AuthSrvImpl(const bool use_cache, AbstractXfroutClient& xfrout_client);
+    ~AuthSrvImpl();
     isc::data::ElementPtr setDbFile(const isc::data::ElementPtr config);
 
+    bool processNormalQuery(const IOMessage& io_message, Message& message,
+                            MessageRenderer& response_renderer);
+    bool processAxfrQuery(const IOMessage& io_message, Message& message,
+                            MessageRenderer& response_renderer);
+    bool processNotify(const IOMessage& io_message, Message& message, 
+                            MessageRenderer& response_renderer);
     std::string db_file_;
-    ModuleCCSession* cs_;
+    ModuleCCSession* config_session_;
     MetaDataSrc data_sources_;
     /// We keep a pointer to the currently running sqlite datasource
     /// so that we can specifically remove that one should the database
@@ -74,11 +86,24 @@ public:
 
     bool verbose_mode_;
 
+    AbstractSession* xfrin_session_;
+
+    bool xfrout_connected_;
+    AbstractXfroutClient& xfrout_client_;
+
     /// Currently non-configurable, but will be.
     static const uint16_t DEFAULT_LOCAL_UDPSIZE = 4096;
+
+    /// Hot spot cache
+    isc::datasrc::HotCache cache_;
 };
 
-AuthSrvImpl::AuthSrvImpl() : cs_(NULL), verbose_mode_(false)
+AuthSrvImpl::AuthSrvImpl(const bool use_cache,
+                         AbstractXfroutClient& xfrout_client) :
+    config_session_(NULL), verbose_mode_(false),
+    xfrin_session_(NULL),
+    xfrout_connected_(false),
+    xfrout_client_(xfrout_client)
 {
     // cur_datasrc_ is automatically initialized by the default constructor,
     // effectively being an empty (sqlite) data source.  once ccsession is up
@@ -86,11 +111,22 @@ AuthSrvImpl::AuthSrvImpl() : cs_(NULL), verbose_mode_(false)
 
     // add static data source
     data_sources_.addDataSrc(ConstDataSrcPtr(new StaticDataSrc));
+
+    // enable or disable the cache
+    cache_.setEnabled(use_cache);
 }
 
-AuthSrv::AuthSrv() : impl_(new AuthSrvImpl) {
+AuthSrvImpl::~AuthSrvImpl() {
+    if (xfrout_connected_) {
+        xfrout_client_.disconnect();
+        xfrout_connected_ = false;
+    }
 }
 
+AuthSrv::AuthSrv(const bool use_cache, AbstractXfroutClient& xfrout_client) :
+    impl_(new AuthSrvImpl(use_cache, xfrout_client))
+{}
+
 AuthSrv::~AuthSrv() {
     delete impl_;
 }
@@ -118,8 +154,9 @@ makeErrorMessage(Message& message, MessageRenderer& renderer,
     const Opcode& opcode = message.getOpcode();
     vector<QuestionPtr> questions;
 
-    // If this is an error to a query, we should also copy the question section.
-    if (opcode == Opcode::QUERY()) {
+    // If this is an error to a query or notify, we should also copy the
+    // question section.
+    if (opcode == Opcode::QUERY() || opcode == Opcode::NOTIFY()) {
         questions.assign(message.beginQuestion(), message.endQuestion());
     }
 
@@ -140,8 +177,7 @@ makeErrorMessage(Message& message, MessageRenderer& renderer,
 
     if (verbose_mode) {
         cerr << "[b10-auth] sending an error response (" <<
-            boost::lexical_cast<string>(renderer.getLength())
-             << " bytes):\n" << message.toText() << endl;
+            renderer.getLength() << " bytes):\n" << message.toText() << endl;
     }
 }
 }
@@ -157,20 +193,26 @@ AuthSrv::getVerbose() const {
 }
 
 void
-AuthSrv::setConfigSession(ModuleCCSession* cs) {
-    impl_->cs_ = cs;
+AuthSrv::setXfrinSession(AbstractSession* xfrin_session) {
+    impl_->xfrin_session_ = xfrin_session;
+}
+
+void
+AuthSrv::setConfigSession(ModuleCCSession* config_session) {
+    impl_->config_session_ = config_session;
 }
 
 ModuleCCSession*
 AuthSrv::configSession() const {
-    return (impl_->cs_);
+    return (impl_->config_session_);
 }
 
 bool
-AuthSrv::processMessage(InputBuffer& request_buffer, Message& message,
-                        MessageRenderer& response_renderer,
-                        const bool udp_buffer)
+AuthSrv::processMessage(const IOMessage& io_message, Message& message,
+                        MessageRenderer& response_renderer)
 {
+    InputBuffer request_buffer(io_message.getData(), io_message.getDataSize());
+
     // First, check the header part.  If we fail even for the base header,
     // just drop the message.
     try {
@@ -179,7 +221,8 @@ AuthSrv::processMessage(InputBuffer& request_buffer, Message& message,
         // Ignore all responses.
         if (message.getHeaderFlag(MessageFlag::QR())) {
             if (impl_->verbose_mode_) {
-                cerr << "[b10-auth] received unexpected response, ignoring" << endl;
+                cerr << "[b10-auth] received unexpected response, ignoring"
+                     << endl;
             }
             return (false);
         }
@@ -192,8 +235,8 @@ AuthSrv::processMessage(InputBuffer& request_buffer, Message& message,
         message.fromWire(request_buffer);
     } catch (const DNSProtocolError& error) {
         if (impl_->verbose_mode_) {
-            cerr << "[b10-auth] returning " <<  error.getRcode().toText() << ": "
-                 << error.what() << endl;
+            cerr << "[b10-auth] returning " <<  error.getRcode().toText()
+                 << ": " << error.what() << endl;
         }
         makeErrorMessage(message, response_renderer, error.getRcode(),
                          impl_->verbose_mode_);
@@ -213,8 +256,9 @@ AuthSrv::processMessage(InputBuffer& request_buffer, Message& message,
 
     // Perform further protocol-level validation.
 
-    // In this implementation, we only support normal queries
-    if (message.getOpcode() != Opcode::QUERY()) {
+    if (message.getOpcode() == Opcode::NOTIFY()) {
+        return (impl_->processNotify(io_message, message, response_renderer));
+    } else if (message.getOpcode() != Opcode::QUERY()) {
         if (impl_->verbose_mode_) {
             cerr << "[b10-auth] unsupported opcode" << endl;
         }
@@ -229,6 +273,25 @@ AuthSrv::processMessage(InputBuffer& request_buffer, Message& message,
         return (true);
     }
 
+    ConstQuestionPtr question = *message.beginQuestion();
+    const RRType &qtype = question->getType();
+    if (qtype == RRType::AXFR()) {
+        return (impl_->processAxfrQuery(io_message, message,
+                                        response_renderer));
+    } else if (qtype == RRType::IXFR()) {
+        makeErrorMessage(message, response_renderer, Rcode::NOTIMP(),
+                         impl_->verbose_mode_);
+        return (true);
+    } else {
+        return (impl_->processNormalQuery(io_message, message,
+                                          response_renderer));
+    }
+}
+
+bool
+AuthSrvImpl::processNormalQuery(const IOMessage& io_message, Message& message,
+                                MessageRenderer& response_renderer)
+{
     const bool dnssec_ok = message.isDNSSECSupported();
     const uint16_t remote_bufsize = message.getUDPSize();
 
@@ -239,28 +302,162 @@ AuthSrv::processMessage(InputBuffer& request_buffer, Message& message,
     message.setUDPSize(AuthSrvImpl::DEFAULT_LOCAL_UDPSIZE);
 
     try {
-        Query query(message, dnssec_ok);
-        impl_->data_sources_.doQuery(query);
+        Query query(message, cache_, dnssec_ok);
+        data_sources_.doQuery(query);
     } catch (const Exception& ex) {
-        if (impl_->verbose_mode_) {
-            cerr << "[b10-auth] Internal error, returning SERVFAIL: " << ex.what() << endl;
+        if (verbose_mode_) {
+            cerr << "[b10-auth] Internal error, returning SERVFAIL: " <<
+                ex.what() << endl;
         }
         makeErrorMessage(message, response_renderer, Rcode::SERVFAIL(),
-                         impl_->verbose_mode_);
+                         verbose_mode_);
         return (true);
     }
 
+    const bool udp_buffer =
+        (io_message.getSocket().getProtocol() == IPPROTO_UDP);
     response_renderer.setLengthLimit(udp_buffer ? remote_bufsize : 65535);
     message.toWire(response_renderer);
-    if (impl_->verbose_mode_) {
-        cerr << "[b10-auth] sending a response (" <<
-            boost::lexical_cast<string>(response_renderer.getLength())
+    if (verbose_mode_) {
+        cerr << "[b10-auth] sending a response ("
+             << response_renderer.getLength()
              << " bytes):\n" << message.toText() << endl;
     }
 
     return (true);
 }
 
+
+bool
+AuthSrvImpl::processAxfrQuery(const IOMessage& io_message, Message& message,
+                            MessageRenderer& response_renderer)
+{
+    if (io_message.getSocket().getProtocol() == IPPROTO_UDP) {
+        if (verbose_mode_) {
+            cerr << "[b10-auth] AXFR query over UDP isn't allowed" << endl;
+        }
+        makeErrorMessage(message, response_renderer, Rcode::FORMERR(),
+                         verbose_mode_);
+        return (true);
+    }
+
+    try {
+        if (!xfrout_connected_) {
+            xfrout_client_.connect();
+            xfrout_connected_ = true;
+        }
+        xfrout_client_.sendXfroutRequestInfo(
+            io_message.getSocket().getNative(),
+            io_message.getData(),
+            io_message.getDataSize());
+    } catch (const XfroutError& err) {
+        if (xfrout_connected_) {
+            // disconnect() may trigger an exception, but since we try it
+            // only if we've successfully opened it, it shouldn't happen in
+            // normal condition.  Should this occur, we'll propagate it to the
+            // upper layer.
+            xfrout_client_.disconnect();
+            xfrout_connected_ = false;
+        }
+        
+        if (verbose_mode_) {
+            cerr << "[b10-auth] Error in handling XFR request: " << err.what()
+                 << endl;
+        }
+        makeErrorMessage(message, response_renderer, Rcode::SERVFAIL(),
+                         verbose_mode_);
+        return (true);
+    }
+    return (false);
+}
+
+bool
+AuthSrvImpl::processNotify(const IOMessage& io_message, Message& message, 
+                           MessageRenderer& response_renderer) 
+{
+    // The incoming notify must contain exactly one question for SOA of the
+    // zone name.
+    if (message.getRRCount(Section::QUESTION()) != 1) {
+        if (verbose_mode_) {
+                cerr << "[b10-auth] invalid number of questions in notify: "
+                     << message.getRRCount(Section::QUESTION()) << endl;
+        }
+        makeErrorMessage(message, response_renderer, Rcode::FORMERR(),
+                         verbose_mode_);
+        return (true);
+    }
+    ConstQuestionPtr question = *message.beginQuestion();
+    if (question->getType() != RRType::SOA()) {
+        if (verbose_mode_) {
+                cerr << "[b10-auth] invalid question RR type in notify: "
+                     << question->getType() << endl;
+        }
+        makeErrorMessage(message, response_renderer, Rcode::FORMERR(),
+                         verbose_mode_);
+        return (true);
+    }
+
+    // According to RFC 1996, rcode should be "no error" and AA bit should be
+    // on, but we don't check these conditions.  This behavior is compatible
+    // with BIND 9.
+
+    // TODO check with the conf-mgr whether current server is the auth of the
+    // zone
+
+    // In the code that follows, we simply ignore the notify if any internal
+    // error happens rather than returning (e.g.) SERVFAIL.  RFC 1996 is
+    // silent about such cases, but there doesn't seem to be anything we can
+    // improve at the primary server side by sending an error anyway.
+    if (xfrin_session_ == NULL) {
+        if (verbose_mode_) {
+            cerr << "[b10-auth] "
+                "session interface for xfrin is not available" << endl;
+        }
+        return (false);
+    }
+    
+    const string remote_ip_address =
+        io_message.getRemoteEndpoint().getAddress().toText();
+    static const string command_template_start =
+        "{\"command\": [\"notify\", {\"zone_name\" : \"";
+    static const string command_template_master = "\", \"master\" : \"";
+    static const string command_template_rrclass = "\", \"rrclass\" : \"";
+    static const string command_template_end = "\"}]}";
+
+    try {
+        ElementPtr notify_command = Element::fromJSON(
+                command_template_start + question->getName().toText() + 
+                command_template_master + remote_ip_address +
+                command_template_rrclass + question->getClass().toText() +
+                command_template_end);
+        const unsigned int seq =
+            xfrin_session_->group_sendmsg(notify_command, "Xfrin",
+                                          "*", "*");
+        ElementPtr env, answer, parsed_answer;
+        xfrin_session_->group_recvmsg(env, answer, false, seq);
+        int rcode;
+        parsed_answer = parseAnswer(rcode, answer);
+        if (rcode != 0) {
+            if (verbose_mode_) {
+                cerr << "[b10-auth] failed to notify Xfrin: "
+                     << parsed_answer->str() << endl; 
+            }
+            return (false);
+        }
+    } catch (const Exception& ex) {
+        if (verbose_mode_) {
+            cerr << "[b10-auth] failed to notify Xfrin: " << ex.what() << endl;
+        }
+        return (false);
+    }
+
+    message.makeResponse();
+    message.setHeaderFlag(MessageFlag::AA());
+    message.setRcode(Rcode::NOERROR());
+    message.toWire(response_renderer);
+    return (true);
+}
+
 ElementPtr
 AuthSrvImpl::setDbFile(const isc::data::ElementPtr config) {
     ElementPtr answer = isc::config::createAnswer();
@@ -269,13 +466,26 @@ AuthSrvImpl::setDbFile(const isc::data::ElementPtr config) {
     if (config && config->contains("database_file")) {
         db_file_ = config->get("database_file")->stringValue();
         final = config;
-    } else if (cs_ != NULL) {
+    } else if (config_session_ != NULL) {
         bool is_default;
         string item("database_file");
-        ElementPtr value = cs_->getValue(is_default, item);
-        db_file_ = value->stringValue();
-        final = Element::createFromString("{}");
+        ElementPtr value = config_session_->getValue(is_default, item);
+        final = Element::createMap();
+
+        // If the value is the default, and we are running from
+        // a specific directory ('from build'), we need to use
+        // a different value than the default (which may not exist)
+        // (btw, this should not be done here in the end, i think
+        //  the from-source script should have a check for this,
+        //  but for that we need offline access to config, so for
+        //  now this is a decent solution)
+        if (is_default && getenv("B10_FROM_BUILD")) {
+            value = Element::create(string(getenv("B10_FROM_BUILD")) +
+                                    "/bind10_zones.sqlite3");
+        }
         final->set(item, value);
+
+        db_file_ = value->stringValue();
     } else {
         return (answer);
     }

src/bin/auth/auth_srv.h

@@ -28,6 +28,14 @@ class InputBuffer;
 class Message;
 class MessageRenderer;
 }
+
+namespace xfr {
+class AbstractXfroutClient;
+};
+}
+
+namespace asio_link {
+class IOMessage;
 }
 
 class AuthSrvImpl;
@@ -36,28 +44,49 @@ class AuthSrv {
     ///
     /// \name Constructors, Assignment Operator and Destructor.
     ///
-    /// Note: The copy constructor and the assignment operator are intentionally
-    /// defined as private.
+    /// Note: The copy constructor and the assignment operator are
+    /// intentionally defined as private.
     //@{
 private:
     AuthSrv(const AuthSrv& source);
     AuthSrv& operator=(const AuthSrv& source);
 public:
-    explicit AuthSrv();
+    /// The constructor.
+    ///
+    /// \param use_cache Whether to enable hot spot cache for lookup results.
+    /// \param xfrout_client Communication interface with a separate xfrout
+    /// process.  It's normally a reference to an xfr::XfroutClient object,
+    /// but can refer to a local mock object for testing (or other
+    /// experimental) purposes.
+    AuthSrv(const bool use_cache,
+            isc::xfr::AbstractXfroutClient& xfrout_client);
     ~AuthSrv();
     //@}
     /// \return \c true if the \message contains a response to be returned;
     /// otherwise \c false.
-    bool processMessage(isc::dns::InputBuffer& request_buffer,
+    bool processMessage(const asio_link::IOMessage& io_message,
                         isc::dns::Message& message,
-                        isc::dns::MessageRenderer& response_renderer,
-                        bool udp_buffer);
+                        isc::dns::MessageRenderer& response_renderer);
     void setVerbose(bool on);
     bool getVerbose() const;
-    void serve(std::string zone_name);
     isc::data::ElementPtr updateConfig(isc::data::ElementPtr config);
     isc::config::ModuleCCSession* configSession() const;
-    void setConfigSession(isc::config::ModuleCCSession* cs);
+    void setConfigSession(isc::config::ModuleCCSession* config_session);
+
+    ///
+    /// Note: this interface is tentative.  We'll revisit the ASIO and session
+    /// frameworks, at which point the session will probably be passed on
+    /// construction of the server.
+    ///
+    /// \param xfrin_session A Session object over which NOTIFY message
+    /// information is exchanged with a XFRIN handler.
+    /// The session must be established before setting in the server
+    /// object.
+    /// Ownership isn't transferred: the caller is responsible for keeping
+    /// this object to be valid while the server object is working and for
+    /// disconnecting the session and destroying the object when the server
+    ///
+    void setXfrinSession(isc::cc::AbstractSession* xfrin_session);
 private:
     AuthSrvImpl* impl_;
 };

src/bin/auth/main.cc

@@ -14,8 +14,6 @@
 
 // $Id$
 
-#include "config.h"
-
 #include <sys/types.h>
 #include <sys/socket.h>
 #include <sys/select.h>
@@ -39,16 +37,19 @@
 #include <cc/data.h>
 #include <config/ccsession.h>
 
-#include "spec_config.h"
-#include "common.h"
-#include "auth_srv.h"
-#include "asio_link.h"
+#include <xfr/xfrout_client.h>
+
+#include <auth/spec_config.h>
+#include <auth/common.h>
+#include <auth/auth_srv.h>
+#include <auth/asio_link.h>
 
 using namespace std;
 using namespace isc::data;
 using namespace isc::cc;
 using namespace isc::config;
 using namespace isc::dns;
+using namespace isc::xfr;
 
 namespace {
 
@@ -86,7 +87,7 @@ my_command_handler(const string& command, const ElementPtr args) {
 
 void
 usage() {
-    cerr << "Usage: b10-auth [-p port] [-4|-6]" << endl;
+    cerr << "Usage: b10-auth [-a address] [-p port] [-4|-6] [-nv]" << endl;
     exit(1);
 }
 } // end of anonymous namespace
@@ -95,9 +96,10 @@ int
 main(int argc, char* argv[]) {
     int ch;
     const char* port = DNSPORT;
-    bool use_ipv4 = true, use_ipv6 = true;
+    const char* address = NULL;
+    bool use_ipv4 = true, use_ipv6 = true, cache = true;
 
-    while ((ch = getopt(argc, argv, "46p:v")) != -1) {
+    while ((ch = getopt(argc, argv, "46a:np:v")) != -1) {
         switch (ch) {
         case '4':
             // Note that -4 means "ipv4 only", we need to set "use_ipv6" here,
@@ -110,6 +112,12 @@ main(int argc, char* argv[]) {
             // The same note as -4 applies.
             use_ipv4 = false;
             break;
+        case 'n':
+            cache = false;
+            break;
+        case 'a':
+            address = optarg;
+            break;
         case 'p':
             port = optarg;
             break;
@@ -131,8 +139,19 @@ main(int argc, char* argv[]) {
         usage();
     }
 
-    // initialize command channel
+    if ((!use_ipv4 || !use_ipv6) && address != NULL) {
+        cerr << "[b10-auth] Error: -4|-6 and -a can't coexist" << endl;
+        usage();
+    }
+
     int ret = 0;
+
+    // XXX: we should eventually pass io_service here.
+    Session* cc_session = NULL;
+    Session* xfrin_session = NULL;
+    bool xfrin_session_established = false; // XXX (see Trac #287)
+    ModuleCCSession* config_session = NULL;
+    XfroutClient xfrout_client(UNIX_SOCKET_FILE);
     try {
         string specfile;
         if (getenv("B10_FROM_BUILD")) {
@@ -142,26 +161,64 @@ main(int argc, char* argv[]) {
             specfile = string(AUTH_SPECFILE_LOCATION);
         }
 
-        auth_server = new AuthSrv;
+        auth_server = new AuthSrv(cache, xfrout_client);
         auth_server->setVerbose(verbose_mode);
-
-        io_service = new asio_link::IOService(auth_server, port, use_ipv4,
-                                              use_ipv6);
-
-        ModuleCCSession cs(specfile, io_service->get_io_service(), my_config_handler, my_command_handler);
-
-        auth_server->setConfigSession(&cs);
+        cout << "[b10-auth] Server created." << endl;
+
+        if (address != NULL) {
+            // XXX: we can only specify at most one explicit address.
+            // This also means the server cannot run in the dual address
+            // family mode if explicit addresses need to be specified.
+            // We don't bother to fix this problem, however.  The -a option
+            // is a short term workaround until we support dynamic listening
+            // port allocation.
+            io_service = new asio_link::IOService(auth_server, *port,
+                                                  *address);
+        } else {
+            io_service = new asio_link::IOService(auth_server, *port,
+                                                  use_ipv4, use_ipv6);
+        }
+        cout << "[b10-auth] IOService created." << endl;
+
+        cc_session = new Session(io_service->get_io_service());
+        cout << "[b10-auth] Configuration session channel created." << endl;
+
+        config_session = new ModuleCCSession(specfile, *cc_session,
+                                             my_config_handler,
+                                             my_command_handler);
+        cout << "[b10-auth] Configuration channel established." << endl;
+
+        xfrin_session = new Session(io_service->get_io_service());
+        cout << "[b10-auth] Xfrin session channel created." << endl;
+        xfrin_session->establish(NULL);
+        xfrin_session_established = true;
+        cout << "[b10-auth] Xfrin session channel established." << endl;
+
+        // XXX: with the current interface to asio_link we have to create
+        // auth_server before io_service while Session needs io_service.
+        // In a next step of refactoring we should make asio_link independent
+        // from auth_server, and create io_service, auth_server, and
+        // sessions in that order.
+        auth_server->setXfrinSession(xfrin_session);
+        auth_server->setConfigSession(config_session);
         auth_server->updateConfig(ElementPtr());
 
-        
         cout << "[b10-auth] Server started." << endl;
         io_service->run();
     } catch (const std::exception& ex) {
-        cerr << "[b10-auth] " << ex.what() << endl;
+        cerr << "[b10-auth] Initialization failed: " << ex.what() << endl;
         ret = 1;
     }
 
+    if (xfrin_session_established) {
+        xfrin_session->disconnect();
+    }
+
+    delete xfrin_session;
+    delete config_session;
+    delete cc_session;
     delete io_service;
     delete auth_server;
+
     return (ret);
 }

src/bin/auth/tests/Makefile.am

@@ -14,16 +14,19 @@ run_unittests_SOURCES = $(top_srcdir)/src/lib/dns/tests/unittest_util.h
 run_unittests_SOURCES += $(top_srcdir)/src/lib/dns/tests/unittest_util.cc
 run_unittests_SOURCES += ../auth_srv.h ../auth_srv.cc
 run_unittests_SOURCES += auth_srv_unittest.cc
+run_unittests_SOURCES += asio_link_unittest.cc
 run_unittests_SOURCES += run_unittests.cc
 run_unittests_CPPFLAGS = $(AM_CPPFLAGS) $(GTEST_INCLUDES)
 run_unittests_LDFLAGS = $(AM_LDFLAGS) $(GTEST_LDFLAGS)
 run_unittests_LDADD = $(GTEST_LDADD)
 run_unittests_LDADD += $(SQLITE_LIBS)
 run_unittests_LDADD +=  $(top_builddir)/src/lib/datasrc/.libs/libdatasrc.a
-run_unittests_LDADD +=  $(top_builddir)/src/lib/dns/.libs/libdns.a
+run_unittests_LDADD +=  $(top_builddir)/src/lib/dns/.libs/libdns++.a
 run_unittests_LDADD += $(top_builddir)/src/lib/config/.libs/libcfgclient.a
-run_unittests_LDADD += $(top_builddir)/src/lib/cc/libcc.a
+run_unittests_LDADD += $(top_builddir)/src/lib/cc/.libs/libcc.a
 run_unittests_LDADD += $(top_builddir)/src/lib/exceptions/.libs/libexceptions.a
+run_unittests_LDADD += $(top_builddir)/src/bin/auth/libasio_link.a
+run_unittests_LDADD += $(top_builddir)/src/lib/xfr/.libs/libxfr.a
 endif
 
 noinst_PROGRAMS = $(TESTS)

src/bin/auth/tests/asio_link_unittest.cc

@@ -0,0 +1,357 @@
+// Copyright (C) 2010  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+// $Id$
+
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netdb.h>
+
+#include <stdint.h>
+
+#include <functional>
+#include <string>
+#include <vector>
+
+#include <gtest/gtest.h>
+
+#include <exceptions/exceptions.h>
+
+#include <dns/tests/unittest_util.h>
+
+#include <auth/asio_link.h>
+
+using isc::UnitTestUtil;
+using namespace std;
+using namespace asio_link;
+
+namespace {
+const char* const TEST_PORT = "53535";
+const char* const TEST_IPV6_ADDR = "::1";
+const char* const TEST_IPV4_ADDR = "127.0.0.1";
+// This data is intended to be valid as a DNS/TCP-like message: the first
+// two octets encode the length of the rest of the data.  This is crucial
+// for the tests below.
+const uint8_t test_data[] = {0, 4, 1, 2, 3, 4};
+
+TEST(IOAddressTest, fromText) {
+    IOAddress io_address_v4("192.0.2.1");
+    EXPECT_EQ("192.0.2.1", io_address_v4.toText());
+
+    IOAddress io_address_v6("2001:db8::1234");
+    EXPECT_EQ("2001:db8::1234", io_address_v6.toText());
+
+    // bogus IPv4 address-like input
+    EXPECT_THROW(IOAddress("192.0.2.2.1"), IOError);
+
+    // bogus IPv6 address-like input
+    EXPECT_THROW(IOAddress("2001:db8:::1234"), IOError);
+}
+
+TEST(IOEndpointTest, create) {
+    const IOEndpoint* ep;
+    ep = IOEndpoint::create(IPPROTO_UDP, IOAddress("192.0.2.1"), 5300);
+    EXPECT_EQ("192.0.2.1", ep->getAddress().toText());
+    delete ep;
+
+    ep = IOEndpoint::create(IPPROTO_TCP, IOAddress("192.0.2.1"), 5300);
+    EXPECT_EQ("192.0.2.1", ep->getAddress().toText());
+    delete ep;
+
+    ep = IOEndpoint::create(IPPROTO_UDP, IOAddress("2001:db8::1234"), 5300);
+    EXPECT_EQ("2001:db8::1234", ep->getAddress().toText());
+    delete ep;
+
+    ep = IOEndpoint::create(IPPROTO_TCP, IOAddress("2001:db8::1234"), 5300);
+    EXPECT_EQ("2001:db8::1234", ep->getAddress().toText());
+    delete ep;
+
+    EXPECT_THROW(IOEndpoint::create(IPPROTO_IP, IOAddress("192.0.2.1"),
+                                    5300)->getAddress().toText(),
+                 IOError);
+}
+
+TEST(IOSocketTest, dummySockets) {
+    EXPECT_EQ(IPPROTO_UDP, IOSocket::getDummyUDPSocket().getProtocol());
+    EXPECT_EQ(IPPROTO_TCP, IOSocket::getDummyTCPSocket().getProtocol());
+    EXPECT_EQ(-1, IOSocket::getDummyUDPSocket().getNative());
+    EXPECT_EQ(-1, IOSocket::getDummyTCPSocket().getNative());
+}
+
+TEST(IOServiceTest, badPort) {
+    EXPECT_THROW(IOService(NULL, *"65536", true, false), IOError);
+    EXPECT_THROW(IOService(NULL, *"5300.0", true, false), IOError);
+    EXPECT_THROW(IOService(NULL, *"-1", true, false), IOError);
+    EXPECT_THROW(IOService(NULL, *"domain", true, false), IOError);
+}
+
+TEST(IOServiceTest, badAddress) {
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *"192.0.2.1.1"),
+                 IOError);
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *"2001:db8:::1"),
+                 IOError);
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *"localhost"),
+                 IOError);
+}
+
+TEST(IOServiceTest, unavailableAddress) {
+    // These addresses should generally be unavailable as a valid local
+    // address, although there's no guarantee in theory.
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *"255.255.0.0"), IOError);
+
+    // Some OSes would simply reject binding attempt for an AF_INET6 socket
+    // to an IPv4-mapped IPv6 address.  Even if those that allow it, since
+    // the corresponding IPv4 address is the same as the one used in the
+    // AF_INET socket case above, it should at least show the same result
+    // as the previous one.
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *"::ffff:255.255.0.0"), IOError);
+}
+
+TEST(IOServiceTest, duplicateBind) {
+    // In each sub test case, second attempt should fail due to duplicate bind
+
+    // IPv6, "any" address
+    IOService* io_service = new IOService(NULL, *TEST_PORT, false, true);
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, false, true), IOError);
+    delete io_service;
+
+    // IPv6, specific address
+    io_service = new IOService(NULL, *TEST_PORT, *TEST_IPV6_ADDR);
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *TEST_IPV6_ADDR), IOError);
+    delete io_service;
+
+    // IPv4, "any" address
+    io_service = new IOService(NULL, *TEST_PORT, true, false);
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, true, false), IOError);
+    delete io_service;
+
+    // IPv4, specific address
+    io_service = new IOService(NULL, *TEST_PORT, *TEST_IPV4_ADDR);
+    EXPECT_THROW(IOService(NULL, *TEST_PORT, *TEST_IPV4_ADDR), IOError);
+    delete io_service;
+}
+
+struct addrinfo*
+resolveAddress(const int family, const int sock_type, const int protocol) {
+    const char* const addr = (family == AF_INET6) ?
+        TEST_IPV6_ADDR : TEST_IPV4_ADDR;
+
+    struct addrinfo hints;
+    memset(&hints, 0, sizeof(hints));
+    hints.ai_family = family;
+    hints.ai_socktype = sock_type;
+    hints.ai_protocol = protocol;
+
+    struct addrinfo* res;
+    const int error = getaddrinfo(addr, TEST_PORT, &hints, &res);
+    if (error != 0) {
+        isc_throw(IOError, "getaddrinfo failed: " << gai_strerror(error));
+    }
+
+    return (res);
+}
+
+// This fixture is a framework for various types of network operations
+// using the ASIO interfaces.  Each test case creates an IOService object,
+// opens a local "client" socket for testing, sends data via the local socket
+// to the service that would run in the IOService object.
+// A mock callback function (an ASIOCallBack object) is registered with the
+// IOService object, so the test code should be able to examine the data
+// receives on the server side.  It then checks the received data matches
+// expected parameters.
+// If initialization parameters of the IOService should be modified, the test
+// case can do it using the setIOService() method.
+// Note: the set of tests in ASIOLinkTest use actual network services and may
+// involve undesirable side effect such as blocking.
+class ASIOLinkTest : public ::testing::Test {
+protected:
+    ASIOLinkTest();
+    ~ASIOLinkTest() {
+        if (res_ != NULL) {
+            freeaddrinfo(res_);
+        }
+        if (sock_ != -1) {
+            close(sock_);
+        }
+        delete io_service_;
+    }
+    void sendUDP(const int family) {
+        res_ = resolveAddress(family, SOCK_DGRAM, IPPROTO_UDP);
+
+        sock_ = socket(res_->ai_family, res_->ai_socktype, res_->ai_protocol);
+        if (sock_ < 0) {
+            isc_throw(IOError, "failed to open test socket");
+        }
+        const int cc = sendto(sock_, test_data, sizeof(test_data), 0,
+                              res_->ai_addr, res_->ai_addrlen);
+        if (cc != sizeof(test_data)) {
+            isc_throw(IOError, "unexpected sendto result: " << cc);
+        }
+        io_service_->run();
+    }
+    void sendTCP(const int family) {
+        res_ = resolveAddress(family, SOCK_STREAM, IPPROTO_TCP);
+
+        sock_ = socket(res_->ai_family, res_->ai_socktype, res_->ai_protocol);
+        if (sock_ < 0) {
+            isc_throw(IOError, "failed to open test socket");
+        }
+        if (connect(sock_, res_->ai_addr, res_->ai_addrlen) < 0) {
+            isc_throw(IOError, "failed to connect to the test server");
+        }
+        const int cc = send(sock_, test_data, sizeof(test_data), 0);
+        if (cc != sizeof(test_data)) {
+            isc_throw(IOError, "unexpected sendto result: " << cc);
+        }
+        io_service_->run();
+    }
+    void setIOService(const char& address) {
+        delete io_service_;
+        io_service_ = NULL;
+        io_service_ = new IOService(NULL, *TEST_PORT, address);
+        io_service_->setCallBack(ASIOCallBack(this));
+    }
+    void setIOService(const bool use_ipv4, const bool use_ipv6) {
+        delete io_service_;
+        io_service_ = NULL;
+        io_service_ = new IOService(NULL, *TEST_PORT, use_ipv4, use_ipv6);
+        io_service_->setCallBack(ASIOCallBack(this));
+    }
+    void doTest(const int family, const int protocol) {
+        if (protocol == IPPROTO_UDP) {
+            sendUDP(family);
+        } else {
+            sendTCP(family);
+        }
+
+        // There doesn't seem to be an effective test for the validity of
+        // 'native'.
+        // One thing we are sure is it must be different from our local socket.
+        EXPECT_NE(sock_, callback_native_);
+        EXPECT_EQ(protocol, callback_protocol_);
+        EXPECT_EQ(family == AF_INET6 ? TEST_IPV6_ADDR : TEST_IPV4_ADDR,
+                  callback_address_);
+
+        const uint8_t* expected_data =
+            protocol == IPPROTO_UDP ? test_data : test_data + 2;
+        const size_t expected_datasize =
+            protocol == IPPROTO_UDP ? sizeof(test_data) :
+            sizeof(test_data) - 2;
+        EXPECT_PRED_FORMAT4(UnitTestUtil::matchWireData, &callback_data_[0],
+                            callback_data_.size(),
+                            expected_data, expected_datasize);
+    }
+private:
+    class ASIOCallBack : public std::unary_function<IOMessage, void> {
+    public:
+        ASIOCallBack(ASIOLinkTest* test_obj) : test_obj_(test_obj) {}
+        void operator()(const IOMessage& io_message) const {
+            test_obj_->callBack(io_message);
+        }
+    private:
+        ASIOLinkTest* test_obj_;
+    };
+    void callBack(const IOMessage& io_message) {
+        callback_protocol_ = io_message.getSocket().getProtocol();
+        callback_native_ = io_message.getSocket().getNative();
+        callback_address_ =
+            io_message.getRemoteEndpoint().getAddress().toText();
+        callback_data_.assign(
+            static_cast<const uint8_t*>(io_message.getData()),
+            static_cast<const uint8_t*>(io_message.getData()) +
+            io_message.getDataSize());
+        io_service_->stop();
+    }
+protected:
+    IOService* io_service_;
+    int callback_protocol_;
+    int callback_native_;
+    string callback_address_;
+    vector<uint8_t> callback_data_;
+    int sock_;
+private:
+    struct addrinfo* res_;
+};
+
+ASIOLinkTest::ASIOLinkTest() :
+    io_service_(NULL), sock_(-1), res_(NULL)
+{
+    setIOService(true, true);
+}
+
+TEST_F(ASIOLinkTest, v6UDPSend) {
+    doTest(AF_INET6, IPPROTO_UDP);
+}
+
+TEST_F(ASIOLinkTest, v6TCPSend) {
+    doTest(AF_INET6, IPPROTO_TCP);
+}
+
+TEST_F(ASIOLinkTest, v4UDPSend) {
+    doTest(AF_INET, IPPROTO_UDP);
+}
+
+TEST_F(ASIOLinkTest, v4TCPSend) {
+    doTest(AF_INET, IPPROTO_TCP);
+}
+
+TEST_F(ASIOLinkTest, v6UDPSendSpecific) {
+    // Explicitly set a specific address to be bound to the socket.
+    // The subsequent test does not directly ensures the underlying socket
+    // is bound to the expected address, but the success of the tests should
+    // reasonably suggest it works as intended.
+    // Specifying an address also implicitly means the service runs in a
+    // single address-family mode.  In tests using TCP we can confirm that
+    // by trying to make a connection and seeing a failure.  In UDP, it'd be
+    // more complicated because we need to use a connected socket and catch
+    // an error on a subsequent read operation.  We could do it, but for
+    // simplicity we only tests the easier cases for now.
+
+    setIOService(*TEST_IPV6_ADDR);
+    doTest(AF_INET6, IPPROTO_UDP);
+}
+
+TEST_F(ASIOLinkTest, v6TCPSendSpecific) {
+    setIOService(*TEST_IPV6_ADDR);
+    doTest(AF_INET6, IPPROTO_TCP);
+
+    EXPECT_THROW(sendTCP(AF_INET), IOError);
+}
+
+TEST_F(ASIOLinkTest, v4UDPSendSpecific) {
+    setIOService(*TEST_IPV4_ADDR);
+    doTest(AF_INET, IPPROTO_UDP);
+}
+
+TEST_F(ASIOLinkTest, v4TCPSendSpecific) {
+    setIOService(*TEST_IPV4_ADDR);
+    doTest(AF_INET, IPPROTO_TCP);
+
+    EXPECT_THROW(sendTCP(AF_INET6), IOError);
+}
+
+TEST_F(ASIOLinkTest, v6TCPOnly) {
+    // Open only IPv6 TCP socket.  A subsequent attempt of establishing an
+    // IPv4/TCP connection should fail.  See above for why we only test this
+    // for TCP.
+    setIOService(false, true);
+    EXPECT_THROW(sendTCP(AF_INET), IOError);
+}
+
+TEST_F(ASIOLinkTest, v4TCPOnly) {
+    setIOService(true, false);
+    EXPECT_THROW(sendTCP(AF_INET6), IOError);
+}
+
+}

src/bin/auth/tests/auth_srv_unittest.cc

@@ -14,6 +14,8 @@
 
 // $Id$
 
+#include <config.h>
+
 #include <gtest/gtest.h>
 
 #include <dns/buffer.h>
@@ -24,33 +26,105 @@
 #include <dns/rrtype.h>
 
 #include <cc/data.h>
+#include <cc/session.h>
+
+#include <xfr/xfrout_client.h>
 
 #include <auth/auth_srv.h>
+#include <auth/asio_link.h>
 
 #include <dns/tests/unittest_util.h>
 
 using isc::UnitTestUtil;
 using namespace std;
+using namespace isc::cc;
 using namespace isc::dns;
 using namespace isc::data;
+using namespace isc::xfr;
+using namespace asio_link;
 
 namespace {
-const char* CONFIG_TESTDB =
+const char* const CONFIG_TESTDB =
     "{\"database_file\": \"" TEST_DATA_DIR "/example.sqlite3\"}";
 // The following file must be non existent and must be non"creatable" (see
 // the sqlite3 test).
-const char* BADCONFIG_TESTDB =
+const char* const BADCONFIG_TESTDB =
     "{ \"database_file\": \"" TEST_DATA_DIR "/nodir/notexist\"}";
+const char* const DEFAULT_REMOTE_ADDRESS = "192.0.2.1";
 
 class AuthSrvTest : public ::testing::Test {
+private:
+    class MockXfroutClient : public AbstractXfroutClient {
+    public:
+        MockXfroutClient() :
+            is_connected_(false), connect_ok_(true), send_ok_(true),
+            disconnect_ok_(true)
+        {}
+        virtual void connect();
+        virtual void disconnect();
+        virtual int sendXfroutRequestInfo(int tcp_sock, const void* msg_data,
+                                          uint16_t msg_len);
+        bool isConnected() const { return (is_connected_); }
+        void disableConnect() { connect_ok_ = false; }
+        void disableDisconnect() { disconnect_ok_ = false; }
+        void enableDisconnect() { disconnect_ok_ = true; }
+        void disableSend() { send_ok_ = false; }
+    private:
+        bool is_connected_;
+        bool connect_ok_;
+        bool send_ok_;
+        bool disconnect_ok_;
+    };
+
+    class MockSession : public AbstractSession {
+    public:
+        MockSession() :
+            // by default we return a simple "success" message.
+            msg_(Element::fromJSON("{\"result\": [0, \"SUCCESS\"]}")),
+            send_ok_(true), receive_ok_(true)
+        {}
+        virtual void establish(const char* socket_file);
+        virtual void disconnect();
+        virtual int group_sendmsg(ElementPtr msg, string group,
+                                  string instance, string to);
+        virtual bool group_recvmsg(ElementPtr& envelope, ElementPtr& msg,
+                                   bool nonblock, int seq);
+        virtual void subscribe(string group, string instance);
+        virtual void unsubscribe(string group, string instance);
+        virtual void startRead(boost::function<void()> read_callback);
+        virtual int reply(ElementPtr& envelope, ElementPtr& newmsg);
+        virtual bool hasQueuedMsgs();
+
+        void setMessage(ElementPtr msg) { msg_ = msg; }
+        void disableSend() { send_ok_ = false; }
+        void disableReceive() { receive_ok_ = false; }
+
+        ElementPtr sent_msg;
+        string msg_destination;
+    private:
+        ElementPtr msg_;
+        bool send_ok_;
+        bool receive_ok_;
+    };
+
 protected:
-    AuthSrvTest() : request_message(Message::RENDER),
+    AuthSrvTest() : server(true, xfrout),
+                    request_message(Message::RENDER),
                     parse_message(Message::PARSE), default_qid(0x1035),
                     opcode(Opcode(Opcode::QUERY())), qname("www.example.com"),
-                    qclass(RRClass::IN()), qtype(RRType::A()), ibuffer(NULL),
-                    request_obuffer(0), request_renderer(request_obuffer),
+                    qclass(RRClass::IN()), qtype(RRType::A()),
+                    io_message(NULL), endpoint(NULL), request_obuffer(0),
+                    request_renderer(request_obuffer),
                     response_obuffer(0), response_renderer(response_obuffer)
-    {}
+    {
+        server.setXfrinSession(&notify_session);
+    }
+    ~AuthSrvTest() {
+        delete io_message;
+        delete endpoint;
+    }
+    MockSession notify_session;
+    MockXfroutClient xfrout;
     AuthSrv server;
     Message request_message;
     Message parse_message;
@@ -59,16 +133,114 @@ protected:
     const Name qname;
     const RRClass qclass;
     const RRType qtype;
-    InputBuffer* ibuffer;
+    IOMessage* io_message;
+    const IOEndpoint* endpoint;
     OutputBuffer request_obuffer;
     MessageRenderer request_renderer;
     OutputBuffer response_obuffer;
     MessageRenderer response_renderer;
     vector<uint8_t> data;
 
-    void createDataFromFile(const char* const datafile);
+    void createDataFromFile(const char* const datafile, int protocol);
+    void createRequestMessage(const Opcode& opcode, const Name& request_name,
+                              const RRClass& rrclass, const RRType& rrtype);
+    void createRequestPacket(const Opcode& opcode, const Name& request_name,
+                             const RRClass& rrclass, const RRType& rrtype,
+                             int protocol);
+    void createRequestPacket(int protocol);
 };
 
+void
+AuthSrvTest::MockSession::establish(const char* socket_file UNUSED_PARAM) {}
+
+void
+AuthSrvTest::MockSession::disconnect() {}
+
+void
+AuthSrvTest::MockSession::subscribe(string group UNUSED_PARAM,
+                                    string instance UNUSED_PARAM)
+{}
+
+void
+AuthSrvTest::MockSession::unsubscribe(string group UNUSED_PARAM,
+                                      string instance UNUSED_PARAM)
+{}
+
+void
+AuthSrvTest::MockSession::startRead(
+    boost::function<void()> read_callback UNUSED_PARAM)
+{}
+
+int
+AuthSrvTest::MockSession::reply(ElementPtr& envelope UNUSED_PARAM,
+                                ElementPtr& newmsg UNUSED_PARAM)
+{
+    return (-1);
+}
+
+bool
+AuthSrvTest::MockSession::hasQueuedMsgs() {
+    return (false);
+}
+
+int
+AuthSrvTest::MockSession::group_sendmsg(ElementPtr msg, string group,
+                                        string instance UNUSED_PARAM,
+                                        string to UNUSED_PARAM)
+{
+    if (!send_ok_) {
+        isc_throw(XfroutError, "mock session send is disabled for test");
+    }
+
+    sent_msg = msg;
+    msg_destination = group;
+    return (0);
+}
+
+bool
+AuthSrvTest::MockSession::group_recvmsg(ElementPtr& envelope UNUSED_PARAM,
+                                        ElementPtr& msg,
+                                        bool nonblock UNUSED_PARAM,
+                                        int seq UNUSED_PARAM)
+{
+    if (!receive_ok_) {
+        isc_throw(XfroutError, "mock session receive is disabled for test");
+    }
+
+    msg = msg_;
+    return (true);
+}
+
+void
+AuthSrvTest::MockXfroutClient::connect() {
+    if (!connect_ok_) {
+        isc_throw(XfroutError, "xfrout connection disabled for test");
+    }
+    is_connected_ = true;
+}
+
+void
+AuthSrvTest::MockXfroutClient::disconnect() {
+    if (!disconnect_ok_) {
+        isc_throw(XfroutError,
+                  "closing xfrout connection is disabled for test");
+    }
+    is_connected_ = false;
+}
+
+int
+AuthSrvTest::MockXfroutClient::sendXfroutRequestInfo(
+    const int tcp_sock UNUSED_PARAM,
+    const void* msg_data UNUSED_PARAM,
+    const uint16_t msg_len UNUSED_PARAM)
+{
+    if (!send_ok_) {
+        isc_throw(XfroutError, "xfrout connection send is disabled for test");
+    }
+    return (0);
+}
+
+
 // These are flags to indicate whether the corresponding flag bit of the
 // DNS header is to be set in the test cases.  (Note that the flag values
 // is irrelevant to their wire-format values)
@@ -81,12 +253,56 @@ const unsigned int AD_FLAG = 0x20;
 const unsigned int CD_FLAG = 0x40;
 
 void
-AuthSrvTest::createDataFromFile(const char* const datafile) {
-    delete ibuffer;
+AuthSrvTest::createDataFromFile(const char* const datafile,
+                                const int protocol = IPPROTO_UDP)
+{
+    delete io_message;
     data.clear();
 
+    delete endpoint;
+    endpoint = IOEndpoint::create(protocol,
+                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 5300);
     UnitTestUtil::readWireData(datafile, data);
-    ibuffer = new InputBuffer(&data[0], data.size());
+    io_message = new IOMessage(&data[0], data.size(),
+                               protocol == IPPROTO_UDP ?
+                               IOSocket::getDummyUDPSocket() :
+                               IOSocket::getDummyTCPSocket(), *endpoint);
+}
+
+void
+AuthSrvTest::createRequestMessage(const Opcode& opcode,
+                                  const Name& request_name,
+                                  const RRClass& rrclass,
+                                  const RRType& rrtype)
+{
+    request_message.clear(Message::RENDER);
+    request_message.setOpcode(opcode);
+    request_message.setQid(default_qid);
+    request_message.addQuestion(Question(request_name, rrclass, rrtype));
+}
+
+void
+AuthSrvTest::createRequestPacket(const Opcode& opcode,
+                                 const Name& request_name,
+                                 const RRClass& rrclass, const RRType& rrtype,
+                                 const int protocol = IPPROTO_UDP)
+{
+    createRequestMessage(opcode, request_name, rrclass, rrtype);
+    createRequestPacket(protocol);
+}
+
+void
+AuthSrvTest::createRequestPacket(const int protocol = IPPROTO_UDP) {
+    request_message.toWire(request_renderer);
+
+    delete io_message;
+    endpoint = IOEndpoint::create(protocol,
+                                  IOAddress(DEFAULT_REMOTE_ADDRESS), 5300);
+    io_message = new IOMessage(request_renderer.getData(),
+                               request_renderer.getLength(),
+                               protocol == IPPROTO_UDP ?
+                               IOSocket::getDummyUDPSocket() :
+                               IOSocket::getDummyTCPSocket(), *endpoint);
 }
 
 void
@@ -115,15 +331,19 @@ headerCheck(const Message& message, const qid_t qid, const Rcode& rcode,
 
 // Unsupported requests.  Should result in NOTIMP.
 TEST_F(AuthSrvTest, unsupportedRequest) {
-    for (unsigned int i = 1; i < 16; ++i) {
+    for (unsigned int i = 0; i < 16; ++i) {
         // set Opcode to 'i', which iterators over all possible codes except
-        // the standard query (0)
+        // the standard query and notify
+        if (i == Opcode::QUERY().getCode() ||
+            i == Opcode::NOTIFY().getCode()) {
+            continue;
+        }
         createDataFromFile("simplequery_fromWire");
         data[2] = ((i << 3) & 0xff);
 
         parse_message.clear(Message::PARSE);
-        EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                              response_renderer, true));
+        EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                              response_renderer));
         headerCheck(parse_message, default_qid, Rcode::NOTIMP(), i, QR_FLAG,
                     0, 0, 0, 0);
     }
@@ -141,8 +361,8 @@ TEST_F(AuthSrvTest, verbose) {
 // Multiple questions.  Should result in FORMERR.
 TEST_F(AuthSrvTest, multiQuestion) {
     createDataFromFile("multiquestion_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
     headerCheck(parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
                 QR_FLAG, 2, 0, 0, 0);
 
@@ -162,8 +382,8 @@ TEST_F(AuthSrvTest, multiQuestion) {
 // dropped.
 TEST_F(AuthSrvTest, shortMessage) {
     createDataFromFile("shortmessage_fromWire");
-    EXPECT_EQ(false, server.processMessage(*ibuffer, parse_message,
-                                           response_renderer, true));
+    EXPECT_EQ(false, server.processMessage(*io_message, parse_message,
+                                           response_renderer));
 }
 
 // Response messages.  Must be silently dropped, whether it's a valid response
@@ -171,26 +391,26 @@ TEST_F(AuthSrvTest, shortMessage) {
 TEST_F(AuthSrvTest, response) {
     // A valid (although unusual) response
     createDataFromFile("simpleresponse_fromWire");
-    EXPECT_EQ(false, server.processMessage(*ibuffer, parse_message,
-                                           response_renderer, true));
+    EXPECT_EQ(false, server.processMessage(*io_message, parse_message,
+                                           response_renderer));
 
     // A response with a broken question section.  must be dropped rather than
     // returning FORMERR.
     createDataFromFile("shortresponse_fromWire");
-    EXPECT_EQ(false, server.processMessage(*ibuffer, parse_message,
-                                           response_renderer, true));
+    EXPECT_EQ(false, server.processMessage(*io_message, parse_message,
+                                           response_renderer));
 
     // A response to iquery.  must be dropped rather than returning NOTIMP.
     createDataFromFile("iqueryresponse_fromWire");
-    EXPECT_EQ(false, server.processMessage(*ibuffer, parse_message,
-                                           response_renderer, true));
+    EXPECT_EQ(false, server.processMessage(*io_message, parse_message,
+                                           response_renderer));
 }
 
 // Query with a broken question
 TEST_F(AuthSrvTest, shortQuestion) {
     createDataFromFile("shortquestion_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
     // Since the query's question is broken, the question section of the
     // response should be empty.
     headerCheck(parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
@@ -200,8 +420,8 @@ TEST_F(AuthSrvTest, shortQuestion) {
 // Query with a broken answer section
 TEST_F(AuthSrvTest, shortAnswer) {
     createDataFromFile("shortanswer_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
 
     // This is a bogus query, but question section is valid.  So the response
     // should copy the question section.
@@ -219,8 +439,8 @@ TEST_F(AuthSrvTest, shortAnswer) {
 // Query with unsupported version of EDNS.
 TEST_F(AuthSrvTest, ednsBadVers) {
     createDataFromFile("queryBadEDNS_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
 
     // The response must have an EDNS OPT RR in the additional section.
     // Note that the DNSSEC DO bit is cleared even if this bit in the query
@@ -231,12 +451,248 @@ TEST_F(AuthSrvTest, ednsBadVers) {
     EXPECT_FALSE(parse_message.isDNSSECSupported());
 }
 
+TEST_F(AuthSrvTest, AXFROverUDP) {
+    // AXFR over UDP is invalid and should result in FORMERR.
+    createRequestPacket(opcode, Name("example.com"), RRClass::IN(),
+                        RRType::AXFR(), IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::FORMERR(), opcode.getCode(),
+                QR_FLAG, 1, 0, 0, 0);
+}
+
+TEST_F(AuthSrvTest, AXFRSuccess) {
+    EXPECT_FALSE(xfrout.isConnected());
+    createRequestPacket(opcode, Name("example.com"), RRClass::IN(),
+                        RRType::AXFR(), IPPROTO_TCP);
+    // On success, the AXFR query has been passed to a separate process,
+    // so we shouldn't have to respond.
+    EXPECT_EQ(false, server.processMessage(*io_message, parse_message,
+                                           response_renderer));
+    EXPECT_TRUE(xfrout.isConnected());
+}
+
+TEST_F(AuthSrvTest, AXFRConnectFail) {
+    EXPECT_FALSE(xfrout.isConnected()); // check prerequisite
+    xfrout.disableConnect();
+    createRequestPacket(opcode, Name("example.com"), RRClass::IN(),
+                        RRType::AXFR(), IPPROTO_TCP);
+    EXPECT_TRUE(server.processMessage(*io_message, parse_message,
+                                      response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::SERVFAIL(),
+                opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
+    EXPECT_FALSE(xfrout.isConnected());
+}
+
+TEST_F(AuthSrvTest, AXFRSendFail) {
+    // first send a valid query, making the connection with the xfr process
+    // open.
+    createRequestPacket(opcode, Name("example.com"), RRClass::IN(),
+                        RRType::AXFR(), IPPROTO_TCP);
+    server.processMessage(*io_message, parse_message, response_renderer);
+    EXPECT_TRUE(xfrout.isConnected());
+
+    xfrout.disableSend();
+    parse_message.clear(Message::PARSE);
+    response_renderer.clear();
+    createRequestPacket(opcode, Name("example.com"), RRClass::IN(),
+                        RRType::AXFR(), IPPROTO_TCP);
+    EXPECT_TRUE(server.processMessage(*io_message, parse_message,
+                                      response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::SERVFAIL(),
+                opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
+
+    // The connection should have been closed due to the send failure.
+    EXPECT_FALSE(xfrout.isConnected());
+}
+
+TEST_F(AuthSrvTest, AXFRDisconnectFail) {
+    // In our usage disconnect() shouldn't fail.  So we'll see the exception
+    // should it be thrown.
+    xfrout.disableSend();
+    xfrout.disableDisconnect();
+    createRequestPacket(opcode, Name("example.com"), RRClass::IN(),
+                        RRType::AXFR(), IPPROTO_TCP);
+    EXPECT_THROW(server.processMessage(*io_message, parse_message,
+                                       response_renderer),
+                 XfroutError);
+    EXPECT_TRUE(xfrout.isConnected());
+    // XXX: we need to re-enable disconnect.  otherwise an exception would be
+    // thrown via the destructor of the server.
+    xfrout.enableDisconnect();
+}
+
+TEST_F(AuthSrvTest, notify) {
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+
+    // An internal command message should have been created and sent to an
+    // external module.  Check them.
+    EXPECT_EQ("Xfrin", notify_session.msg_destination);
+    EXPECT_EQ("notify",
+              notify_session.sent_msg->get("command")->get(0)->stringValue());
+    ElementPtr notify_args = notify_session.sent_msg->get("command")->get(1);
+    EXPECT_EQ("example.com.", notify_args->get("zone_name")->stringValue());
+    EXPECT_EQ(DEFAULT_REMOTE_ADDRESS,
+              notify_args->get("master")->stringValue());
+    EXPECT_EQ("IN", notify_args->get("rrclass")->stringValue());
+
+    // On success, the server should return a response to the notify.
+    headerCheck(parse_message, default_qid, Rcode::NOERROR(),
+                Opcode::NOTIFY().getCode(), QR_FLAG | AA_FLAG, 1, 0, 0, 0);
+
+    // The question must be identical to that of the received notify
+    ConstQuestionPtr question = *parse_message.beginQuestion();
+    EXPECT_EQ(Name("example.com"), question->getName());
+    EXPECT_EQ(RRClass::IN(), question->getClass());
+    EXPECT_EQ(RRType::SOA(), question->getType());
+}
+
+TEST_F(AuthSrvTest, notifyForCHClass) {
+    // Same as the previous test, but for the CH RRClass.
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::CH(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+
+    // Other conditions should be the same, so simply confirm the RR class is
+    // set correctly.
+    ElementPtr notify_args = notify_session.sent_msg->get("command")->get(1);
+    EXPECT_EQ("CH", notify_args->get("rrclass")->stringValue());
+}
+
+TEST_F(AuthSrvTest, notifyEmptyQuestion) {
+    request_message.clear(Message::RENDER);
+    request_message.setOpcode(Opcode::NOTIFY());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setQid(default_qid);
+    request_message.toWire(request_renderer);
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::FORMERR(),
+                Opcode::NOTIFY().getCode(), QR_FLAG, 0, 0, 0, 0);
+}
+
+TEST_F(AuthSrvTest, notifyMultiQuestions) {
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    // add one more SOA question
+    request_message.addQuestion(Question(Name("example.com"), RRClass::IN(),
+                                         RRType::SOA()));
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::FORMERR(),
+                Opcode::NOTIFY().getCode(), QR_FLAG, 2, 0, 0, 0);
+}
+
+TEST_F(AuthSrvTest, notifyNonSOAQuestion) {
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::NS());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::FORMERR(),
+                Opcode::NOTIFY().getCode(), QR_FLAG, 1, 0, 0, 0);
+}
+
+TEST_F(AuthSrvTest, notifyWithoutAA) {
+    // implicitly leave the AA bit off.  our implementation will accept it.
+    createRequestPacket(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::NOERROR(),
+                Opcode::NOTIFY().getCode(), QR_FLAG | AA_FLAG, 1, 0, 0, 0);
+}
+
+TEST_F(AuthSrvTest, notifyWithErrorRcode) {
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    request_message.setRcode(Rcode::SERVFAIL());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::NOERROR(),
+                Opcode::NOTIFY().getCode(), QR_FLAG | AA_FLAG, 1, 0, 0, 0);
+}
+
+TEST_F(AuthSrvTest, notifyWithoutSession) {
+    server.setXfrinSession(NULL);
+
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+
+    // we simply ignore the notify and let it be resent if an internal error
+    // happens.
+    EXPECT_FALSE(server.processMessage(*io_message, parse_message,
+                                       response_renderer));
+}
+
+TEST_F(AuthSrvTest, notifySendFail) {
+    notify_session.disableSend();
+
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+
+    EXPECT_FALSE(server.processMessage(*io_message, parse_message,
+                                       response_renderer));
+}
+
+TEST_F(AuthSrvTest, notifyReceiveFail) {
+    notify_session.disableReceive();
+
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_FALSE(server.processMessage(*io_message, parse_message,
+                                       response_renderer));
+}
+
+TEST_F(AuthSrvTest, notifyWithBogusSessionMessage) {
+    notify_session.setMessage(Element::fromJSON("{\"foo\": 1}"));
+
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_FALSE(server.processMessage(*io_message, parse_message,
+                                       response_renderer));
+}
+
+TEST_F(AuthSrvTest, notifyWithSessionMessageError) {
+    notify_session.setMessage(
+        Element::fromJSON("{\"result\": [1, \"FAIL\"]}"));
+
+    createRequestMessage(Opcode::NOTIFY(), Name("example.com"), RRClass::IN(),
+                        RRType::SOA());
+    request_message.setHeaderFlag(MessageFlag::AA());
+    createRequestPacket(IPPROTO_UDP);
+    EXPECT_FALSE(server.processMessage(*io_message, parse_message,
+                                       response_renderer));
+}
+
 void
 updateConfig(AuthSrv* server, const char* const dbfile,
              const bool expect_success)
 {
     const ElementPtr config_answer =
-        server->updateConfig(Element::createFromString(dbfile));
+        server->updateConfig(Element::fromJSON(dbfile));
     EXPECT_EQ(Element::map, config_answer->getType());
     EXPECT_TRUE(config_answer->contains("result"));
 
@@ -253,8 +709,8 @@ TEST_F(AuthSrvTest, updateConfig) {
     // response should have the AA flag on, and have an RR in each answer
     // and authority section.
     createDataFromFile("examplequery_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
     headerCheck(parse_message, default_qid, Rcode::NOERROR(), opcode.getCode(),
                 QR_FLAG | AA_FLAG, 1, 1, 1, 0);
 }
@@ -267,10 +723,10 @@ TEST_F(AuthSrvTest, datasourceFail) {
     // in a SERVFAIL response, and the answer and authority sections should
     // be empty.
     createDataFromFile("badExampleQuery_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
-    headerCheck(parse_message, default_qid, Rcode::SERVFAIL(), opcode.getCode(),
-                QR_FLAG, 1, 0, 0, 0);
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
+    headerCheck(parse_message, default_qid, Rcode::SERVFAIL(),
+                opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
 }
 
 TEST_F(AuthSrvTest, updateConfigFail) {
@@ -282,8 +738,8 @@ TEST_F(AuthSrvTest, updateConfigFail) {
 
     // The original data source should still exist.
     createDataFromFile("examplequery_fromWire");
-    EXPECT_EQ(true, server.processMessage(*ibuffer, parse_message,
-                                          response_renderer, true));
+    EXPECT_EQ(true, server.processMessage(*io_message, parse_message,
+                                          response_renderer));
     headerCheck(parse_message, default_qid, Rcode::NOERROR(), opcode.getCode(),
                 QR_FLAG | AA_FLAG, 1, 1, 1, 0);
 }

src/bin/bind10/bind10.py.in

@@ -56,7 +56,11 @@ import errno
 import time
 import select
 import random
+import socket
 from optparse import OptionParser, OptionValueError
+import io
+import pwd
+import posix
 
 import isc.cc
 
@@ -108,21 +112,38 @@ to avoid being restarted at exactly 10 seconds."""
             when = time.time()
         return max(when, self.restart_time)
 
+class ProcessInfoError(Exception): pass
+
 class ProcessInfo:
     """Information about a process"""
 
     dev_null = open(os.devnull, "w")
 
     def __init__(self, name, args, env={}, dev_null_stdout=False,
-                 dev_null_stderr=False):
+                 dev_null_stderr=False, uid=None, username=None):
         self.name = name 
         self.args = args
         self.env = env
         self.dev_null_stdout = dev_null_stdout
         self.dev_null_stderr = dev_null_stderr
         self.restart_schedule = RestartSchedule()
+        self.uid = uid
+        self.username = username
         self._spawn()
 
+    def _setuid(self):
+        """Function used before running a program that needs to run as a
+        different user."""
+        if self.uid is not None:
+            try:
+                posix.setuid(self.uid)
+            except OSError as e:
+                if e.errno == errno.EPERM:
+                    # if we failed to change user due to permission report that
+                    raise ProcessInfoError("Unable to change to user %s (uid %d)" % (self.username, self.uid))
+                else:
+                    # otherwise simply re-raise whatever error we found
+                    raise
 
     def _spawn(self):
         if self.dev_null_stdout:
@@ -138,24 +159,51 @@ class ProcessInfo:
         # on construction (self.env).
         spawn_env = os.environ
         spawn_env.update(self.env)
-        if not 'B10_FROM_SOURCE' in os.environ:
+        if 'B10_FROM_SOURCE' not in os.environ:
             spawn_env['PATH'] = "@@LIBEXECDIR@@:" + spawn_env['PATH']
         self.process = subprocess.Popen(self.args,
                                         stdin=subprocess.PIPE,
                                         stdout=spawn_stdout,
                                         stderr=spawn_stderr,
                                         close_fds=True,
-                                        env=spawn_env,)
+                                        env=spawn_env,
+                                        preexec_fn=self._setuid)
         self.pid = self.process.pid
         self.restart_schedule.set_run_start_time()
 
     def respawn(self):
         self._spawn()
 
+class IPAddr:
+    """Stores an IPv4 or IPv6 address."""
+    family = None
+    addr = None
+
+    def __init__(self, addr):
+        try:
+            a = socket.inet_pton(socket.AF_INET, addr)
+            self.family = socket.AF_INET
+            self.addr = a
+            return
+        except:
+            pass
+
+        try:
+            a = socket.inet_pton(socket.AF_INET6, addr)
+            self.family = socket.AF_INET6
+            self.addr = a
+            return
+        except Exception as e:
+            raise e
+    
+    def __str__(self):
+        return socket.inet_ntop(self.family, self.addr)
+
 class BoB:
     """Boss of BIND class."""
     
-    def __init__(self, msgq_socket_file=None, auth_port=5300, verbose=False):
+    def __init__(self, msgq_socket_file=None, auth_port=5300, address='',
+                 nocache=False, verbose=False, setuid=None, username=None):
         """Initialize the Boss of BIND. This is a singleton (only one
         can run).
         
@@ -166,11 +214,17 @@ class BoB:
         self.verbose = verbose
         self.msgq_socket_file = msgq_socket_file
         self.auth_port = auth_port
+        self.address = None
+        if address:
+            self.address = IPAddr(address)
         self.cc_session = None
         self.ccs = None
         self.processes = {}
         self.dead_processes = {}
         self.runnable = False
+        self.uid = setuid
+        self.username = username
+        self.nocache = nocache
 
     def config_handler(self, new_config):
         if self.verbose:
@@ -225,12 +279,14 @@ class BoB:
                 sys.stdout.write("[bind10] Starting b10-msgq\n")
         try:
             c_channel = ProcessInfo("b10-msgq", ["b10-msgq"], c_channel_env,
-                                    True, not self.verbose)
+                                    True, not self.verbose, uid=self.uid,
+                                    username=self.username)
         except Exception as e:
             return "Unable to start b10-msgq; " + str(e)
         self.processes[c_channel.pid] = c_channel
         if self.verbose:
-            sys.stdout.write("[bind10] Started b10-msgq (PID %d)\n" % c_channel.pid)
+            sys.stdout.write("[bind10] Started b10-msgq (PID %d)\n" % 
+                             c_channel.pid)
 
         # now connect to the c-channel
         cc_connect_start = time.time()
@@ -250,7 +306,8 @@ class BoB:
             sys.stdout.write("[bind10] Starting b10-cfgmgr\n")
         try:
             bind_cfgd = ProcessInfo("b10-cfgmgr", ["b10-cfgmgr"],
-                                    c_channel_env)
+                                    c_channel_env, uid=self.uid,
+                                    username=self.username)
         except Exception as e:
             c_channel.process.kill()
             return "Unable to start b10-cfgmgr; " + str(e)
@@ -272,30 +329,20 @@ class BoB:
         if self.verbose:
             sys.stdout.write("[bind10] ccsession started\n")
 
-        # start the xfrout before auth-server, to make sure every xfr-query can
-        # be processed properly.
-        xfrout_args = ['b10-xfrout']
-        if self.verbose:
-            sys.stdout.write("[bind10] Starting b10-xfrout\n")
-            xfrout_args += ['-v']
-        try:
-            xfrout = ProcessInfo("b10-xfrout", xfrout_args, 
-                                 c_channel_env )
-        except Exception as e:
-            c_channel.process.kill()
-            bind_cfgd.process.kill()
-            return "Unable to start b10-xfrout; " + str(e)
-        self.processes[xfrout.pid] = xfrout
-        if self.verbose:
-            sys.stdout.write("[bind10] Started b10-xfrout (PID %d)\n" % xfrout.pid)
-
         # start b10-auth
         # XXX: this must be read from the configuration manager in the future
         authargs = ['b10-auth', '-p', str(self.auth_port)]
+        if self.address:
+            authargs += ['-a', str(self.address)]
+        if self.nocache:
+            authargs += ['-n']
         if self.verbose:
-            sys.stdout.write("[bind10] Starting b10-auth using port %d\n" %
-                             self.auth_port)
             authargs += ['-v']
+            sys.stdout.write("Starting b10-auth using port %d" %
+                             self.auth_port)
+            if self.address:
+                sys.stdout.write(" on %s" % str(self.address))
+            sys.stdout.write("\n")
         try:
             auth = ProcessInfo("b10-auth", authargs,
                                c_channel_env)
@@ -308,6 +355,28 @@ class BoB:
         if self.verbose:
             sys.stdout.write("[bind10] Started b10-auth (PID %d)\n" % auth.pid)
 
+        # everything after the authoritative server can run as non-root
+        if self.uid is not None:
+            posix.setuid(self.uid)
+
+        # start the xfrout before auth-server, to make sure every xfr-query can
+        # be processed properly.
+        xfrout_args = ['b10-xfrout']
+        if self.verbose:
+            sys.stdout.write("[bind10] Starting b10-xfrout\n")
+            xfrout_args += ['-v']
+        try:
+            xfrout = ProcessInfo("b10-xfrout", xfrout_args, 
+                                 c_channel_env )
+        except Exception as e:
+            c_channel.process.kill()
+            bind_cfgd.process.kill()
+            return "Unable to start b10-xfrout; " + str(e)
+        self.processes[xfrout.pid] = xfrout
+        if self.verbose:
+            sys.stdout.write("[bind10] Started b10-xfrout (PID %d)\n" % 
+                             xfrout.pid)
+
         # start b10-xfrin
         xfrin_args = ['b10-xfrin']
         if self.verbose:
@@ -324,7 +393,8 @@ class BoB:
             return "Unable to start b10-xfrin; " + str(e)
         self.processes[xfrind.pid] = xfrind
         if self.verbose:
-            sys.stdout.write("[bind10] Started b10-xfrin (PID %d)\n" % xfrind.pid)
+            sys.stdout.write("[bind10] Started b10-xfrin (PID %d)\n" % 
+                             xfrind.pid)
 
         # start the b10-cmdctl
         # XXX: we hardcode port 8080
@@ -344,7 +414,8 @@ class BoB:
             return "Unable to start b10-cmdctl; " + str(e)
         self.processes[cmd_ctrld.pid] = cmd_ctrld
         if self.verbose:
-            sys.stdout.write("[bind10] Started b10-cmdctl (PID %d)\n" % cmd_ctrld.pid)
+            sys.stdout.write("[bind10] Started b10-cmdctl (PID %d)\n" % 
+                             cmd_ctrld.pid)
 
         self.runnable = True
 
@@ -353,7 +424,7 @@ class BoB:
     def stop_all_processes(self):
         """Stop all processes."""
         cmd = { "command": ['shutdown']}
-        self.cc_session.group_sendmsg(cmd, 'Boss', 'Cmd-Ctrld')
+        self.cc_session.group_sendmsg(cmd, 'Boss', 'Cmdctl')
         self.cc_session.group_sendmsg(cmd, "Boss", "ConfigManager")
         self.cc_session.group_sendmsg(cmd, "Boss", "Auth")
         self.cc_session.group_sendmsg(cmd, "Boss", "Xfrout")
@@ -435,11 +506,16 @@ class BoB:
                 sys.stdout.write("[bind10] Unknown child pid %d exited.\n" % pid)
 
     def restart_processes(self):
-        """Restart any dead processes."""
+        """Restart any dead processes.
+        Returns the time when the next process is ready to be restarted. 
+          If the server is shutting down, returns 0.
+          If there are no processes, returns None.
+        The values returned can be safely passed into select() as the 
+        timeout value."""
         next_restart = None
         # if we're shutting down, then don't restart
         if not self.runnable:
-            return next_restart
+            return 0
         # otherwise look through each dead process and try to restart
         still_dead = {}
         now = time.time()
@@ -507,20 +583,76 @@ def check_port(option, opt_str, value, parser):
     else:
         raise OptionValueError("Unknown option " + opt_str)
   
+def check_addr(option, opt_str, value, parser):
+    """Function to insure that the address we are passed is actually 
+    a valid address. Used by OptionParser() on startup."""
+    try:
+        IPAddr(value)
+    except:
+        raise OptionValueError("%s requires a valid IPv4 or IPv6 address" % opt_str)
+    if (opt_str == '-a' or opt_str == '--address'):
+        parser.values.address = value
+    else:
+        raise OptionValueError("Unknown option " + opt_str)
+  
 def main():
     global options
     global boss_of_bind
+    # Enforce line buffering on stdout, even when not a TTY
+    sys.stdout = io.TextIOWrapper(sys.stdout.detach(), line_buffering=True)
+
+
     # Parse any command-line options.
     parser = OptionParser(version=__version__)
-    parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
-                      help="display more about what is going on")
+    parser.add_option("-a", "--address", dest="address", type="string",
+                      action="callback", callback=check_addr, default='',
+                      help="address the b10-auth daemon will use (default: listen on all addresses)")
+    parser.add_option("-m", "--msgq-socket-file", dest="msgq_socket_file",
+                      type="string", default=None,
+                      help="UNIX domain socket file the b10-msgq daemon will use")
+    parser.add_option("-n", "--no-cache", action="store_true", dest="nocache",
+                      default=False, help="disable hot-spot cache in b10-auth")
     parser.add_option("-p", "--port", dest="auth_port", type="string",
                       action="callback", callback=check_port, default="5300",
                       help="port the b10-auth daemon will use (default 5300)")
-    parser.add_option("-m", "--msgq-socket-file", dest="msgq_socket_file",
+    parser.add_option("-u", "--user", dest="user",
                       type="string", default=None,
-                      help="UNIX domain socket file the b10-msgq daemon will use")
+                      help="Change user after startup (must run as root)")
+    parser.add_option("-v", "--verbose", dest="verbose", action="store_true",
+                      help="display more about what is going on")
     (options, args) = parser.parse_args()
+    if args:
+        parser.print_help()
+        sys.exit(1)
+
+    # Check user ID.
+    setuid = None
+    username = None
+    if options.user:
+        # Try getting information about the user, assuming UID passed.
+        try:
+            pw_ent = pwd.getpwuid(int(options.user))
+            setuid = pw_ent.pw_uid
+            username = pw_ent.pw_name
+        except ValueError:
+            pass
+        except KeyError:
+            pass
+
+        # Next try getting information about the user, assuming user name 
+        # passed.
+        # If the information is both a valid user name and user number, we
+        # prefer the name because we try it second. A minor point, hopefully.
+        try:
+            pw_ent = pwd.getpwnam(options.user)
+            setuid = pw_ent.pw_uid
+            username = pw_ent.pw_name
+        except KeyError:
+            pass
+
+        if setuid is None:
+            sys.stderr.write("bind10: invalid user: '%s'\n" % options.user)
+            sys.exit(1)
 
     # Announce startup.
     if options.verbose:
@@ -543,11 +675,13 @@ def main():
 
     # Go bob!
     boss_of_bind = BoB(options.msgq_socket_file, int(options.auth_port),
-                       options.verbose)
+                       options.address, options.nocache, options.verbose,
+                       setuid, username)
     startup_result = boss_of_bind.startup()
     if startup_result:
         sys.stderr.write("[bind10] Error on startup: %s\n" % startup_result)
         sys.exit(1)
+    sys.stdout.write("[bind10] BIND 10 started\n")
 
     # In our main loop, we check for dead processes or messages 
     # on the c-channel.
@@ -584,6 +718,7 @@ def main():
     # shutdown
     signal.signal(signal.SIGCHLD, signal.SIG_DFL)
     boss_of_bind.shutdown()
+    sys.exit(0)
 
 if __name__ == "__main__":
     main()

src/bin/bind10/run_bind10.sh.in

@@ -23,7 +23,8 @@ BIND10_PATH=@abs_top_builddir@/src/bin/bind10
 PATH=@abs_top_builddir@/src/bin/msgq:@abs_top_builddir@/src/bin/auth:@abs_top_builddir@/src/bin/cfgmgr:@abs_top_builddir@/src/bin/cmdctl:@abs_top_builddir@/src/bin/xfrin:@abs_top_builddir@/src/bin/xfrout:$PATH
 export PATH
 
-PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/xfr/.libs
+PYTHONPATH=@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/xfr/.libs
+#PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/xfr/.libs
 export PYTHONPATH
 
 B10_FROM_SOURCE=@abs_top_srcdir@

src/bin/bind10/tests/args_test.py

@@ -0,0 +1,134 @@
+"""
+This program tests the boss process to make sure that it runs while
+dropping permissions. It must be run as a user that can set permission.
+"""
+import unittest
+import os
+import sys
+import subprocess
+import select
+import time
+import pwd
+
+# Set to a valid user name on the system to run setuid() test
+#SUID_USER=None
+SUID_USER="shane"
+
+BIND10_EXE="../run_bind10.sh"
+TIMEOUT=3
+
+class TestBossArgs(unittest.TestCase):
+    def _waitForString(self, bob, s):
+        found_string = False
+        start_time = time.time()
+        while time.time() < start_time + TIMEOUT:
+            (r,w,x) = select.select((bob.stdout,), (), (), TIMEOUT) 
+            if bob.stdout in r:
+                s = bob.stdout.readline()
+                if s == '':
+                    break
+                if s.startswith(s): 
+                    found_string = True
+                    break
+        return found_string
+
+    def testNoArgs(self):
+        """Run bind10 without any arguments"""
+        bob = subprocess.Popen(args=(BIND10_EXE,),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        started_ok = self._waitForString(bob, '[bind10] BIND 10 started')
+        time.sleep(0.1)
+        bob.terminate()
+        bob.wait()
+        self.assertTrue(started_ok)
+
+    def testBadOption(self):
+        """Run bind10 with a bogus option"""
+        bob = subprocess.Popen(args=(BIND10_EXE, "--badoption"),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        failed = self._waitForString(bob, 'bind10: error: no such option: --badoption')
+        time.sleep(0.1)
+        bob.terminate()
+        self.assertTrue(bob.wait() == 2)
+        self.assertTrue(failed)
+
+    def testArgument(self):
+        """Run bind10 with an argument (this is not allowed)"""
+        bob = subprocess.Popen(args=(BIND10_EXE, "argument"),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        failed = self._waitForString(bob, 'Usage: bind10 [options]')
+        time.sleep(0.1)
+        bob.terminate()
+        self.assertTrue(bob.wait() == 1)
+        self.assertTrue(failed)
+
+    def testBadUser(self):
+        """Run bind10 with a bogus user"""
+        bob = subprocess.Popen(args=(BIND10_EXE, "-u", "bogus_user"),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        failed = self._waitForString(bob, "bind10: invalid user: 'bogus_user'")
+        time.sleep(0.1)
+        bob.terminate()
+        self.assertTrue(bob.wait() == 1)
+        self.assertTrue(failed)
+
+    def testBadUid(self):
+        """Run bind10 with a bogus user ID"""
+        bob = subprocess.Popen(args=(BIND10_EXE, "-u", "999999999"),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        failed = self._waitForString(bob, "bind10: invalid user: '999999999'")
+        time.sleep(0.1)
+        bob.terminate()
+        self.assertTrue(bob.wait() == 1)
+        self.assertTrue(failed)
+
+    def testFailSetUser(self):
+        """Try the -u option when we don't run as root"""
+        global SUID_USER
+        if SUID_USER is None:
+            self.skipTest("test needs a valid user (set when run)")
+        if os.getuid() == 0:
+            self.skipTest("test must not be run as root (uid is 0)")
+        # XXX: we depend on the "nobody" user
+        bob = subprocess.Popen(args=(BIND10_EXE, "-u", "nobody"),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        failed = self._waitForString(bob, "[bind10] Error on startup: Unable to start b10-msgq; Unable to change to user nobody")
+        time.sleep(0.1)
+        bob.terminate()
+        self.assertTrue(bob.wait() == 1)
+        self.assertTrue(failed)
+
+    def testSetUser(self):
+        """Try the -u option"""
+        global SUID_USER
+        if SUID_USER is None:
+            self.skipTest("test needs a valid user (set when run)")
+        if os.getuid() != 0:
+            self.skipTest("test must run as root (uid is not 0)")
+        if os.geteuid() != 0:
+            self.skipTest("test must run as root (euid is not 0)")
+
+        bob = subprocess.Popen(args=(BIND10_EXE, "-u", SUID_USER),
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+        started_ok = self._waitForString(bob, '[bind10] BIND 10 started')
+        self.assertTrue(started_ok)
+        ps = subprocess.Popen(args=("ps", "axo", "user,pid"),
+                              stdout=subprocess.PIPE)
+        s = ps.stdout.readline()
+        ps_user = None
+        while True:
+            s = ps.stdout.readline()
+            if s == '': break
+            (user, pid) = s.split()
+            if int(pid) == bob.pid:
+                ps_user = user.decode()
+                break
+        self.assertTrue(ps_user is not None)
+        self.assertTrue(ps_user == SUID_USER)
+        time.sleep(0.1)
+        bob.terminate()
+        x = bob.wait()
+        self.assertTrue(bob.wait() == 0)
+
+if __name__ == '__main__':
+    unittest.main()

src/bin/bind10/tests/bind10_test.in

@@ -27,5 +27,6 @@ PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_srcdir@/src/bin/bind10
 export PYTHONPATH
 
 cd ${BIND10_PATH}/tests
-exec ${PYTHON_EXEC} -O bind10_test.py $*
+${PYTHON_EXEC} -O bind10_test.py $*
+exec ${PYTHON_EXEC} -O args_test.py $*
 

src/bin/bind10/tests/bind10_test.py

@@ -1,4 +1,4 @@
-from bind10 import ProcessInfo, BoB
+from bind10 import ProcessInfo, BoB, IPAddr
 
 # XXX: environment tests are currently disabled, due to the preprocessor
 #      setup that we have now complicating the environment
@@ -7,6 +7,7 @@ import unittest
 import sys
 import os
 import signal
+import socket
 
 class TestProcessInfo(unittest.TestCase):
     def setUp(self):
@@ -71,12 +72,36 @@ class TestProcessInfo(unittest.TestCase):
         self.assertTrue(type(pi.pid) is int)
         self.assertNotEqual(pi.pid, old_pid)
 
+class TestIPAddr(unittest.TestCase):
+    def test_v6ok(self):
+        addr = IPAddr('2001:4f8::1')
+        self.assertEqual(addr.family, socket.AF_INET6)
+        self.assertEqual(addr.addr, socket.inet_pton(socket.AF_INET6, '2001:4f8::1'))
+
+    def test_v4ok(self):
+        addr = IPAddr('127.127.127.127')
+        self.assertEqual(addr.family, socket.AF_INET)
+        self.assertEqual(addr.addr, socket.inet_aton('127.127.127.127'))
+
+    def test_badaddr(self):
+        self.assertRaises(socket.error, IPAddr, 'foobar')
+        self.assertRaises(socket.error, IPAddr, 'foo::bar')
+        self.assertRaises(socket.error, IPAddr, '123')
+        self.assertRaises(socket.error, IPAddr, '123.456.789.0')
+        self.assertRaises(socket.error, IPAddr, '127/8')
+        self.assertRaises(socket.error, IPAddr, '0/0')
+        self.assertRaises(socket.error, IPAddr, '1.2.3.4/32')
+        self.assertRaises(socket.error, IPAddr, '0')
+        self.assertRaises(socket.error, IPAddr, '')
+
 class TestBoB(unittest.TestCase):
     def test_init(self):
         bob = BoB()
         self.assertEqual(bob.verbose, False)
         self.assertEqual(bob.msgq_socket_file, None)
+        self.assertEqual(bob.auth_port, 5300)
         self.assertEqual(bob.cc_session, None)
+        self.assertEqual(bob.address, None)
         self.assertEqual(bob.processes, {})
         self.assertEqual(bob.dead_processes, {})
         self.assertEqual(bob.runnable, False)
@@ -90,6 +115,27 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.dead_processes, {})
         self.assertEqual(bob.runnable, False)
 
+    def test_init_alternate_auth_port(self):
+        bob = BoB(None, 9999)
+        self.assertEqual(bob.verbose, False)
+        self.assertEqual(bob.msgq_socket_file, None)
+        self.assertEqual(bob.auth_port, 9999)
+        self.assertEqual(bob.cc_session, None)
+        self.assertEqual(bob.address, None)
+        self.assertEqual(bob.processes, {})
+        self.assertEqual(bob.dead_processes, {})
+        self.assertEqual(bob.runnable, False)
+
+    def test_init_alternate_address(self):
+        bob = BoB(None, 5300, '127.127.127.127')
+        self.assertEqual(bob.verbose, False)
+        self.assertEqual(bob.auth_port, 5300)
+        self.assertEqual(bob.msgq_socket_file, None)
+        self.assertEqual(bob.cc_session, None)
+        self.assertEqual(bob.address.addr, socket.inet_aton('127.127.127.127'))
+        self.assertEqual(bob.processes, {})
+        self.assertEqual(bob.dead_processes, {})
+        self.assertEqual(bob.runnable, False)
     # verbose testing...
 
 if __name__ == '__main__':

src/bin/bindctl/Makefile.am

@@ -9,8 +9,6 @@ python_PYTHON = __init__.py bindcmd.py cmdparse.py exception.py moduleinfo.py my
 pythondir = $(pyexecdir)/bindctl
 
 bindctldir = $(DESTDIR)$(pkgdatadir)
-bindctl_DATA = bindctl.pem
-EXTRA_DIST += bindctl.pem
 
 CLEANFILES = bindctl
 
@@ -26,14 +24,3 @@ bindctl: bindctl-source.py
 	       -e "s|@@SYSCONFDIR@@|@sysconfdir@|" \
 	       -e "s|@@LIBEXECDIR@@|$(pkglibexecdir)|" bindctl-source.py >$@
 	chmod a+x $@
-
-if INSTALL_CONFIGURATIONS
-
-# TODO: permissions handled later
-install-data-local:
-	$(mkinstalldirs) $(DESTDIR)/@sysconfdir@/@PACKAGE@   
-	if test ! -f $(DESTDIR)$(sysconfdir)/@PACKAGE@/bindctl.pem; then	\
-	  $(INSTALL_DATA) $(srcdir)/bindctl.pem $(DESTDIR)$(sysconfdir)/@PACKAGE@/ ;	\
-	fi
-
-endif

src/bin/bindctl/TODO

@@ -18,3 +18,4 @@ functions should be updated first.):
    If the default user is saved in file, its password shouldn't be saved in plaintext.
 7. Need to think of what exactly to do with responses received to commands
    (currently it simply print the map)
+8. Remove bindctl-source.py.in(replace with bindctl-source.py) when merging to trunk.

src/bin/bindctl/bindcmd.py

@@ -35,7 +35,10 @@ import os, time, random, re
 import getpass
 from hashlib import sha1
 import csv
-import ast
+import json
+import pwd
+import getpass
+import traceback
 
 try:
     from collections import OrderedDict
@@ -49,6 +52,8 @@ try:
 except ImportError:
     my_readline = sys.stdin.readline
 
+CSV_FILE_NAME = 'default_user.csv'
+FAIL_TO_CONNECT_WITH_CMDCTL = "Fail to connect with b10-cmdctl module, is it running?"
 CONFIG_MODULE_NAME = 'config'
 CONST_BINDCTL_HELP = """
 usage: <module name> <command name> [param1 = value1 [, param2 = value2]]
@@ -58,10 +63,34 @@ Type \"<module_name> help\" for help on the specific module.
 Type \"<module_name> <command_name> help\" for help on the specific command.
 \nAvailable module names: """
 
+class ValidatedHTTPSConnection(http.client.HTTPSConnection):
+    '''Overrides HTTPSConnection to support certification 
+    validation. '''
+    def __init__(self, host, ca_certs):
+        http.client.HTTPSConnection.__init__(self, host)
+        self.ca_certs = ca_certs
+
+    def connect(self):
+        ''' Overrides the connect() so that we do 
+        certificate validation. '''
+        sock = socket.create_connection((self.host, self.port),
+                                        self.timeout)
+        if self._tunnel_host:
+            self.sock = sock
+            self._tunnel()
+       
+        req_cert = ssl.CERT_NONE
+        if self.ca_certs:
+            req_cert = ssl.CERT_REQUIRED
+        self.sock = ssl.wrap_socket(sock, self.key_file,
+                                    self.cert_file,
+                                    cert_reqs=req_cert,
+                                    ca_certs=self.ca_certs)
+
 class BindCmdInterpreter(Cmd):
     """simple bindctl example."""    
 
-    def __init__(self, server_port = 'localhost:8080', pem_file = "bindctl.pem"):
+    def __init__(self, server_port = 'localhost:8080', pem_file = None):
         Cmd.__init__(self)
         self.location = ""
         self.prompt_end = '> '
@@ -70,19 +99,11 @@ class BindCmdInterpreter(Cmd):
         self.modules = OrderedDict()
         self.add_module_info(ModuleInfo("help", desc = "Get help for bindctl"))
         self.server_port = server_port
-        self.pem_file = pem_file
-        self._connect_to_cmd_ctrld()
+        self.conn = ValidatedHTTPSConnection(self.server_port,
+                                             ca_certs=pem_file)
         self.session_id = self._get_session_id()
-
-    def _connect_to_cmd_ctrld(self):
-        '''Connect to cmdctl in SSL context. '''
-        try:
-            self.conn = http.client.HTTPSConnection(self.server_port,
-                          cert_file=self.pem_file)
-        except  Exception as e:
-            print(e, "can't connect to %s, please make sure cmd-ctrld is running" %
-                  self.server_port)
-
+        self.config_data = None
+        
     def _get_session_id(self):
         '''Generate one session id for the connection. '''
         rand = os.urandom(16)
@@ -93,17 +114,59 @@ class BindCmdInterpreter(Cmd):
         return digest
     
     def run(self):
-        '''Parse commands inputted from user and send them to cmdctl. '''
+        '''Parse commands from user and send them to cmdctl. '''
         try:
             if not self.login_to_cmdctl():
-                return False
+                return 
 
-            # Get all module information from cmd-ctrld
-            self.config_data = isc.config.UIModuleCCSession(self)
-            self._update_commands()
             self.cmdloop()
+        except FailToLogin as err:
+            print(err)
+            print(FAIL_TO_CONNECT_WITH_CMDCTL)
+            traceback.print_exc()
         except KeyboardInterrupt:
-            return True
+            print('\nExit from bindctl')
+
+    def _get_saved_user_info(self, dir, file_name):
+        ''' Read all the available username and password pairs saved in 
+        file(path is "dir + file_name"), Return value is one list of elements
+        ['name', 'password'], If get information failed, empty list will be 
+        returned.'''
+        if (not dir) or (not os.path.exists(dir)):
+            return []
+
+        try:
+            csvfile = None
+            users = []
+            csvfile = open(dir + file_name)
+            users_info = csv.reader(csvfile)
+            for row in users_info:
+                users.append([row[0], row[1]])
+        except (IOError, IndexError) as e:
+            pass
+        finally:
+            if csvfile:
+                csvfile.close()
+            return users
+
+    def _save_user_info(self, username, passwd, dir, file_name):
+        ''' Save username and password in file "dir + file_name"
+        If it's saved properly, return True, or else return False. '''
+        try:
+            if not os.path.exists(dir):
+                os.mkdir(dir, 0o700)
+
+            csvfilepath = dir + file_name 
+            csvfile = open(csvfilepath, 'w')
+            os.chmod(csvfilepath, 0o600)
+            writer = csv.writer(csvfile)
+            writer.writerow([username, passwd])
+            csvfile.close()
+        except Exception as e:
+            print(e, "\nCannot write %s%s; default user is not stored" % (dir, file_name))
+            return False
+
+        return True
 
     def login_to_cmdctl(self):
         '''Login to cmdctl with the username and password inputted 
@@ -112,33 +175,21 @@ class BindCmdInterpreter(Cmd):
         time, username and password saved in 'default_user.csv' will be
         used first.
         '''
-        csvfile = None
-        bsuccess = False
-        try:
-            cvsfilepath = ""
-            if ('HOME' in os.environ):
-                cvsfilepath = os.environ['HOME']
-                cvsfilepath += os.sep + '.bind10' + os.sep
-            cvsfilepath += 'default_user.csv'
-            csvfile = open(cvsfilepath)
-            users = csv.reader(csvfile)
-            for row in users:
-                param = {'username': row[0], 'password' : row[1]}
+        csv_file_dir = pwd.getpwnam(getpass.getuser()).pw_dir
+        csv_file_dir += os.sep + '.bind10' + os.sep
+        users = self._get_saved_user_info(csv_file_dir, CSV_FILE_NAME)
+        for row in users:
+            param = {'username': row[0], 'password' : row[1]}
+            try:
                 response = self.send_POST('/login', param)
                 data = response.read().decode()
-                if response.status == http.client.OK:
-                    print(data + ' login as ' + row[0] )
-                    bsuccess = True
-                    break
-        except IOError as e:
-            pass
-        except Exception as e:
-            print(e)
-        finally:
-            if csvfile:
-                csvfile.close()
-            if bsuccess:
-                return True
+            except socket.error:
+                traceback.print_exc()
+                raise FailToLogin()
+
+            if response.status == http.client.OK:
+                print(data + ' login as ' + row[0] )
+                return True 
 
         count = 0
         print("[TEMP MESSAGE]: username :root  password :bind10")
@@ -151,34 +202,18 @@ class BindCmdInterpreter(Cmd):
             username = input("Username:")
             passwd = getpass.getpass()
             param = {'username': username, 'password' : passwd}
-            response = self.send_POST('/login', param)
-            data = response.read().decode()
-            print(data)
-            
+            try:
+                response = self.send_POST('/login', param)
+                data = response.read().decode()
+                print(data)
+            except socket.error as e:
+                traceback.print_exc()
+                raise FailToLogin()
+
             if response.status == http.client.OK:
-                cvsfilepath = ""
-                try:
-                    if ('HOME' in os.environ):
-                        cvsfilepath = os.environ['HOME']
-                        cvsfilepath += os.sep + '.bind10' + os.sep
-                        if not os.path.exists(cvsfilepath):
-                                os.mkdir(cvsfilepath, 0o700)
-                    else:
-                        print("Cannot determine location of $HOME. Not storing default user")
-                        return True
-                    cvsfilepath += 'default_user.csv'
-                    csvfile = open(cvsfilepath, 'w')
-                    os.chmod(cvsfilepath, 0o600)
-                    writer = csv.writer(csvfile)
-                    writer.writerow([username, passwd])
-                    csvfile.close()
-                except Exception as e:
-                    # just not store it
-                    print("Cannot write ~/.bind10/default_user.csv; default user is not stored")
-                    print(e)
+                self._save_user_info(username, passwd, csv_file_dir, CSV_FILE_NAME)
                 return True
 
-
     def _update_commands(self):
         '''Update the commands of all modules. '''
         for module_name in self.config_data.get_config_item_list():
@@ -211,7 +246,22 @@ class BindCmdInterpreter(Cmd):
         headers = {"cookie" : self.session_id}
         self.conn.request('POST', url, param, headers)
         return self.conn.getresponse()
-        
+
+    def _update_all_modules_info(self):
+        ''' Get all modules' information from cmdctl, including
+        specification file and configuration data. This function
+        should be called before interpreting command line or complete-key
+        is entered. This may not be the best way to keep bindctl
+        and cmdctl share same modules information, but it works.'''
+        if self.config_data is not None:
+            self.config_data.update_specs_and_config()
+        else:
+            self.config_data = isc.config.UIModuleCCSession(self)
+        self._update_commands()
+
+    def precmd(self, line):
+        self._update_all_modules_info()
+        return line 
 
     def postcmd(self, stop, line):
         '''Update the prompt after every command'''
@@ -308,8 +358,11 @@ class BindCmdInterpreter(Cmd):
         if cmd.module != CONFIG_MODULE_NAME:
             for param_name in cmd.params:
                 param_spec = command_info.get_param_with_name(param_name).param_spec
-                cmd.params[param_name] = isc.config.config_data.convert_type(param_spec, cmd.params[param_name])
-
+                try:
+                    cmd.params[param_name] = isc.config.config_data.convert_type(param_spec, cmd.params[param_name])
+                except isc.cc.data.DataTypeError as e:
+                    raise isc.cc.data.DataTypeError('Invalid parameter value for \"%s\", the type should be \"%s\" \n' 
+                                                     % (param_name, param_spec['item_type']) + str(e))
     
     def _handle_cmd(self, cmd):
         '''Handle a command entered by the user'''
@@ -356,6 +409,7 @@ class BindCmdInterpreter(Cmd):
 
     def complete(self, text, state):
         if 0 == state:
+            self._update_all_modules_info()
             text = text.strip()
             hints = []
             cur_line = my_readline()
@@ -441,13 +495,14 @@ class BindCmdInterpreter(Cmd):
             cmd = BindCmdParse(line)
             self._validate_cmd(cmd)
             self._handle_cmd(cmd)
-        except BindCtlException as e:
-            print("Error! ", e)
-            self._print_correct_usage(e)
-        except isc.cc.data.DataTypeError as e:
-            print("Error! ", e)
-            self._print_correct_usage(e)
-            
+        except (IOError, http.client.HTTPException) as err:
+            print('Error!', err)
+            print(FAIL_TO_CONNECT_WITH_CMDCTL)
+        except BindCtlException as err:
+            print("Error! ", err)
+            self._print_correct_usage(err)
+        except isc.cc.data.DataTypeError as err:
+            print("Error! ", err)
             
     def _print_correct_usage(self, ept):        
         if isinstance(ept, CmdUnknownModuleSyntaxError):
@@ -488,6 +543,16 @@ class BindCmdInterpreter(Cmd):
                     identifier = cmd.params['identifier']
                 else:
                     identifier += cmd.params['identifier']
+
+                # Check if the module is known; for unknown modules
+                # we currently deny setting preferences, as we have
+                # no way yet to determine if they are ok.
+                module_name = identifier.split('/')[1]
+                if self.config_data is None or \
+                   not self.config_data.have_specification(module_name):
+                    print("Error: Module '" + module_name + "' unknown or not running")
+                    return
+
             if cmd.command == "show":
                 values = self.config_data.get_value_maps(identifier)
                 for value_map in values:
@@ -513,7 +578,7 @@ class BindCmdInterpreter(Cmd):
                 else:
                     parsed_value = None
                     try:
-                        parsed_value = ast.literal_eval(cmd.params['value'])
+                        parsed_value = json.loads(cmd.params['value'])
                     except Exception as exc:
                         # ok could be an unquoted string, interpret as such
                         parsed_value = cmd.params['value']
@@ -556,7 +621,7 @@ class BindCmdInterpreter(Cmd):
         if (len(cmd.params) != 0):
             cmd_params = json.dumps(cmd.params)
 
-        print("send the message to cmd-ctrld")        
+        print("send the command to cmd-ctrld")        
         reply = self.send_POST(url, cmd.params)
         data = reply.read().decode()
         print("received reply:", data)

src/bin/bindctl/bindctl-source.py.in

@@ -97,13 +97,16 @@ def check_addr(option, opt_str, value, parser):
 
 def set_bindctl_options(parser):
     parser.add_option('-p', '--port', dest = 'port', type = 'int',
-            action = 'callback', callback=check_port,
-            default = '8080', help = 'port for cmdctl of bind10')
+                      action = 'callback', callback=check_port,
+                      default = '8080', help = 'port for cmdctl of bind10')
 
     parser.add_option('-a', '--address', dest = 'addr', type = 'string',
-            action = 'callback', callback=check_addr,
-            default = '127.0.0.1', help = 'IP address for cmdctl of bind10')
+                      action = 'callback', callback=check_addr,
+                      default = '127.0.0.1', help = 'IP address for cmdctl of bind10')
 
+    parser.add_option('-c', '--certificate-chain', dest = 'cert_chain', 
+                      type = 'string', action = 'store',
+                      help = 'PEM formatted server certificate validation chain file')
 
 if __name__ == '__main__':
     try:
@@ -111,14 +114,7 @@ if __name__ == '__main__':
         set_bindctl_options(parser)
         (options, args) = parser.parse_args()
         server_addr = options.addr + ':' + str(options.port)
-        # If B10_FROM_SOURCE is set in the environment, we use PEM file
-        # from a directory relative to that, otherwise we use the one
-        # installed on the system
-        if "B10_FROM_SOURCE" in os.environ:
-            SYSCONF_PATH = os.environ["B10_FROM_SOURCE"] + "/src/bin/bindctl"
-        else:
-            SYSCONF_PATH = "@@SYSCONFDIR@@/@PACKAGE@"
-        tool = BindCmdInterpreter(server_addr, pem_file = SYSCONF_PATH + "/bindctl.pem")
+        tool = BindCmdInterpreter(server_addr, pem_file=options.cert_chain)
         prepare_config_commands(tool)
         tool.run()
     except Exception as e:

src/bin/bindctl/bindctl.pem

@@ -1,36 +0,0 @@
------BEGIN RSA PRIVATE KEY-----
-MIICXAIBAAKBgQDpICWxJGKMvUhLFPbf5n8ZWogqjYcQqqoHqHVRHYjyiey6FZdt
-ZkY2s1gYh0G0NXtimlIgic+vEcFe7vdmyKntW7DYDaqAj0KrED7RKAj8324jNbSJ
-HtLP4evvJep3vsoNtTvNuceQJ46vukxyxgg3DuC9kVqPuD8CZ1Rq4ATyiwIDAQAB
-AoGBAOJlOtV+DUq6Y2Ou91VXRiU8GzKgAQP5iWgoe84Ljbxkn4XThBxVD2j94Fbp
-u7AjpDCMx6cbzpoo9w6XqaGizAmAehIfTE3eFYs74N/FM09Wg2OSDyxMY0jgyECU
-A4ukjlPwcGDbmgbmlY3i+FVHp+zCgtZEsMC1IAosMac1BoX5AkEA/lrXWaVtH8bo
-mut3GBaXvubZMdaUr0BUd5a9q+tt4dQcKG1kFqgCNKhNhBIcpiMVcz+jGmOuopNA
-8dnUGqv3FQJBAOqiJ54ZvOTWNDpJIe02wIXRxRmc1xhHFCqYP23KxBVrAcTYB19J
-lesov/hEbnGLCbKS/naZJ1zrTImUPNRLqx8CQCzDtA7U7GWhTiKluioFH+O7IRKC
-X1yQh80cPHlbT9VkzSfYSLssCmdWD35k6aHbntTPqFbmoD+AhveJjKi9BxkCQDwX
-1c+/RcrSNcQr0N2hZUOgyztZGRnlsnuKTMyA3yGhK23P6mt0PEpjQG+Ej0jTVGOB
-FF0pspQwy4R9C+tPif8CQH36NNlXBfVNmT7kDtyLmaE6pID0vY9duX56BJbU1R0x
-SQ8/LcfJagk6gvp08OyYCPA+WZ7u/bas9R/nMTCLivc=
------END RSA PRIVATE KEY-----
------BEGIN CERTIFICATE-----
-MIIDhzCCAvCgAwIBAgIJALwngNFik7ONMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
-VQQGEwJjbjEQMA4GA1UECBMHYmVpamluZzEQMA4GA1UEBxMHYmVpamluZzEOMAwG
-A1UEChMFY25uaWMxDjAMBgNVBAsTBWNubmljMRMwEQYDVQQDEwp6aGFuZ2xpa3Vu
-MSIwIAYJKoZIhvcNAQkBFhN6aGFuZ2xpa3VuQGNubmljLmNuMB4XDTEwMDEwNzEy
-NDcxOFoXDTExMDEwNzEyNDcxOFowgYoxCzAJBgNVBAYTAmNuMRAwDgYDVQQIEwdi
-ZWlqaW5nMRAwDgYDVQQHEwdiZWlqaW5nMQ4wDAYDVQQKEwVjbm5pYzEOMAwGA1UE
-CxMFY25uaWMxEzARBgNVBAMTCnpoYW5nbGlrdW4xIjAgBgkqhkiG9w0BCQEWE3po
-YW5nbGlrdW5AY25uaWMuY24wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAOkg
-JbEkYoy9SEsU9t/mfxlaiCqNhxCqqgeodVEdiPKJ7LoVl21mRjazWBiHQbQ1e2Ka
-UiCJz68RwV7u92bIqe1bsNgNqoCPQqsQPtEoCPzfbiM1tIke0s/h6+8l6ne+yg21
-O825x5Anjq+6THLGCDcO4L2RWo+4PwJnVGrgBPKLAgMBAAGjgfIwge8wHQYDVR0O
-BBYEFJKM/O0ViGlwtb3JEci/DLTO/7DaMIG/BgNVHSMEgbcwgbSAFJKM/O0ViGlw
-tb3JEci/DLTO/7DaoYGQpIGNMIGKMQswCQYDVQQGEwJjbjEQMA4GA1UECBMHYmVp
-amluZzEQMA4GA1UEBxMHYmVpamluZzEOMAwGA1UEChMFY25uaWMxDjAMBgNVBAsT
-BWNubmljMRMwEQYDVQQDEwp6aGFuZ2xpa3VuMSIwIAYJKoZIhvcNAQkBFhN6aGFu
-Z2xpa3VuQGNubmljLmNuggkAvCeA0WKTs40wDAYDVR0TBAUwAwEB/zANBgkqhkiG
-9w0BAQUFAAOBgQBh5N6isMAQAFFD+pbfpppjQlO4vUNcEdzPdeuBFaf9CsX5ZdxV
-jmn1ZuGm6kRzqUPwPSxvCIAY0wuSu1g7YREPAZ3XBVwcg6262iGOA6n7E+nv5PLz
-EuZ1oUg+IfykUIoflKH6xZB4MyPL+EgkMT+i9BrngaXHXF8tEO30YppMiA==
------END CERTIFICATE-----

src/bin/bindctl/cmdparse.py

@@ -24,15 +24,19 @@ except ImportError:
     from bindctl.mycollections import OrderedDict
 
 param_name_str = "^\s*(?P<param_name>[\w]+)\s*=\s*"
-param_value_str = "(?P<param_value>[\w\.:/-]+)"
-param_value_with_quota_str = "[\"\'](?P<param_value>[\w\.:, /-]+)[\"\']"
+
+# The value string can be a sequence without space or comma 
+# characters, or a string surroundedby quotation marks(such marks
+# can be part of string in an escaped form)
+#param_value_str  = "(?P<param_value>[\"\'].+?(?<!\\\)[\"\']|[^\'\"][^, ]+)"
+param_value_str  = "(?P<param_value>[^\'\" ][^, ]+)"
+param_value_with_quota_str  = "[\"\'](?P<param_value>.+?)(?<!\\\)[\"\']"
 next_params_str = "(?P<blank>\s*)(?P<comma>,?)(?P<next_params>.*)$"
 
 PARAM_WITH_QUOTA_PATTERN = re.compile(param_name_str + 
-                                      param_value_with_quota_str +
+                                      param_value_with_quota_str + 
                                       next_params_str)
 PARAM_PATTERN = re.compile(param_name_str + param_value_str + next_params_str)
-                           
 # Used for module and command name
 NAME_PATTERN = re.compile("^\s*(?P<name>[\w]+)(?P<blank>\s*)(?P<others>.*)$")
 
@@ -98,7 +102,6 @@ class BindCmdParse:
                 
             groups = PARAM_PATTERN.match(param_text) or \
                      PARAM_WITH_QUOTA_PATTERN.match(param_text)
-            
             if not groups:
                 # ok, fill in the params in the order entered
                 params = re.findall("([^\" ]+|\".*\")", param_text)

src/bin/bindctl/exception.py

@@ -115,3 +115,10 @@ class CmdMissParamSyntaxError(CmdSyntaxError):
     def __str__(self):
         return str("Parameter '%s' is missed for command '%s' of module '%s'" % 
                    (self.param, self.command, self.module))
+
+
+class FailToLogin(BindCtlException):
+    def __str__(self):
+        return "Fail to login to cmdctl"
+
+

src/bin/bindctl/tests/bindctl_test.py

@@ -16,6 +16,7 @@
 
 import unittest
 import isc.cc.data
+import os
 from bindctl import cmdparse
 from bindctl import bindcmd
 from bindctl.moduleinfo import *
@@ -50,12 +51,31 @@ class TestCmdLex(unittest.TestCase):
             assert cmd.params["zone_name"] == "cnnic.cn"
             assert cmd.params["file"] == "cnnic.cn.file"
             assert cmd.params["master"] == '1.1.1.1'
+
+    def testCommandWithParamters_2(self):
+        '''Test whether the parameters in key=value can be parsed properly.'''
+        cmd = cmdparse.BindCmdParse('zone cmd name = 1:34::2')
+        self.assertEqual(cmd.params['name'], '1:34::2')
+
+        cmd = cmdparse.BindCmdParse('zone cmd name = 1\"\'34**&2 value=44\"\'\"')
+        self.assertEqual(cmd.params['name'], '1\"\'34**&2')
+        self.assertEqual(cmd.params['value'], '44\"\'\"')
+
+        cmd = cmdparse.BindCmdParse('zone cmd name = 1\"\'34**&2 ,value=  44\"\'\"')
+        self.assertEqual(cmd.params['name'], '1\"\'34**&2')
+        self.assertEqual(cmd.params['value'], '44\"\'\"')
             
+        cmd = cmdparse.BindCmdParse('zone cmd name =  1\'34**&2value=44\"\'\" value = \"==============\'')
+        self.assertEqual(cmd.params['name'], '1\'34**&2value=44\"\'\"')
+        self.assertEqual(cmd.params['value'], '==============')
+
+        cmd = cmdparse.BindCmdParse('zone cmd name =    \"1234, 567890 \" value ==&*/')
+        self.assertEqual(cmd.params['name'], '1234, 567890 ')
+        self.assertEqual(cmd.params['value'], '=&*/')
             
     def testCommandWithListParam(self):
-            cmd = cmdparse.BindCmdParse("zone set zone_name='cnnic.cn', master='1.1.1.1, 2.2.2.2'")
-            assert cmd.params["master"] == '1.1.1.1, 2.2.2.2'            
-        
+        cmd = cmdparse.BindCmdParse("zone set zone_name='cnnic.cn', master='1.1.1.1, 2.2.2.2'")
+        assert cmd.params["master"] == '1.1.1.1, 2.2.2.2'            
         
     def testCommandWithHelpParam(self):
         cmd = cmdparse.BindCmdParse("zone add help")
@@ -217,8 +237,37 @@ class TestNameSequence(unittest.TestCase):
             assert self.random_names[i] == cmd_names[i+1]
             assert self.random_names[i] == module_names[i+1]
             i = i + 1
-        
+
+    def test_apply_cfg_command(self):
+        self.tool.location = '/'
+        cmd = cmdparse.BindCmdParse("config set identifier=\"foo/bar\" value=\"5\"")
+        self.tool.apply_config_cmd(cmd)
     
+class FakeBindCmdInterpreter(bindcmd.BindCmdInterpreter):
+    def __init__(self):
+        pass
+
+class TestBindCmdInterpreter(unittest.TestCase):
+
+    def _create_invalid_csv_file(self, csvfilename):
+        import csv
+        csvfile = open(csvfilename, 'w')
+        writer = csv.writer(csvfile)
+        writer.writerow(['name1'])
+        writer.writerow(['name2'])
+        csvfile.close()
+
+    def test_get_saved_user_info(self):
+        cmd = FakeBindCmdInterpreter()
+        users = cmd._get_saved_user_info('/notexist', 'cvs_file.cvs')
+        self.assertEqual([], users)
+        
+        csvfilename = 'csv_file.csv'
+        self._create_invalid_csv_file(csvfilename)
+        users = cmd._get_saved_user_info('./', csvfilename)
+        self.assertEqual([], users)
+        os.remove(csvfilename)
+
 if __name__== "__main__":
     unittest.main()
     

src/bin/cfgmgr/Makefile.am

@@ -1,8 +1,10 @@
+SUBDIRS = tests
+
 pkglibexecdir = $(libexecdir)/@PACKAGE@
 
 pkglibexec_SCRIPTS = b10-cfgmgr
 
-CLEANFILES = b10-cfgmgr
+CLEANFILES = b10-cfgmgr b10-cfgmgr.pyc
 
 b10_cfgmgrdir = @localstatedir@/@PACKAGE@
 #B10_cfgmgr_DATA = 

src/bin/cfgmgr/b10-cfgmgr.py.in

@@ -40,7 +40,8 @@ def signal_handler(signal, frame):
     if cm:
         cm.running = False
 
-if __name__ == "__main__":
+def main():
+    global cm
     try:
         cm = ConfigManager(DATA_PATH)
         signal.signal(signal.SIGINT, signal_handler)
@@ -55,3 +56,6 @@ if __name__ == "__main__":
         print("[b10-cfgmgr] Interrupted, exiting")
     if cm:
         cm.write_config()
+
+if __name__ == "__main__":
+    main()

src/bin/cfgmgr/tests/Makefile.am

@@ -0,0 +1,13 @@
+PYTESTS = b10-cfgmgr_test.py
+
+EXTRA_DIST = $(PYTESTS)
+
+# later will have configure option to choose this, like: coverage run --branch
+PYCOVERAGE = $(PYTHON)
+# test using command-line arguments, so use check-local target instead of TESTS
+check-local:
+	for pytest in $(PYTESTS) ; do \
+	echo Running test: $$pytest ; \
+	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/cfgmgr \
+	$(PYCOVERAGE) $(abs_builddir)/$$pytest ; \
+	done

src/bin/cfgmgr/tests/b10-cfgmgr_test.py.in

@@ -0,0 +1,95 @@
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# $Id: cfgmgr_test.py 2126 2010-06-16 14:40:22Z jelte $
+
+#
+# Tests for the configuration manager run script
+#
+
+import unittest
+import os
+import sys
+
+class MyConfigManager:
+    def __init__(self, path):
+        self._path = path
+        self.read_config_called = False
+        self.notify_boss_called = False
+        self.run_called = False
+        self.write_config_called = False
+        self.running = True
+
+    def read_config(self):
+        self.read_config_called = True
+
+    def notify_boss(self):
+        self.notify_boss_called = True
+
+    def run(self):
+        self.run_called = True
+
+    def write_config(self):
+        self.write_config_called = True
+
+class TestConfigManagerStartup(unittest.TestCase):
+    def test_cfgmgr(self):
+        # some creative module use;
+        # b10-cfgmgr has a hypen, so we use __import__
+        # this also gives us the chance to override the imported
+        # module ConfigManager in it.
+        b = __import__("b10-cfgmgr")
+        b.ConfigManager = MyConfigManager
+
+        b.main()
+
+        self.assertTrue(b.cm.read_config_called)
+        self.assertTrue(b.cm.notify_boss_called)
+        self.assertTrue(b.cm.run_called)
+        self.assertTrue(b.cm.write_config_called)
+
+        self.assertTrue(b.cm.running)
+        b.signal_handler(None, None)
+        self.assertFalse(b.cm.running)
+
+        # TODO: take value from the 'global config module'
+        # (and rename the .in away from this file again)
+        data_path = "@localstatedir@/@PACKAGE@".replace("${prefix}", "@prefix@")
+        self.assertEqual(data_path, b.DATA_PATH)
+
+        # remove the 'module' again, or later tests may fail
+        # (if it is already present it won't be loaded again)
+        sys.modules.pop("b10-cfgmgr")
+
+    def test_cfgmgr_from_source(self):
+        tmp_env_var = "/just/some/dir"
+        env_var = None
+        if "B10_FROM_SOURCE" in os.environ:
+            env_var = os.environ["B10_FROM_SOURCE"]
+
+        os.environ["B10_FROM_SOURCE"] = tmp_env_var
+        b = __import__("b10-cfgmgr", globals(), locals())
+        b.ConfigManager = MyConfigManager
+        self.assertEqual(tmp_env_var, b.DATA_PATH)
+
+        if env_var != None:
+            os.environ["B10_FROM_SOURCE"] = env_var
+
+        sys.modules.pop("b10-cfgmgr")
+
+
+if __name__ == '__main__':
+    unittest.main()
+

src/bin/cmdctl/Makefile.am

@@ -17,9 +17,8 @@ b10_cmdctl_DATA = $(CMDCTL_CONFIGURATIONS)
 b10_cmdctl_DATA += cmdctl.spec
  
 EXTRA_DIST = $(CMDCTL_CONFIGURATIONS)
-EXTRA_DIST += cmdctl.spec
 
-CLEANFILES=	b10-cmdctl cmdctl.pyc
+CLEANFILES=	b10-cmdctl cmdctl.pyc cmdctl.spec
 
 man_MANS = b10-cmdctl.8
 EXTRA_DIST += $(man_MANS) b10-cmdctl.xml
@@ -31,6 +30,9 @@ b10-cmdctl.8: b10-cmdctl.xml
 
 endif
 
+cmdctl.spec: cmdctl.spec.pre
+	$(SED) -e "s|@@SYSCONFDIR@@|$(sysconfdir)|" cmdctl.spec.pre >$@
+
 # TODO: does this need $$(DESTDIR) also?
 # this is done here since configure.ac AC_OUTPUT doesn't expand exec_prefix
 b10-cmdctl: cmdctl.py

src/bin/cmdctl/TODO

@@ -1,8 +1,7 @@
-. Refine code for b10-cmdctl.
-. Add value type check according module specification.
 . Add return code for RESTful API document of b10-cmdctl.
-. Add more unit tests for b10-cmdctl.
 . Update man page for b10-cmdctl?
-
-. Add id to each command, so the receiver knows if the response is what it wants.
-. Make cmdctl can be configured through bindctl.(after id is added)
+. Add check for the content of key/certificate file
+  (when cmdctl starts or is configured by bindctl).
+. Use only one msgq/session to communicate with other modules?
+. Add more test cases, especially about the cases where CmdctlException
+  is raised

src/bin/cmdctl/cmdctl.py.in

@@ -41,6 +41,7 @@ import csv
 import random
 import time
 import signal
+from isc.config import ccsession
 from optparse import OptionParser, OptionValueError
 from hashlib import sha1
 try:
@@ -50,29 +51,27 @@ except ImportError:
 
 __version__ = 'BIND10'
 URL_PATTERN = re.compile('/([\w]+)(?:/([\w]+))?/?')
+CONFIG_DATA_URL = 'config_data'
+MODULE_SPEC_URL = 'module_spec'
 
-# If B10_FROM_SOURCE is set in the environment, we use data files
+
+# If B10_FROM_BUILD is set in the environment, we use data files
 # from a directory relative to that, otherwise we use the ones
 # installed on the system
-if "B10_FROM_SOURCE" in os.environ:
-    SPECFILE_PATH = os.environ["B10_FROM_SOURCE"] + "/src/bin/cmdctl"
-    SYSCONF_PATH = os.environ["B10_FROM_SOURCE"] + "/src/bin/cmdctl"
+if "B10_FROM_BUILD" in os.environ:
+    SPECFILE_PATH = os.environ["B10_FROM_BUILD"] + "/src/bin/cmdctl"
 else:
     PREFIX = "@prefix@"
     DATAROOTDIR = "@datarootdir@"
     SPECFILE_PATH = "@datadir@/@PACKAGE@".replace("${datarootdir}", DATAROOTDIR).replace("${prefix}", PREFIX)
-    SYSCONF_PATH = "@sysconfdir@/@PACKAGE@".replace("${prefix}", PREFIX)
 SPECFILE_LOCATION = SPECFILE_PATH + os.sep + "cmdctl.spec"
-USER_INFO_FILE = SYSCONF_PATH + os.sep + "cmdctl-accounts.csv"
-PRIVATE_KEY_FILE = SYSCONF_PATH + os.sep + "cmdctl-keyfile.pem"
-CERTIFICATE_FILE = SYSCONF_PATH + os.sep + "cmdctl-certfile.pem"
-        
+
+class CmdctlException(Exception):
+    pass
+       
 class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
     '''https connection request handler.
-    Currently only GET and POST are supported.
-
-    '''
-
+    Currently only GET and POST are supported.  '''
     def do_GET(self):
         '''The client should send its session id in header with 
         the name 'cookie'
@@ -167,13 +166,13 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
         user_name = user_info.get('username')
         if not user_name:
             return False, ["need user name"]
-        if not self.server.user_infos.get(user_name):
+        if not self.server.get_user_info(user_name):
             return False, ["user doesn't exist"]
 
         user_pwd = user_info.get('password')
         if not user_pwd:
             return False, ["need password"]
-        local_info = self.server.user_infos.get(user_name)
+        local_info = self.server.get_user_info(user_name)
         pwd_hashval = sha1((user_pwd + local_info[1]).encode())
         if pwd_hashval.hexdigest() != local_info[0]:
             return False, ["password doesn't match"] 
@@ -197,9 +196,6 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
                 pass
 
         rcode, reply = self.server.send_command_to_module(mod, cmd, param)
-        if self.server._verbose:
-            print('[b10-cmdctl] Finish send message \'%s\' to module %s' % (cmd, mod))
-
         ret = http.client.OK
         if rcode != 0:
             ret = http.client.BAD_REQUEST
@@ -214,65 +210,167 @@ class CommandControl():
     '''Get all modules' config data/specification from configmanager.
     receive command from client and resend it to proper module.
     '''
-
-    def __init__(self, verbose = False):
+    def __init__(self, httpserver, verbose = False):
+        ''' httpserver: the http server which use the object of
+        CommandControl to communicate with other modules. '''
         self._verbose = verbose
-        self.cc = isc.cc.Session()
-        self.cc.group_subscribe('Cmd-Ctrld')
-        self.module_spec = self.get_module_specification()
-        self.config_data = self.get_config_data()
+        self._httpserver = httpserver
+        self._lock = threading.Lock()
+        self._setup_session()
+        self.modules_spec = self._get_modules_specification()
+        self._config_data = self._get_config_data_from_config_manager()
+        self._serving = True
+        self._start_msg_handle_thread()
+
+    def _setup_session(self):
+        '''Setup the session for receving the commands
+        sent from other modules. There are two sessions 
+        for cmdctl, one(self.module_cc) is used for receiving 
+        commands sent from other modules, another one (self._cc) 
+        is used to send the command from Bindctl or other tools 
+        to proper modules.''' 
+        self._cc = isc.cc.Session()
+        self._module_cc = isc.config.ModuleCCSession(SPECFILE_LOCATION,
+                                              self.config_handler,
+                                              self.command_handler)
+        self._module_name = self._module_cc.get_module_spec().get_module_name()
+        self._cmdctl_config_data = self._module_cc.get_full_config()
+        self._module_cc.start()
+    
+    def _accounts_file_check(self, filepath):
+        ''' Check whether the accounts file is valid, each row
+        should be a list with 3 items.'''
+        csvfile = None
+        errstr = None
+        try:
+            csvfile = open(filepath)
+            reader = csv.reader(csvfile)
+            for row in reader:
+                a = (row[0], row[1], row[2])
+        except (IOError, IndexError) as e:
+            errstr = 'Invalid accounts file: ' + str(e)
+        finally:
+            if csvfile:
+                csvfile.close()
+
+        return errstr
+
+    def _config_data_check(self, new_config):
+        ''' Check whether the new config data is valid or
+        not. '''
+        errstr = None
+        for key in new_config:
+            if key == 'version':
+                continue
+            elif key in ['key_file', 'cert_file']:
+                #TODO, only check whether the file exist,
+                # further check need to be done: eg. whether
+                # the private/certificate is valid.
+                path = new_config[key]
+                if not os.path.exists(path):
+                    errstr = "the file doesn't exist: " + path
+            elif key == 'accounts_file':
+                errstr = self._accounts_file_check(new_config[key])
+            else:
+                errstr = 'unknown config item: ' + key
+            
+            if errstr != None:
+                self.log_info('Fail to apply config data, ' + errstr) 
+                return ccsession.create_answer(1, errstr)
+
+        return ccsession.create_answer(0)
 
+    def config_handler(self, new_config):
+        answer = self._config_data_check(new_config)
+        rcode, val = ccsession.parse_answer(answer)
+        if rcode != 0:
+            return answer
+
+        with self._lock:
+            for key in new_config:
+                if key in self._cmdctl_config_data:
+                    self._cmdctl_config_data[key] = new_config[key]
+        return answer
+
+    def command_handler(self, command, args):
+        answer = ccsession.create_answer(0)
+        if command == ccsession.COMMAND_MODULE_SPECIFICATION_UPDATE:
+            with self._lock:
+                self.modules_spec[args[0]] = args[1]
+
+        elif command == ccsession.COMMAND_SHUTDOWN:
+            #When cmdctl get 'shutdown' command from boss, 
+            #shutdown the outer httpserver.
+            self._httpserver.shutdown()
+            self._serving = False
+
+        elif command == 'print_settings':
+            answer = ccsession.create_answer(0, self._cmdctl_config_data)
+        else:
+            answer = ccsession.create_answer(1, 'unknown command: ' + command)
+
+        return answer
+
+    def _start_msg_handle_thread(self):
+        ''' Start one thread to handle received message from msgq.'''
+        td = threading.Thread(target=self._handle_msg_from_msgq)
+        td.daemon = True
+        td.start()
+
+    def _handle_msg_from_msgq(self):
+        '''Process all the received commands with module session. '''
+        while self._serving:
+            self._module_cc.check_command() 
+ 
     def _parse_command_result(self, rcode, reply):
         '''Ignore the error reason when command rcode isn't 0, '''
         if rcode != 0:
             return {}
         return reply
 
-    def get_config_data(self):
+    def _get_config_data_from_config_manager(self):
         '''Get config data for all modules from configmanager '''
-        rcode, reply = self.send_command('ConfigManager', isc.config.ccsession.COMMAND_GET_CONFIG)
+        rcode, reply = self.send_command('ConfigManager', ccsession.COMMAND_GET_CONFIG)
         return self._parse_command_result(rcode, reply)
 
-
-    def update_config_data(self, module_name, command_name):
+    def _update_config_data(self, module_name, command_name):
         '''Get lastest config data for all modules from configmanager '''
-        if module_name == 'ConfigManager' and command_name == isc.config.ccsession.COMMAND_SET_CONFIG:
-            self.config_data = self.get_config_data()
+        if module_name == 'ConfigManager' and command_name == ccsession.COMMAND_SET_CONFIG:
+            data = self._get_config_data_from_config_manager()
+            with self._lock:
+                self._config_data = data
 
-    def get_module_specification(self):
-        rcode, reply = self.send_command('ConfigManager', isc.config.ccsession.COMMAND_GET_MODULE_SPEC)
+    def get_config_data(self):
+        with self._lock:
+            data = self._config_data
+        return data
+
+    def get_modules_spec(self):
+        with self._lock:
+            spec = self.modules_spec
+        return spec
+
+    def _get_modules_specification(self):
+        '''Get all the modules' specification files. '''
+        rcode, reply = self.send_command('ConfigManager', ccsession.COMMAND_GET_MODULE_SPEC)
         return self._parse_command_result(rcode, reply)
 
-    def handle_recv_msg(self):
-        '''Handle received message, if 'shutdown' is received, return False'''
-        (message, env) = self.cc.group_recvmsg(True)
-        command, arg = isc.config.ccsession.parse_command(message)
-        while command:
-            if command == isc.config.ccsession.COMMAND_MODULE_SPECIFICATION_UPDATE:
-                self.module_spec[arg[0]] = arg[1]
-            elif command == isc.config.ccsession.COMMAND_SHUTDOWN:
-                return False
-            (message, env) = self.cc.group_recvmsg(True)
-            command, arg = isc.config.ccsession.parse_command(message)
-        
-        return True
-    
     def send_command_with_check(self, module_name, command_name, params = None):
         '''Before send the command to modules, check if module_name, command_name
         parameters are legal according the spec file of the module.
-        Return rcode, dict.
+        Return rcode, dict. TODO, the rcode should be defined properly.
         rcode = 0: dict is the correct returned value.
         rcode > 0: dict is : { 'error' : 'error reason' }
         '''
-
         # core module ConfigManager does not have a specification file
         if module_name == 'ConfigManager':
             return self.send_command(module_name, command_name, params)
 
-        if module_name not in self.module_spec.keys():
+        specs = self.get_modules_spec()
+        if module_name not in specs.keys():
             return 1, {'error' : 'unknown module'}
        
-        spec_obj = isc.config.module_spec.ModuleSpec(self.module_spec[module_name], False)
+        spec_obj = isc.config.module_spec.ModuleSpec(specs[module_name], False)
         errors = []
         if not spec_obj.validate_command(command_name, params, errors):
             return 1, {'error': errors[0]}
@@ -281,123 +379,161 @@ class CommandControl():
 
     def send_command(self, module_name, command_name, params = None):
         '''Send the command from bindctl to proper module. '''
-        
-        errstr = 'no error'
+        errstr = 'unknown error'
         if self._verbose:
-            self.log_info('[b10-cmdctl] send command \'%s\' to %s\n' %(command_name, module_name))
-        try:
-            msg = isc.config.ccsession.create_command(command_name, params)
-            seq = self.cc.group_sendmsg(msg, module_name)
+            self.log_info("Begin send command '%s' to module '%s'" %(command_name, module_name))
+
+        if module_name == self._module_name:
+            # Process the command sent to cmdctl directly. 
+            answer = self.command_handler(command_name, params)
+        else:
+            msg = ccsession.create_command(command_name, params)
+            seq = self._cc.group_sendmsg(msg, module_name)
             #TODO, it may be blocked, msqg need to add a new interface waiting in timeout.
-            answer, env = self.cc.group_recvmsg(False, seq)
-            if answer:
-                try:
-                    rcode, arg = isc.config.ccsession.parse_answer(answer)
-                    if rcode == 0:
-                        self.update_config_data(module_name, command_name)
-                        if arg != None:
-                            return rcode, arg
-                        else:
-                            return rcode, {}
+            answer, env = self._cc.group_recvmsg(False, seq)
+
+        if self._verbose:
+            self.log_info("Finish send command '%s' to module '%s'" % (command_name, module_name))
+
+        if answer:
+            try:
+                rcode, arg = ccsession.parse_answer(answer)
+                if rcode == 0:
+                    self._update_config_data(module_name, command_name)
+                    if arg != None:
+                        return rcode, arg
                     else:
-                        # todo: exception
-                        errstr = str(answer['result'][1])
-                except isc.config.ccsession.ModuleCCSessionError as mcse:
-                    errstr = str("Error in ccsession answer:") + str(mcse)
-                    self.log_info(answer)
-        except Exception as e:
-            errstr = str(e)
-            self.log_info('\'%s\':[b10-cmdctl] fail send command \'%s\' to %s\n' % (e, command_name, module_name))
+                        return rcode, {}
+                else:
+                    # TODO: exception
+                    errstr = str(answer['result'][1])
+            except ccsession.ModuleCCSessionError as mcse:
+                errstr = str("Error in ccsession answer:") + str(mcse)
+                self.log_info(errstr)
         
         return 1, {'error': errstr}
     
     def log_info(self, msg):
         sys.stdout.write("[b10-cmdctl] %s\n" % str(msg))
 
+    def get_cmdctl_config_data(self):
+        ''' If running in source code tree, use keyfile, certificate
+        and user accounts file in source code. '''
+        if "B10_FROM_SOURCE" in os.environ:
+            sysconf_path = os.environ["B10_FROM_SOURCE"] + "/src/bin/cmdctl/"
+            accountsfile  = sysconf_path + "cmdctl-accounts.csv"
+            keyfile = sysconf_path + "cmdctl-keyfile.pem"
+            certfile = sysconf_path + "cmdctl-certfile.pem"
+            return (keyfile, certfile, accountsfile)
+
+        with self._lock:
+            keyfile = self._cmdctl_config_data.get('key_file')
+            certfile = self._cmdctl_config_data.get('cert_file')
+            accountsfile = self._cmdctl_config_data.get('accounts_file')
+
+        return (keyfile, certfile, accountsfile)
+
 class SecureHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
     '''Make the server address can be reused.'''
     allow_reuse_address = True
 
-    def __init__(self, server_address, RequestHandlerClass, idle_timeout = 1200, verbose = False):
+    def __init__(self, server_address, RequestHandlerClass, 
+                 CommandControlClass,
+                 idle_timeout = 1200, verbose = False):
         '''idle_timeout: the max idle time for login'''
-        http.server.HTTPServer.__init__(self, server_address, RequestHandlerClass)
+        try:
+            http.server.HTTPServer.__init__(self, server_address, RequestHandlerClass)
+        except socket.error as err:
+            raise CmdctlException("Error creating server, because: %s \n" % str(err))
+
         self.user_sessions = {}
         self.idle_timeout = idle_timeout
-        self.cmdctrl = CommandControl()
-        self.__is_shut_down = threading.Event()
-        self.__serving = False
+        self.cmdctl = CommandControlClass(self, verbose)
         self._verbose = verbose
-        self.user_infos = {}
-        self._read_user_info()
+        self._lock = threading.Lock()
+        self._user_infos = {}
+        self._accounts_file = None
+
+    def _create_user_info(self, accounts_file):
+        '''Read all user's name and its' salt, hashed password 
+        from accounts file.'''
+        if (self._accounts_file == accounts_file) and (len(self._user_infos) > 0): 
+            return
+
+        with self._lock:
+            self._user_infos = {}
+            csvfile = None
+            try:
+                csvfile = open(accounts_file)
+                reader = csv.reader(csvfile)
+                for row in reader:
+                    self._user_infos[row[0]] = [row[1], row[2]]
+            except (IOError, IndexError) as e:
+                self.log_info("Fail to read user database, %s" % e)                
+            finally:
+                if csvfile:
+                    csvfile.close()
+
+        self._accounts_file = accounts_file
+        if len(self._user_infos) == 0:
+            self.log_info("Fail to get user information, will deny any user")                
+         
+    def get_user_info(self, username):
+        '''Get user's salt and hashed string. If the user
+        doesn't exist, return None, or else, the list 
+        [salt, hashed password] will be returned.'''
+        with self._lock:
+            info = self._user_infos.get(username)
+        return info
 
-    def _read_user_info(self):
-        '''Read all user's name and its' password from csv file.'''
-        csvfile = None
-        try:
-            csvfile = open(USER_INFO_FILE)
-            reader = csv.reader(csvfile)
-            for row in reader:
-                self.user_infos[row[0]] = [row[1], row[2]]
-        except Exception as e:
-            self.log_info('[b10-cmdctl] Fail to read user information :\'%s\'\n' % e)                
-        finally:
-            if csvfile:
-                csvfile.close()
-        
     def save_user_session_id(self, session_id):
-        # Record user's id and login time.
+        ''' Record user's id and login time. '''
         self.user_sessions[session_id] = time.time()
         
-    def get_request(self):
-        '''Get client request socket and wrap it in SSL context. '''
-        newsocket, fromaddr = self.socket.accept()
+    def _check_key_and_cert(self, key, cert):
+        # TODO, check the content of key/certificate file 
+        if not os.path.exists(key):
+            raise CmdctlException("key file '%s' doesn't exist " % key)
+
+        if not os.path.exists(cert):
+            raise CmdctlException("certificate file '%s' doesn't exist " % cert)
+
+    def _wrap_socket_in_ssl_context(self, sock, key, cert):
         try:
-            connstream = ssl.wrap_socket(newsocket,
-                                     server_side = True,
-                                     certfile = CERTIFICATE_FILE,
-                                     keyfile = PRIVATE_KEY_FILE,
-                                     ssl_version = ssl.PROTOCOL_SSLv23)
-            return (connstream, fromaddr)
-        except ssl.SSLError as e :
-            self.log_info('[b10-cmdctl] deny client\'s invalid connection:\'%s\'\n' % e)
-            self.close_request(newsocket)
+            self._check_key_and_cert(key, cert)
+            ssl_sock = ssl.wrap_socket(sock,
+                                      server_side = True,
+                                      certfile = cert,
+                                      keyfile = key,
+                                      ssl_version = ssl.PROTOCOL_SSLv23)
+            return ssl_sock 
+        except (ssl.SSLError, CmdctlException) as err :
+            self.log_info("Deny client's connection because %s" % str(err))
+            self.close_request(sock)
             # raise socket error to finish the request
             raise socket.error
-            
+
+    def get_request(self):
+        '''Get client request socket and wrap it in SSL context. '''
+        key, cert, account_file = self.cmdctl.get_cmdctl_config_data()
+        self._create_user_info(account_file)
+        newsocket, fromaddr = self.socket.accept()
+        ssl_sock = self._wrap_socket_in_ssl_context(newsocket, key, cert)
+        return (ssl_sock, fromaddr)
 
     def get_reply_data_for_GET(self, id, module):
         '''Currently only support the following three url GET request '''
         rcode, reply = http.client.NO_CONTENT, []        
         if not module:
-            if id == 'config_data':
-               rcode, reply = http.client.OK, self.cmdctrl.config_data
-            elif id == 'module_spec':
-                rcode, reply = http.client.OK, self.cmdctrl.module_spec
+            if id == CONFIG_DATA_URL:
+                rcode, reply = http.client.OK, self.cmdctl.get_config_data()
+            elif id == MODULE_SPEC_URL:
+                rcode, reply = http.client.OK, self.cmdctl.get_modules_spec()
         
         return rcode, reply 
 
-        
-    def serve_forever(self, poll_interval = 0.5):
-        '''Start cmdctl as one tcp server. '''
-        self.__serving = True
-        self.__is_shut_down.clear()
-        while self.__serving:
-            if not self.cmdctrl.handle_recv_msg():
-                break
-
-            r, w, e = select.select([self], [], [], poll_interval)
-            if r:
-                self._handle_request_noblock()
-
-        self.__is_shut_down.set()
-    
-    def shutdown(self):
-        self.__serving = False
-        self.__is_shut_down.wait()
-
-
     def send_command_to_module(self, module_name, command_name, params):
-        return self.cmdctrl.send_command_with_check(module_name, command_name, params)
+        return self.cmdctl.send_command_with_check(module_name, command_name, params)
    
     def log_info(self, msg):
         sys.stdout.write("[b10-cmdctl] %s\n" % str(msg))
@@ -417,7 +553,8 @@ def run(addr = 'localhost', port = 8080, idle_timeout = 1200, verbose = False):
     ''' Start cmdctl as one https server. '''
     if verbose:
         sys.stdout.write("[b10-cmdctl] starting on %s port:%d\n" %(addr, port))
-    httpd = SecureHTTPServer((addr, port), SecureHTTPRequestHandler, idle_timeout, verbose)
+    httpd = SecureHTTPServer((addr, port), SecureHTTPRequestHandler, 
+                             CommandControl, idle_timeout, verbose)
     httpd.serve_forever()
 
 def check_port(option, opt_str, value, parser):
@@ -453,22 +590,24 @@ def set_cmd_options(parser):
     parser.add_option("-v", "--verbose", dest="verbose", action="store_true", default=False,
             help="display more about what is going on")
 
-
 if __name__ == '__main__':
+    set_signal_handler()
+    parser = OptionParser(version = __version__)
+    set_cmd_options(parser)
+    (options, args) = parser.parse_args()
+    result = 1                  # in case of failure
     try:
-        parser = OptionParser(version = __version__)
-        set_cmd_options(parser)
-        (options, args) = parser.parse_args()
-        set_signal_handler()
         run(options.addr, options.port, options.idle_timeout, options.verbose)
-    except isc.cc.SessionError as se:
+        result = 0
+    except isc.cc.SessionError as err:
         sys.stderr.write("[b10-cmdctl] Error creating b10-cmdctl, "
-                "is the command channel daemon running?\n")        
+                         "is the command channel daemon running?\n")        
     except KeyboardInterrupt:
-        sys.stderr.write("[b10-cmdctl] exit http server\n")
+        sys.stderr.write("[b10-cmdctl] exit from Cmdctl\n")
+    except CmdctlException as err:
+        sys.stderr.write("[b10-cmdctl] " + str(err) + "\n")
 
     if httpd:
         httpd.shutdown()
 
-
-
+    sys.exit(result)

src/bin/cmdctl/cmdctl.spec

@@ -6,41 +6,31 @@
       {
         "item_name": "key_file",
         "item_type": "string",
-        "item_optional": False,
-        "item_default": 'cmdctl-keyfile.pem'
+        "item_optional": false,
+        "item_default": "@@SYSCONFDIR@@/@PACKAGE@/cmdctl-keyfile.pem"
       },
       {
         "item_name": "cert_file",
         "item_type": "string",
-        "item_optional": False,
-        "item_default": 'cmdctl-certfile.pem'
+        "item_optional": false,
+        "item_default": "@@SYSCONFDIR@@/@PACKAGE@/cmdctl-certfile.pem"
       },
       {
         "item_name": "accounts_file",
         "item_type": "string",
-        "item_optional": False,
-        "item_default": 'cmdctl-accounts.csv'
+        "item_optional": false,
+        "item_default": "@@SYSCONFDIR@@/@PACKAGE@/cmdctl-accounts.csv"
       }
     ],
     "commands": [
       {
-        "command_name": "print_message",
-        "command_description": "Print the given message to stdout",
-        "command_args": [ {
-          "item_name": "message",
-          "item_type": "string",
-          "item_optional": False,
-          "item_default": ""
-        } ]
-      },
-      {
         "command_name": "print_settings",
         "command_description": "Print some_string and some_int to stdout",
         "command_args": []
       },
       {
         "command_name": "shutdown",
-        "command_description": "Shut down cmdctl",
+        "command_description": "shutdown cmdctl",
         "command_args": []
       }
     ]

src/bin/cmdctl/tests/Makefile.am

@@ -8,5 +8,7 @@ check-local:
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
 	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/cmdctl \
+	CMDCTL_SPEC_PATH=$(abs_top_builddir)/src/bin/cmdctl \
+	CMDCTL_SRC_PATH=$(abs_top_srcdir)/src/bin/cmdctl \
 	$(PYCOVERAGE) $(abs_srcdir)/$$pytest ; \
 	done

src/bin/cmdctl/tests/cmdctl_test.in

@@ -18,10 +18,10 @@
 PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
 export PYTHON_EXEC
 
-BINDCTL_TEST_PATH=@abs_top_srcdir@/src/bin/cmdctl/tests
+CMDCTL_TEST_PATH=@abs_top_srcdir@/src/bin/cmdctl/tests
 PYTHONPATH=@abs_top_srcdir@/src/bin/cmdctl
 export PYTHONPATH
 
-cd ${BINDCTL_TEST_PATH}
+cd ${CMDCTL_TEST_PATH}
 exec ${PYTHON_EXEC} -O cmdctl_test.py $*
 

src/bin/cmdctl/tests/cmdctl_test.py

@@ -16,8 +16,18 @@
 
 import unittest
 import socket
+import tempfile
+import sys
 from cmdctl import *
 
+SPEC_FILE_PATH = '..' + os.sep
+if 'CMDCTL_SPEC_PATH' in os.environ:
+    SPEC_FILE_PATH = os.environ['CMDCTL_SPEC_PATH'] + os.sep
+
+SRC_FILE_PATH = '..' + os.sep
+if 'CMDCTL_SRC_PATH' in os.environ:
+    SRC_FILE_PATH = os.environ['CMDCTL_SRC_PATH'] + os.sep
+
 # Rewrite the class for unittest.
 class MySecureHTTPRequestHandler(SecureHTTPRequestHandler):
     def __init__(self):
@@ -42,17 +52,20 @@ class MySecureHTTPRequestHandler(SecureHTTPRequestHandler):
         os.remove('tmp.file')
     
 
-class MySecureHTTPServer(SecureHTTPServer):
+class FakeSecureHTTPServer(SecureHTTPServer):
     def __init__(self):
         self.user_sessions = {}
+        self.cmdctl = FakeCommandControlForTestRequestHandler()
+        self._verbose = True 
+        self._user_infos = {}
         self.idle_timeout = 1200
-        self.cmdctrl = MyCommandControl()
-        self._verbose = False
+        self._lock = threading.Lock()
 
-class MyCommandControl(CommandControl):
+class FakeCommandControlForTestRequestHandler(CommandControl):
     def __init__(self):
-        self.config_data = {}
-        self.module_spec = {}
+        self._config_data = {}
+        self.modules_spec = {}
+        self._lock = threading.Lock()
 
     def send_command(self, mod, cmd, param):
         return 0, {}
@@ -60,14 +73,17 @@ class MyCommandControl(CommandControl):
 
 class TestSecureHTTPRequestHandler(unittest.TestCase):
     def setUp(self):
+        self.old_stdout = sys.stdout
+        sys.stdout = open(os.devnull, 'w')
         self.handler = MySecureHTTPRequestHandler()
-        self.handler.server = MySecureHTTPServer()
+        self.handler.server = FakeSecureHTTPServer()
         self.handler.server.user_sessions = {}
-        self.handler.server.user_infos = {}
+        self.handler.server._user_infos = {}
         self.handler.headers = {}
         self.handler.rfile = open("check.tmp", 'w+b')
 
     def tearDown(self):
+        sys.stdout = self.old_stdout
         self.handler.rfile.close()
         os.remove('check.tmp')
 
@@ -145,7 +161,7 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
     def test_check_user_name_and_pwd(self):
         self.handler.headers = {}
         ret, msg = self.handler._check_user_name_and_pwd()
-        self.assertTrue(ret == False)
+        self.assertFalse(ret)
         self.assertEqual(msg, ['invalid username or password'])
 
     def test_check_user_name_and_pwd_1(self):
@@ -154,9 +170,9 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
         self.handler.headers['Content-Length'] = len
         self.handler.rfile.seek(0, 0)
 
-        self.handler.server.user_infos['root'] = ['aa', 'aaa']
+        self.handler.server._user_infos['root'] = ['aa', 'aaa']
         ret, msg = self.handler._check_user_name_and_pwd()
-        self.assertTrue(ret == False)
+        self.assertFalse(ret)
         self.assertEqual(msg, ['password doesn\'t match'])
 
     def test_check_user_name_and_pwd_2(self):
@@ -166,7 +182,7 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
         self.handler.rfile.seek(0, 0)
 
         ret, msg = self.handler._check_user_name_and_pwd()
-        self.assertTrue(ret == False)
+        self.assertFalse(ret)
         self.assertEqual(msg, ['invalid username or password'])
 
     def test_check_user_name_and_pwd_3(self):
@@ -176,7 +192,7 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
         self.handler.rfile.seek(0, 0)
 
         ret, msg = self.handler._check_user_name_and_pwd()
-        self.assertTrue(ret == False)
+        self.assertFalse(ret)
         self.assertEqual(msg, ['need user name'])
 
     def test_check_user_name_and_pwd_4(self):
@@ -185,9 +201,9 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
         self.handler.headers['Content-Length'] = len
         self.handler.rfile.seek(0, 0)
 
-        self.handler.server.user_infos['root'] = ['aa', 'aaa']
+        self.handler.server._user_infos['root'] = ['aa', 'aaa']
         ret, msg = self.handler._check_user_name_and_pwd()
-        self.assertTrue(ret == False)
+        self.assertFalse(ret)
         self.assertEqual(msg, ['need password'])
 
     def test_check_user_name_and_pwd_5(self):
@@ -197,7 +213,7 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
         self.handler.rfile.seek(0, 0)
 
         ret, msg = self.handler._check_user_name_and_pwd()
-        self.assertTrue(ret == False)
+        self.assertFalse(ret)
         self.assertEqual(msg, ['user doesn\'t exist'])
 
     def test_do_POST(self):
@@ -247,8 +263,8 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
 
         self.handler.rfile.seek(0, 0)
         self.handler.path = '/module/command'
-        self.handler.server.cmdctrl.module_spec = {}
-        self.handler.server.cmdctrl.module_spec['module'] = self._gen_module_spec()
+        self.handler.server.cmdctl.modules_spec = {}
+        self.handler.server.cmdctl.modules_spec['module'] = self._gen_module_spec()
         rcode, reply = self.handler._handle_post_request()
         self.assertEqual(http.client.OK, rcode)
 
@@ -259,10 +275,178 @@ class TestSecureHTTPRequestHandler(unittest.TestCase):
 
         self.handler.rfile.seek(0, 0)
         self.handler.path = '/module/command'
-        self.handler.server.cmdctrl.module_spec = {}
-        self.handler.server.cmdctrl.module_spec['module'] = self._gen_module_spec()
+        self.handler.server.cmdctl.modules_spec = {}
+        self.handler.server.cmdctl.modules_spec['module'] = self._gen_module_spec()
         rcode, reply = self.handler._handle_post_request()
         self.assertEqual(http.client.BAD_REQUEST, rcode)
 
+class MyCommandControl(CommandControl):
+    def _get_modules_specification(self):
+        return {}
+
+    def _get_config_data_from_config_manager(self):
+        return {}
+
+    def _setup_session(self):
+        spec_file = SPEC_FILE_PATH + 'cmdctl.spec'
+        module_spec = isc.config.module_spec_from_file(spec_file)
+        config = isc.config.config_data.ConfigData(module_spec)
+        self._module_name = 'Cmdctl'
+        self._cmdctl_config_data = config.get_full_config()
+
+    def _handle_msg_from_msgq(self): 
+        pass
+
+class TestCommandControl(unittest.TestCase):
+
+    def setUp(self):
+        self.old_stdout = sys.stdout
+        sys.stdout = open(os.devnull, 'w')
+        self.cmdctl = MyCommandControl(None, True)
+   
+    def tearDown(self):
+        sys.stdout = self.old_stdout
+
+    def _check_config(self, cmdctl):
+        key, cert, account = cmdctl.get_cmdctl_config_data()
+        self.assertIsNotNone(key)
+        self.assertIsNotNone(cert)
+        self.assertIsNotNone(account)
+
+    def test_get_cmdctl_config_data(self):
+        old_env = os.environ
+        if 'B10_FROM_SOURCE' in os.environ:
+            del os.environ['B10_FROM_SOURCE']
+        self.cmdctl.get_cmdctl_config_data() 
+        self._check_config(self.cmdctl)
+        os.environ = old_env
+
+        old_env = os.environ
+        os.environ['B10_FROM_SOURCE'] = '../'
+        self._check_config(self.cmdctl)
+        os.environ = old_env
+    
+    def test_parse_command_result(self):
+        self.assertEqual({}, self.cmdctl._parse_command_result(1, {'error' : 1}))
+        self.assertEqual({'a': 1}, self.cmdctl._parse_command_result(0, {'a' : 1}))
+
+    def _check_answer(self, answer, rcode_, msg_):
+        rcode, msg = ccsession.parse_answer(answer)
+        self.assertEqual(rcode, rcode_)
+        self.assertEqual(msg, msg_)
+
+    def test_command_handler(self):
+        answer = self.cmdctl.command_handler('unknown-command', None)
+        self._check_answer(answer, 1, 'unknown command: unknown-command')
+
+        answer = self.cmdctl.command_handler('print_settings', None)
+        rcode, msg = ccsession.parse_answer(answer)
+        self.assertEqual(rcode, 0)
+        self.assertTrue(msg != None)
+
+    def test_check_config_handler(self):
+        answer = self.cmdctl.config_handler({'non-exist': 123})
+        self._check_answer(answer, 1, 'unknown config item: non-exist')
+
+        old_env = os.environ
+        os.environ['B10_FROM_SOURCE'] = '../'
+        self._check_config(self.cmdctl)
+        os.environ = old_env
+
+        answer = self.cmdctl.config_handler({'key_file': '/user/non-exist_folder'})
+        self._check_answer(answer, 1, "the file doesn't exist: /user/non-exist_folder")
+
+        answer = self.cmdctl.config_handler({'cert_file': '/user/non-exist_folder'})
+        self._check_answer(answer, 1, "the file doesn't exist: /user/non-exist_folder")
+
+        answer = self.cmdctl.config_handler({'accounts_file': '/user/non-exist_folder'})
+        self._check_answer(answer, 1, 
+                "Invalid accounts file: [Errno 2] No such file or directory: '/user/non-exist_folder'")
+
+        # Test with invalid accounts file
+        file_name = 'tmp.account.file'
+        temp_file = open(file_name, 'w')
+        writer = csv.writer(temp_file)
+        writer.writerow(['a', 'b'])
+        temp_file.close()
+        answer = self.cmdctl.config_handler({'accounts_file': file_name})
+        self._check_answer(answer, 1, "Invalid accounts file: list index out of range")
+        os.remove(file_name)
+    
+    def test_send_command(self):
+        rcode, value = self.cmdctl.send_command('Cmdctl', 'print_settings', None)
+        self.assertEqual(rcode, 0)
+
+class MySecureHTTPServer(SecureHTTPServer):
+    def server_bind(self):
+        pass
+
+class TestSecureHTTPServer(unittest.TestCase):
+    def setUp(self):
+        self.old_stdout = sys.stdout
+        self.old_stderr = sys.stderr
+        sys.stdout = open(os.devnull, 'w')
+        sys.stderr = sys.stdout
+        self.server = MySecureHTTPServer(('localhost', 8080), 
+                                         MySecureHTTPRequestHandler,
+                                         MyCommandControl, verbose=True)
+
+    def tearDown(self):
+        sys.stdout = self.old_stdout
+        sys.stderr = self.old_stderr
+
+    def test_addr_in_use(self):
+        server_one = None
+        try:
+            server_one = SecureHTTPServer(('localhost', 53531),
+                                        MySecureHTTPRequestHandler,
+                                        MyCommandControl)
+        except CmdctlException:
+            pass
+        else:
+            self.assertRaises(CmdctlException, SecureHTTPServer,
+                              ('localhost', 53531),
+                              MySecureHTTPRequestHandler, MyCommandControl)
+        if server_one:
+            server_one.server_close()
+
+    def test_create_user_info(self):
+        self.server._create_user_info('/local/not-exist')
+        self.assertEqual(0, len(self.server._user_infos))
+
+        self.server._create_user_info(SRC_FILE_PATH + 'cmdctl-accounts.csv')
+        self.assertEqual(1, len(self.server._user_infos))
+        self.assertTrue('root' in self.server._user_infos)
+
+    def test_check_key_and_cert(self):
+        self.assertRaises(CmdctlException, self.server._check_key_and_cert,
+                         '/local/not-exist', 'cmdctl-keyfile.pem')
+
+        self.server._check_key_and_cert(SRC_FILE_PATH + 'cmdctl-keyfile.pem',
+                                        SRC_FILE_PATH + 'cmdctl-certfile.pem')
+
+    def test_wrap_sock_in_ssl_context(self):
+        sock = socket.socket()
+        self.assertRaises(socket.error, 
+                          self.server._wrap_socket_in_ssl_context,
+                          sock, 
+                          '../cmdctl-keyfile',
+                          '../cmdctl-certfile')
+
+        sock1 = socket.socket()
+        self.server._wrap_socket_in_ssl_context(sock1, 
+                          SRC_FILE_PATH + 'cmdctl-keyfile.pem',
+                          SRC_FILE_PATH + 'cmdctl-certfile.pem')
+
+class TestFuncNotInClass(unittest.TestCase):
+    def test_check_port(self):
+        self.assertRaises(OptionValueError, check_port, None, 'port', -1, None)        
+        self.assertRaises(OptionValueError, check_port, None, 'port', 65536, None)        
+        self.assertRaises(OptionValueError, check_addr, None, 'ipstr', 'a.b.d', None)        
+        self.assertRaises(OptionValueError, check_addr, None, 'ipstr', '1::0:a.b', None)        
+
+
 if __name__== "__main__":
     unittest.main()
+
+

src/bin/host/Makefile.am

@@ -7,7 +7,7 @@ CLEANFILES = *.gcno *.gcda
 
 bin_PROGRAMS = host
 host_SOURCES = host.cc
-host_LDADD = $(top_builddir)/src/lib/dns/.libs/libdns.a
+host_LDADD = $(top_builddir)/src/lib/dns/.libs/libdns++.a
 host_LDADD += $(top_builddir)/src/lib/exceptions/.libs/libexceptions.a
 
 #man_MANS = host.1

src/bin/host/host.cc

@@ -19,17 +19,19 @@
 #include <sys/time.h>       // for gettimeofday
 #include <sys/socket.h>     // networking functions and definitions on FreeBSD
 
+#include <unistd.h>
+
 #include <string>
 #include <iostream>
 
-#include "dns/buffer.h"
-#include "dns/name.h"
-#include "dns/message.h"
-#include "dns/messagerenderer.h"
-#include "dns/rrclass.h"
-#include "dns/rrtype.h"
-#include "dns/rrset.h"
-#include "dns/message.h"
+#include <dns/buffer.h>
+#include <dns/name.h>
+#include <dns/message.h>
+#include <dns/messagerenderer.h>
+#include <dns/rrclass.h>
+#include <dns/rrtype.h>
+#include <dns/rrset.h>
+#include <dns/message.h>
 
 using namespace std;
 using namespace isc::dns;

src/bin/loadzone/Makefile.am

@@ -1,3 +1,5 @@
+SUBDIRS = tests/correct
+SUBDIRS += tests/error
 bin_SCRIPTS = b10-loadzone
 
 CLEANFILES = b10-loadzone
@@ -22,23 +24,27 @@ install-data-local:
 	$(mkinstalldirs) $(DESTDIR)/@localstatedir@/@PACKAGE@
 # TODO: permissions handled later
 
-EXTRA_DIST += testdata/README
-EXTRA_DIST += testdata/dsset-subzone.example.com.
-EXTRA_DIST += testdata/example.com
-EXTRA_DIST += testdata/example.com.signed
-EXTRA_DIST += testdata/Kexample.com.+005+04456.key
-EXTRA_DIST += testdata/Kexample.com.+005+04456.private
-EXTRA_DIST += testdata/Kexample.com.+005+33495.key
-EXTRA_DIST += testdata/Kexample.com.+005+33495.private
-EXTRA_DIST += testdata/Ksql1.example.com.+005+12447.key
-EXTRA_DIST += testdata/Ksql1.example.com.+005+12447.private
-EXTRA_DIST += testdata/Ksql1.example.com.+005+33313.key
-EXTRA_DIST += testdata/Ksql1.example.com.+005+33313.private
-EXTRA_DIST += testdata/Ksql2.example.com.+005+38482.key
-EXTRA_DIST += testdata/Ksql2.example.com.+005+38482.private
-EXTRA_DIST += testdata/Ksql2.example.com.+005+63192.key
-EXTRA_DIST += testdata/Ksql2.example.com.+005+63192.private
-EXTRA_DIST += testdata/sql1.example.com
-EXTRA_DIST += testdata/sql1.example.com.signed
-EXTRA_DIST += testdata/sql2.example.com
-EXTRA_DIST += testdata/sql2.example.com.signed
+EXTRA_DIST += tests/normal/README
+EXTRA_DIST += tests/normal/dsset-subzone.example.com
+EXTRA_DIST += tests/normal/example.com
+EXTRA_DIST += tests/normal/example.com.signed
+EXTRA_DIST += tests/normal/Kexample.com.+005+04456.key
+EXTRA_DIST += tests/normal/Kexample.com.+005+04456.private
+EXTRA_DIST += tests/normal/Kexample.com.+005+33495.key
+EXTRA_DIST += tests/normal/Kexample.com.+005+33495.private
+EXTRA_DIST += tests/normal/Ksql1.example.com.+005+12447.key
+EXTRA_DIST += tests/normal/Ksql1.example.com.+005+12447.private
+EXTRA_DIST += tests/normal/Ksql1.example.com.+005+33313.key
+EXTRA_DIST += tests/normal/Ksql1.example.com.+005+33313.private
+EXTRA_DIST += tests/normal/Ksql2.example.com.+005+38482.key
+EXTRA_DIST += tests/normal/Ksql2.example.com.+005+38482.private
+EXTRA_DIST += tests/normal/Ksql2.example.com.+005+63192.key
+EXTRA_DIST += tests/normal/Ksql2.example.com.+005+63192.private
+EXTRA_DIST += tests/normal/sql1.example.com
+EXTRA_DIST += tests/normal/sql1.example.com.signed
+EXTRA_DIST += tests/normal/sql2.example.com
+EXTRA_DIST += tests/normal/sql2.example.com.signed
+
+pytest:
+	$(SHELL) tests/correct/correct_test.sh
+	$(SHELL) tests/error/error_test.sh

src/bin/loadzone/b10-loadzone.py.in

@@ -19,7 +19,8 @@ import sys; sys.path.append ('@@PYTHONPATH@@')
 import re, getopt
 import isc.datasrc
 from isc.datasrc.master import MasterFile
-
+import time
+import os
 #########################################################################
 # usage: print usage note and exit
 #########################################################################
@@ -57,23 +58,32 @@ def main():
     if len(args) != 1:
         usage()
     zonefile = args[0]
-
+    verbose = os.isatty(sys.stdout.fileno())
     try:
-        master = MasterFile(zonefile, initial_origin)
+        master = MasterFile(zonefile, initial_origin, verbose)
     except Exception as e:
-        print("Error reading zone file: " + str(e))
+        sys.stderr.write("Error reading zone file: %s\n" % str(e))
         exit(1)
 
     try:
         zone = master.zonename()
+        if verbose:
+            sys.stdout.write("Using SQLite3 database file %s\n" % dbfile)
+            sys.stdout.write("Zone name is %s\n" % zone)
+            sys.stdout.write("Loading file \"%s\"\n" % zonefile)
     except Exception as e:
-        print("Error reading zone file: " + str(e))
+        sys.stdout.write("\n")
+        sys.stderr.write("Error reading zone file: %s\n" % str(e))
         exit(1)
 
     try:
         isc.datasrc.sqlite3_ds.load(dbfile, zone, master.zonedata)
+        if verbose:
+            master.closeverbose()
+            sys.stdout.write("\nDone.\n")
     except Exception as e:
-        print("Error loading database: " + str(e))
+        sys.stdout.write("\n")
+        sys.stderr.write("Error loading database: %s\n"% str(e))
         exit(1)
 
 if __name__ == "__main__":

src/bin/loadzone/run_loadzone.sh.in

@@ -21,5 +21,5 @@ export PYTHON_EXEC
 PYTHONPATH=@abs_top_builddir@/src/lib/python
 export PYTHONPATH
 
-LOADZONE_PATH=@abs_top_srcdir@/src/bin/loadzone
+LOADZONE_PATH=@abs_top_builddir@/src/bin/loadzone
 exec ${LOADZONE_PATH}/b10-loadzone $*

src/bin/loadzone/tests/correct/Makefile.am

@@ -0,0 +1,25 @@
+PYTESTS = correct_test.sh
+EXTRA_DIST = get_zonedatas.py
+EXTRA_DIST += include.db
+EXTRA_DIST += inclsub.db
+EXTRA_DIST += known.test.out
+EXTRA_DIST += mix1.db
+EXTRA_DIST += mix1sub1.db
+EXTRA_DIST += mix1sub2.db
+EXTRA_DIST += mix2.db
+EXTRA_DIST += mix2sub1.txt
+EXTRA_DIST += mix2sub2.txt
+EXTRA_DIST += ttl1.db
+EXTRA_DIST += ttl2.db
+EXTRA_DIST += ttlext.db
+EXTRA_DIST += example.db
+
+# later will have configure option to choose this, like: coverage run --branch
+PYCOVERAGE = $(PYTHON)
+# test using command-line arguments, so use check-local target instead of TESTS
+check-local:
+	for pytest in $(PYTESTS) ; do \
+	echo Running test: $$pytest ; \
+	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/loadzone \
+	$(SHELL) $(abs_builddir)/$$pytest ; \
+	done

src/bin/loadzone/tests/correct/correct_test.sh.in

@@ -0,0 +1,75 @@
+#! /bin/sh
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
+export PYTHON_EXEC
+
+PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python
+export PYTHONPATH
+
+LOADZONE_PATH=@abs_top_builddir@/src/bin/loadzone
+TEST_FILE_PATH=@abs_top_srcdir@/src/bin/loadzone/tests/correct
+TEST_OUTPUT_PATH=@abs_top_builddir@/src/bin/loadzone//tests/correct
+
+status=0
+echo "Loadzone include. from include.db file"
+cd ${TEST_FILE_PATH}
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 include.db >> /dev/null
+
+echo "loadzone  ttl1. from ttl1.db file"
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 ttl1.db >> /dev/null
+
+echo "loadzone ttl2. from ttl2.db file"
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 ttl2.db >> /dev/null
+
+echo "loadzone mix1. from mix1.db"
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 mix1.db >> /dev/null
+
+echo "loadzone mix2. from mix2.db"
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 mix2.db >> /dev/null
+
+echo "loadzone ttlext. from ttlext.db"
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 ttlext.db >> /dev/null
+
+echo "loadzone example.com. from example.db"
+${LOADZONE_PATH}/b10-loadzone -d ${TEST_OUTPUT_PATH}/zone.sqlite3 example.db >> /dev/null
+
+echo "I:test master file \$INCLUDE semantics"
+echo "I:test master file BIND 8 compatibility TTL and \$TTL semantics"
+echo "I:test master file RFC1035 TTL and \$TTL semantics"
+echo "I:test master file BIND8 compatibility and mixed \$INCLUDE with \$TTL semantics"
+echo "I:test master file RFC1035 TTL and mixed \$INCLUDE with \$TTL semantics"
+echo "I:test master file BIND9 extenstion of TTL"
+echo "I:test master file RFC1035 missing CLASS, TTL, NAME semantics"
+
+${PYTHON_EXEC} ${TEST_FILE_PATH}/get_zonedatas.py ${TEST_OUTPUT_PATH}/zone.sqlite3 > ${TEST_OUTPUT_PATH}/test.out
+echo "Compare test results."
+diff ${TEST_OUTPUT_PATH}/test.out ${TEST_FILE_PATH}/known.test.out || status=1
+
+echo "Clean tmp files."
+rm -f ${TEST_OUTPUT_PATH}/zone.sqlite3
+rm -f ${TEST_OUTPUT_PATH}/test.out
+echo "I:exit status: $status"
+echo "------------------------------------------------------------------------------"
+echo "Ran 7 test files"
+echo ""
+if [ "$status" -eq 1 ] ;then
+    echo "ERROR"
+else
+    echo "OK"
+fi
+exit $status

src/bin/loadzone/tests/correct/example.db

@@ -0,0 +1,14 @@
+;This file includes all kinds of rr form. missing name, ttl and class or not.
+$ORIGIN example.com.
+$TTL 60
+@    IN SOA   ns1.example.com. hostmaster.example.com. (1 43200 900 1814400 7200)
+     IN     20      NS  ns1
+                    NS  ns2
+ns1  IN     30      A   192.168.1.102
+            70      NS  ns3
+     IN             NS  ns4
+     10     IN      MX  10  mail.example.com.
+ns2         80      A   1.1.1.1
+ns3  IN             A   2.2.2.2
+ns4                 A   3.3.3.3
+ns5  90     IN      A   4.4.4.4

src/bin/loadzone/tests/correct/get_zonedatas.py

@@ -0,0 +1,8 @@
+from isc.datasrc import sqlite3_ds
+import sys
+ZONE_FILE = sys.argv[1]
+zonename_set = ["include.", "ttl1.", "ttl2.", "mix1.", "mix2.", "ttlext.", "example.com."]
+for zone_name in zonename_set:
+    for rr_data in sqlite3_ds.get_zone_datas(zone_name, ZONE_FILE):
+        data_len = len(rr_data[2])
+        sys.stdout.write(rr_data[2] + '\t\t' + str(rr_data[4]) + '\tIN\t' + rr_data[5] + '\t' + rr_data[7] + '\n')

src/bin/loadzone/tests/correct/inclsub.db

@@ -0,0 +1,4 @@
+a    300		A	10.0.1.1
+$ORIGIN foo
+b	 300     	A	10.0.2.2
+

src/bin/loadzone/tests/correct/include.db

@@ -0,0 +1,23 @@
+$ORIGIN include.   ; initialize origin
+$TTL 300
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+
+ns			A	127.0.0.1
+
+a			A	10.0.0.1
+$INCLUDE inclsub.db sub   ;  a.include. is the relative domain name origin for the included file
+; use the current domain name
+            A	99.99.99.99
+b			A	10.0.0.2
+$ORIGIN b
+$INCLUDE inclsub.db
+; use the current domain name
+			A	10.0.0.99
+c			A	10.0.0.3

src/bin/loadzone/tests/correct/known.test.out

@@ -0,0 +1,79 @@
+include.		300	IN	SOA	ns.include. hostmaster.include. 1 3600 1800 1814400 3600
+include.		300	IN	NS	ns.include.
+ns.include.		300	IN	A	127.0.0.1
+a.include.		300	IN	A	10.0.0.1
+a.sub.include.		300	IN	A	10.0.1.1
+b.foo.sub.include.		300	IN	A	10.0.2.2
+a.include.		300	IN	A	99.99.99.99
+b.include.		300	IN	A	10.0.0.2
+a.b.include.		300	IN	A	10.0.1.1
+b.foo.b.include.		300	IN	A	10.0.2.2
+b.include.		300	IN	A	10.0.0.99
+c.b.include.		300	IN	A	10.0.0.3
+ttl1.		3	IN	SOA	ns.ttl1. hostmaster.ttl1. 1 3600 1800 1814400 3
+ttl1.		3	IN	NS	ns.ttl1.
+ns.ttl1.		3	IN	A	10.53.0.1
+a.ttl1.		3	IN	TXT	"soa minttl 3"
+b.ttl1.		2	IN	TXT	"explicit ttl 2"
+c.ttl1.		3	IN	TXT	"soa minttl 3"
+d.ttl1.		1	IN	TXT	"default ttl 1"
+e.ttl1.		4	IN	TXT	"explicit ttl 4"
+f.ttl1.		1	IN	TXT	"default ttl 1"
+ttl2.		1	IN	SOA	ns.ttl2. hostmaster.ttl2. 1 3600 1800 1814400 3
+ttl2.		1	IN	NS	ns.ttl2.
+ns.ttl2.		1	IN	A	10.53.0.1
+a.ttl2.		1	IN	TXT	"inherited ttl 1"
+b.ttl2.		2	IN	TXT	"explicit ttl 2"
+c.ttl2.		2	IN	TXT	"inherited ttl 2"
+d.ttl2.		3	IN	TXT	"default ttl 3"
+e.ttl2.		2	IN	TXT	"explicit ttl 2"
+f.ttl2.		3	IN	TXT	"default ttl 3"
+mix1.		3	IN	SOA	ns.mix1. hostmaster.mix1. 1 3600 1800 1814400 3
+mix1.		3	IN	NS	ns.mix1.
+ns.mix1.		3	IN	A	10.53.0.1
+a.mix1.		3	IN	TXT	"soa minttl 3"
+b.mix1.		2	IN	TXT	"explicit ttl 2"
+a.mix1.		3	IN	TXT	"soa minttl 3"
+b.foo.mix1.		3	IN	TXT	"soa minttl 3"
+c.mix1.		3	IN	TXT	"soa minttl 3"
+d.mix1.		1	IN	TXT	"default ttl 1"
+e.mix1.		4	IN	TXT	"explicit ttl 4"
+f.mix1.		1	IN	TXT	"default ttl 1"
+i.mix1.		1	IN	TXT	"default ttl 1"
+g.mix1.		5	IN	TXT	"default ttl 5"
+h.mix1.		5	IN	TXT	"the include ttl 5"
+mix2.		1	IN	SOA	ns.mix2. hostmaster.mix2. 1 3600 1800 1814400 3
+mix2.		1	IN	NS	ns.mix2.
+ns.mix2.		1	IN	A	10.53.0.1
+a.mix2.		1	IN	TXT	"inherited ttl 1"
+h.mix2.		1	IN	TXT	"inherited ttl 1"
+g.mix2.		6	IN	TXT	"inherited ttl 6"
+b.mix2.		6	IN	TXT	"explicit ttl 6"
+c.mix2.		2	IN	TXT	"inherited ttl 2"
+m.mix2.		6	IN	TXT	"explicit ttl 6"
+d.mix2.		3	IN	TXT	"default ttl 3"
+e.mix2.		2	IN	TXT	"explicit ttl 2"
+n.mix2.		3	IN	TXT	"default ttl 3"
+f.mix2.		3	IN	TXT	"default ttl 3"
+g.mix2.		5	IN	TXT	"default ttl 5"
+f.mix2.		5	IN	TXT	"default ttl 5"
+ttlext.		3	IN	SOA	ns.ttlext. hostmaster.ttlext. 1 3600 1800 1814400 3
+ttlext.		3	IN	NS	ns.ttlext.
+ns.ttlext.		3	IN	A	10.53.0.1
+a.ttlext.		3	IN	TXT	"soa minttl 3"
+b.ttlext.		2	IN	TXT	"explicit ttl 2"
+c.ttlext.		3	IN	TXT	"soa minttl 3"
+d.ttlext.		600	IN	TXT	"default ttl 600"
+e.ttlext.		4	IN	TXT	"explicit ttl 4"
+f.ttlext.		600	IN	TXT	"default ttl 600"
+example.com.		60	IN	SOA	ns1.example.com. hostmaster.example.com. 1 43200 900 1814400 7200
+example.com.		20	IN	NS	ns1.example.com.
+example.com.		60	IN	NS	ns2.example.com.
+ns1.example.com.		30	IN	A	192.168.1.102
+ns1.example.com.		70	IN	NS	ns3.example.com.
+ns1.example.com.		60	IN	NS	ns4.example.com.
+ns1.example.com.		10	IN	MX	10 mail.example.com.
+ns2.example.com.		80	IN	A	1.1.1.1
+ns3.example.com.		60	IN	A	2.2.2.2
+ns4.example.com.		60	IN	A	3.3.3.3
+ns5.example.com.		90	IN	A	4.4.4.4

src/bin/loadzone/tests/correct/mix1.db

@@ -0,0 +1,21 @@
+; $Id: ttl1.db,v 1.6 2007/06/19 23:47:04 tbox Exp $
+$ORIGIN mix1.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3
+				)
+			NS	ns
+ns			A	10.53.0.1
+a			TXT	"soa minttl 3"
+b		2	TXT	"explicit ttl 2"
+$INCLUDE mix1sub1.db
+c			TXT	"soa minttl 3"
+$TTL 1
+d			TXT	"default ttl 1"
+e		4	TXT	"explicit ttl 4"
+f			TXT	"default ttl 1"
+$INCLUDE mix1sub2.db
+h       5   TXT "the include ttl 5"

src/bin/loadzone/tests/correct/mix1sub1.db

@@ -0,0 +1,3 @@
+a                       TXT      "soa minttl 3"
+$ORIGIN foo
+b                       TXT      "soa minttl 3"

src/bin/loadzone/tests/correct/mix1sub2.db

@@ -0,0 +1,3 @@
+i                       TXT      "default ttl 1"
+$TTL   5
+g                       TXT      "default ttl 5"

src/bin/loadzone/tests/correct/mix2.db

@@ -0,0 +1,21 @@
+$ORIGIN mix2.
+@		1	IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3
+				)
+			NS	ns
+ns			A	10.53.0.1
+a			TXT	"inherited ttl 1"
+$INCLUDE mix2sub1.txt
+b			TXT	"explicit ttl 6"
+c		2	TXT	"inherited ttl 2"
+m           TXT "explicit ttl 6"
+$TTL 3
+d			TXT	"default ttl 3"
+e		2	TXT	"explicit ttl 2"
+n           TXT "default ttl 3"
+$INCLUDE mix2sub2.txt
+f			TXT	"default ttl 5"

src/bin/loadzone/tests/correct/mix2sub1.txt

@@ -0,0 +1,3 @@
+h                       TXT     "inherited ttl 1"
+$TTL 6
+g                       TXT     "inherited ttl 6"

src/bin/loadzone/tests/correct/mix2sub2.txt

@@ -0,0 +1,3 @@
+f                       TXT     "default  ttl 3"
+$TTL 5
+g                       TXT     "default  ttl 5"

src/bin/loadzone/tests/correct/ttl1.db

@@ -0,0 +1,17 @@
+$ORIGIN ttl1.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3
+				)
+			NS	ns
+ns			A	10.53.0.1
+a			TXT	"soa minttl 3"
+b		2	TXT	"explicit ttl 2"
+c			TXT	"soa minttl 3"
+$TTL 1
+d			TXT	"default ttl 1"
+e		4	TXT	"explicit ttl 4"
+f			TXT	"default ttl 1"

src/bin/loadzone/tests/correct/ttl2.db

@@ -0,0 +1,17 @@
+$ORIGIN ttl2.
+@		1	IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3
+				)
+			NS	ns
+ns			A	10.53.0.1
+a			TXT	"inherited ttl 1"
+b		2	TXT	"explicit ttl 2"
+c			TXT	"inherited ttl 2"
+$TTL 3    ; a new ttl
+d			TXT	"default ttl 3"
+e		2	TXT	"explicit ttl 2"
+f			TXT	"default ttl 3"

src/bin/loadzone/tests/correct/ttlext.db

@@ -0,0 +1,18 @@
+; $Id: ttl1.db,v 1.6 2007/06/19 23:47:04 tbox Exp $
+$ORIGIN ttlext.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3
+				)
+			NS	ns
+ns			A	10.53.0.1
+a			TXT	"soa minttl 3"
+b		2S	TXT	"explicit ttl 2"
+c			TXT	"soa minttl 3"
+$TTL 10M  ; bind9 extention ttl
+d			TXT	"default ttl 600"
+e		4	TXT	"explicit ttl 4"
+f			TXT	"default ttl 600"

src/bin/loadzone/tests/error/Makefile.am

@@ -0,0 +1,25 @@
+PYTESTS = error_test.sh
+
+EXTRA_DIST = error.known
+EXTRA_DIST += formerr1.db 
+EXTRA_DIST += formerr2.db
+EXTRA_DIST += formerr3.db
+EXTRA_DIST += formerr4.db
+EXTRA_DIST += formerr5.db
+EXTRA_DIST += include.txt
+EXTRA_DIST += keyerror1.db
+EXTRA_DIST += keyerror2.db
+EXTRA_DIST += keyerror3.db
+#EXTRA_DIST += nofilenane.db
+EXTRA_DIST += originerr1.db
+EXTRA_DIST += originerr2.db
+
+# later will have configure option to choose this, like: coverage run --branch
+PYCOVERAGE = $(PYTHON)
+# test using command-line arguments, so use check-local target instead of TESTS
+check-local:
+	for pytest in $(PYTESTS) ; do \
+	echo Running test: $$pytest ; \
+	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/bin/loadzone \
+	$(SHELL) $(abs_builddir)/$$pytest ; \
+	done

src/bin/loadzone/tests/error/error.known

@@ -0,0 +1,11 @@
+Error reading zone file: Cannot parse RR, No $ORIGIN: @ IN SOA ns hostmaster 1 3600 1800 1814400 3600
+Error reading zone file: $ORIGIN is not absolute in record:$ORIGIN com
+Error reading zone file: Cannot parse RR: $TL 300
+Error reading zone file: Cannot parse RR: $OIGIN com.
+Error loading database: Error while loading com.: Cannot parse RR: $INLUDE file.txt
+Error loading database: Error while loading com.: Invalid $include format
+Error loading database: Error while loading com.: Cannot parse RR, No $ORIGIN:  include.txt sub
+Error reading zone file: Invalid TTL: ""
+Error reading zone file: Invalid TTL: "M"
+Error loading database: Error while loading com.: Cannot parse RR: b "no type error!"
+Error reading zone file: Could not open bogusfile

src/bin/loadzone/tests/error/error_test.sh.in

@@ -0,0 +1,82 @@
+#! /bin/sh
+
+# Copyright (C) 2010  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
+export PYTHON_EXEC
+
+PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python
+export PYTHONPATH
+
+LOADZONE_PATH=@abs_top_builddir@/src/bin/loadzone
+TEST_OUTPUT_PATH=@abs_top_builddir@/src/bin/loadzone/tests/error
+TEST_FILE_PATH=@abs_top_srcdir@/src/bin/loadzone/tests/error
+
+cd ${LOADZONE_PATH}/tests/error
+
+export LOADZONE_PATH
+status=0
+
+echo "PYTHON PATH: $PYTHONPATH"
+
+echo "Test no \$ORIGIN error in zone file"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/originerr1.db 1> /dev/null 2> error.out
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/originerr2.db 1> /dev/null 2>> error.out
+
+echo "Test: key word TTL spell error"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/keyerror1.db 1> /dev/null 2>> error.out
+
+echo "Test: key word ORIGIN spell error"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/keyerror2.db 1> /dev/null 2>> error.out
+
+echo "Test: key INCLUDE spell error"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/keyerror3.db 1> /dev/null 2>> error.out
+
+echo "Test: include formal error, miss filename"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/formerr1.db 1> /dev/null 2>>error.out
+
+echo "Test: include form error, domain is not absolute"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/formerr2.db 1> /dev/null 2>> error.out
+
+echo "Test: TTL form error, no ttl value"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/formerr3.db 1> /dev/null 2>> error.out
+
+echo "Test: TTL form error, ttl value error"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/formerr4.db 1> /dev/null 2>> error.out
+
+echo "Test: rr form error, no type"
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  ${TEST_FILE_PATH}/formerr5.db 1> /dev/null 2>> error.out
+
+echo "Test: zone file is bogus"
+# since bogusfile doesn't exist anyway, we *don't* specify the directory
+${LOADZONE_PATH}/b10-loadzone -d zone.sqlite3  bogusfile 1> /dev/null 2>> error.out
+
+diff error.out ${TEST_FILE_PATH}/error.known || status=1
+
+echo "Clean tmp file."
+rm -f error.out
+rm -f zone.sqlite3
+
+echo "I:exit status:$status"
+echo "-----------------------------------------------------------------------------"
+echo "Ran 11 test files"
+echo ""
+if [ "$status" -eq 1 ];then
+    echo "ERROR"
+else 
+    echo "OK"
+fi
+exit $status

src/bin/loadzone/tests/error/formerr1.db

@@ -0,0 +1,13 @@
+$TTL 300
+$ORIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+$INCLUDE
+a			A	10.0.0.1

src/bin/loadzone/tests/error/formerr2.db

@@ -0,0 +1,12 @@
+$TTL 300
+com.			IN SOA	ns.com. hostmaster.com. (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns.example.com.
+ns.com.			A	127.0.0.1
+$INCLUDE include.txt sub
+a.com.			A	10.0.0.1

src/bin/loadzone/tests/error/formerr3.db

@@ -0,0 +1,12 @@
+$TTL 
+$ORIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+a			A	10.0.0.1

src/bin/loadzone/tests/error/formerr4.db

@@ -0,0 +1,12 @@
+$TTL M
+$ORIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+a			A	10.0.0.1

src/bin/loadzone/tests/error/formerr5.db

@@ -0,0 +1,13 @@
+$TTL 2M
+$ORIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1 ; ip value
+b               "no type error!"
+a			A	10.0.0.1

src/bin/loadzone/tests/error/include.txt

@@ -0,0 +1 @@
+a  300 A 127.0.0.1

src/bin/loadzone/tests/error/keyerror1.db

@@ -0,0 +1,12 @@
+$TL 300
+@ORIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+a			A	10.0.0.1

src/bin/loadzone/tests/error/keyerror2.db

@@ -0,0 +1,12 @@
+$TTL 300
+$OIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+a			A	10.0.0.1

src/bin/loadzone/tests/error/keyerror3.db

@@ -0,0 +1,13 @@
+$TTL 300
+$ORIGIN com.
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+$INLUDE file.txt
+a			A	10.0.0.1

src/bin/loadzone/tests/error/originerr1.db

@@ -0,0 +1,11 @@
+$TTL 300
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+a			A	10.0.0.1

src/bin/loadzone/tests/error/originerr2.db

@@ -0,0 +1,12 @@
+$TTL 300
+$ORIGIN com
+@			IN SOA	ns hostmaster (
+				1        ; serial
+				3600
+				1800
+				1814400
+				3600
+				)
+			NS	ns
+ns			A	127.0.0.1
+a			A	10.0.0.1

src/bin/xfrin/tests/Makefile.am

@@ -1,16 +1,12 @@
 PYTESTS = xfrin_test.py
 EXTRA_DIST = $(PYTESTS)
 
-if HAVE_BOOST_PYTHON
-
 # later will have configure option to choose this, like: coverage run --branch
 PYCOVERAGE = $(PYTHON)
 # test using command-line arguments, so use check-local target instead of TESTS
 check-local:
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
-	env PYTHONPATH=$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/bin/xfrin:$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python \
+	env PYTHONPATH=$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/bin/xfrin:$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python \
 	$(PYCOVERAGE) $(abs_srcdir)/$$pytest ; \
 	done
-
-endif

src/bin/xfrin/tests/xfrin_test.py

@@ -23,7 +23,7 @@ from xfrin import *
 # Commonly used (mostly constant) test parameters
 #
 TEST_ZONE_NAME = "example.com"
-TEST_RRCLASS = rr_class.IN()
+TEST_RRCLASS = RRClass.IN()
 TEST_DB_FILE = 'db_file'
 TEST_MASTER_IPV4_ADDRESS = '127.0.0.1'
 TEST_MASTER_IPV4_ADDRINFO = (socket.AF_INET, socket.SOCK_STREAM,
@@ -37,16 +37,16 @@ TEST_MASTER_IPV6_ADDRINFO = (socket.AF_INET6, socket.SOCK_STREAM,
 # If some other process uses this port test will fail.
 TEST_MASTER_PORT = '53535'
 
-soa_rdata = create_rdata(rr_type.SOA(), TEST_RRCLASS,
-                         'master.example.com. admin.example.com ' +
-                         '1234 3600 1800 2419200 7200')
-soa_rrset = rrset(name(TEST_ZONE_NAME), TEST_RRCLASS, rr_type.SOA(),
-                  rr_ttl(3600))
+soa_rdata = Rdata(RRType.SOA(), TEST_RRCLASS,
+                  'master.example.com. admin.example.com ' +
+                  '1234 3600 1800 2419200 7200')
+soa_rrset = RRset(Name(TEST_ZONE_NAME), TEST_RRCLASS, RRType.SOA(),
+                  RRTTL(3600))
 soa_rrset.add_rdata(soa_rdata)
-example_axfr_question = question(name(TEST_ZONE_NAME), TEST_RRCLASS,
-                                 rr_type.AXFR())
-example_soa_question = question(name(TEST_ZONE_NAME), TEST_RRCLASS,
-                                 rr_type.SOA())
+example_axfr_question = Question(Name(TEST_ZONE_NAME), TEST_RRCLASS,
+                                 RRType.AXFR())
+example_soa_question = Question(Name(TEST_ZONE_NAME), TEST_RRCLASS,
+                                 RRType.SOA())
 default_questions = [example_axfr_question]
 default_answers = [soa_rrset]
 
@@ -121,26 +121,25 @@ class MockXfrinConnection(XfrinConnection):
         return len(data)
 
     def create_response_data(self, response = True, bad_qid = False,
-                             rcode = rcode.NOERROR(),
+                             rcode = Rcode.NOERROR(),
                              questions = default_questions,
                              answers = default_answers):
-        resp = message(message_mode.RENDER)
+        resp = Message(Message.RENDER)
         qid = self.qid
         if bad_qid:
             qid += 1
         resp.set_qid(qid)
-        resp.set_opcode(op_code.QUERY())
+        resp.set_opcode(Opcode.QUERY())
         resp.set_rcode(rcode)
         if response:
-            resp.set_header_flag(message_flag.QR())
+            resp.set_header_flag(MessageFlag.QR())
         [resp.add_question(q) for q in questions]
-        [resp.add_rrset(section.ANSWER(), a) for a in answers]
+        [resp.add_rrset(Section.ANSWER(), a) for a in answers]
 
-        obuf = output_buffer(0)
-        renderer = message_render(obuf)
+        renderer = MessageRenderer()
         resp.to_wire(renderer)
-        reply_data = struct.pack('H', socket.htons(obuf.get_length()))
-        reply_data += obuf.get_data()
+        reply_data = struct.pack('H', socket.htons(renderer.get_length()))
+        reply_data += renderer.get_data()
 
         return reply_data
 
@@ -158,7 +157,7 @@ class TestXfrinConnection(unittest.TestCase):
             'questions': [example_soa_question],
             'bad_qid': False,
             'response': True,
-            'rcode': rcode.NOERROR(),
+            'rcode': Rcode.NOERROR(),
             'axfr_after_soa': self._create_normal_response_data
             }
 
@@ -189,11 +188,11 @@ class TestXfrinConnection(unittest.TestCase):
         c.close()
 
     def test_init_chclass(self):
-        c = XfrinConnection({}, 'example.com.', rr_class.CH(), TEST_DB_FILE,
+        c = XfrinConnection({}, 'example.com.', RRClass.CH(), TEST_DB_FILE,
                             threading.Event(), TEST_MASTER_IPV4_ADDRINFO)
-        axfrmsg = c._create_query(rr_type.AXFR())
-        self.assertEqual(question_iter(axfrmsg).get_question().get_class(),
-                         rr_class.CH())
+        axfrmsg = c._create_query(RRType.AXFR())
+        self.assertEqual(axfrmsg.get_question()[0].get_class(),
+                         RRClass.CH())
         c.close()
 
     def test_response_with_invalid_msg(self):
@@ -201,41 +200,41 @@ class TestXfrinConnection(unittest.TestCase):
         self.assertRaises(XfrinTestException, self._handle_xfrin_response)
 
     def test_response_without_end_soa(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data()
         self.assertRaises(XfrinTestException, self._handle_xfrin_response)
 
     def test_response_bad_qid(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data(bad_qid = True)
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
     def test_response_non_response(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data(response = False)
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
     def test_response_error_code(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data(
-            rcode=rcode.SERVFAIL())
+            rcode=Rcode.SERVFAIL())
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
     def test_response_multi_question(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data(
             questions=[example_axfr_question, example_axfr_question])
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
     def test_response_empty_answer(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data(answers=[])
         # Should an empty answer trigger an exception?  Even though it's very
         # unusual it's not necessarily invalid.  Need to revisit.
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
     def test_response_non_response(self):
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.conn.reply_data = self.conn.create_response_data(response = False)
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
@@ -247,7 +246,7 @@ class TestXfrinConnection(unittest.TestCase):
 
     def test_soacheck_with_bad_response(self):
         self.conn.response_generator = self._create_broken_response_data
-        self.assertRaises(UserWarning, self.conn._check_soa_serial)
+        self.assertRaises(MessageTooShort, self.conn._check_soa_serial)
 
     def test_soacheck_badqid(self):
         self.soa_response_params['bad_qid'] = True
@@ -260,14 +259,14 @@ class TestXfrinConnection(unittest.TestCase):
         self.assertRaises(XfrinException, self.conn._check_soa_serial)
 
     def test_soacheck_error_code(self):
-        self.soa_response_params['rcode'] = rcode.SERVFAIL()
+        self.soa_response_params['rcode'] = Rcode.SERVFAIL()
         self.conn.response_generator = self._create_soa_response_data
         self.assertRaises(XfrinException, self.conn._check_soa_serial)
 
     def test_response_shutdown(self):
         self.conn.response_generator = self._create_normal_response_data
         self.conn._shutdown_event.set()
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.assertRaises(XfrinException, self._handle_xfrin_response)
 
     def test_response_timeout(self):
@@ -282,13 +281,13 @@ class TestXfrinConnection(unittest.TestCase):
 
     def test_response_bad_message(self):
         self.conn.response_generator = self._create_broken_response_data
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         self.assertRaises(Exception, self._handle_xfrin_response)
 
     def test_response(self):
         # normal case.
         self.conn.response_generator = self._create_normal_response_data
-        self.conn._send_query(rr_type.AXFR())
+        self.conn._send_query(RRType.AXFR())
         # two SOAs, and only these have been transfered.  the 2nd SOA is just
         # a marker, so only 1 RR has been provided in the iteration.
         self.assertEqual(self._handle_xfrin_response(), 1)
@@ -317,7 +316,10 @@ class TestXfrinConnection(unittest.TestCase):
 
     def test_do_soacheck_broken_response(self):
         self.conn.response_generator = self._create_broken_response_data
-        self.assertEqual(self.conn.do_xfrin(True), XFRIN_FAIL)
+        # XXX: TODO: this test failed here, should xfr not raise an
+        # exception but simply drop and return FAIL?
+        #self.assertEqual(self.conn.do_xfrin(True), XFRIN_FAIL)
+        self.assertRaises(MessageTooShort, self.conn.do_xfrin, True)
 
     def test_do_soacheck_badqid(self):
         # the QID mismatch would internally trigger a XfrinException exception,
@@ -410,22 +412,31 @@ class TestXfrin(unittest.TestCase):
         return self.xfr._parse_cmd_params(self.args)
 
     def test_parse_cmd_params(self):
-        name, master_addrinfo, db_file = self._do_parse()
+        name, rrclass, master_addrinfo, db_file = self._do_parse()
         self.assertEqual(master_addrinfo[4][1], int(TEST_MASTER_PORT))
         self.assertEqual(name, TEST_ZONE_NAME)
+        self.assertEqual(rrclass, TEST_RRCLASS)
         self.assertEqual(master_addrinfo[4][0], TEST_MASTER_IPV4_ADDRESS)
         self.assertEqual(db_file, TEST_DB_FILE)
 
     def test_parse_cmd_params_default_port(self):
         del self.args['port']
-        master_addrinfo = self._do_parse()[1]
+        master_addrinfo = self._do_parse()[2]
         self.assertEqual(master_addrinfo[4][1], 53)
 
     def test_parse_cmd_params_ip6master(self):
         self.args['master'] = TEST_MASTER_IPV6_ADDRESS
-        master_addrinfo = self._do_parse()[1]
+        master_addrinfo = self._do_parse()[2]
         self.assertEqual(master_addrinfo[4][0], TEST_MASTER_IPV6_ADDRESS)
 
+    def test_parse_cmd_params_chclass(self):
+        self.args['rrclass'] = 'CH'
+        self.assertEqual(self._do_parse()[1], RRClass.CH())
+
+    def test_parse_cmd_params_bogusclass(self):
+        self.args['rrclass'] = 'XXX'
+        self.assertRaises(XfrinException, self._do_parse)
+
     def test_parse_cmd_params_nozone(self):
         # zone name is mandatory.
         del self.args['zone_name']
@@ -488,12 +499,12 @@ class TestXfrin(unittest.TestCase):
                                                   self.args)['result'][0], 1)
 
     def test_command_handler_retransfer_nomodule(self):
-        dns_module = sys.modules['bind10_dns'] # this must exist
-        del sys.modules['bind10_dns']
+        dns_module = sys.modules['libdns_python'] # this must exist
+        del sys.modules['libdns_python']
         self.assertEqual(self.xfr.command_handler("retransfer",
                                                   self.args)['result'][0], 1)
         # sys.modules is global, so we must recover it
-        sys.modules['bind10_dns'] = dns_module
+        sys.modules['libdns_python'] = dns_module
 
     def test_command_handler_refresh(self):
         # at this level, refresh is no different than retransfer.
@@ -502,6 +513,13 @@ class TestXfrin(unittest.TestCase):
         self.assertEqual(self.xfr.command_handler("refresh",
                                                   self.args)['result'][0], 0)
 
+    def test_command_handler_notify(self):
+        # at this level, refresh is no different than retransfer.
+        self.args['master'] = TEST_MASTER_IPV6_ADDRESS
+        # ...but right now we disable the feature due to security concerns.
+        self.assertEqual(self.xfr.command_handler("notify",
+                                                  self.args)['result'][0], 1)
+
     def test_command_handler_unknown(self):
         self.assertEqual(self.xfr.command_handler("xxx", None)['result'][0], 1)