Merge branch 'master' into trac510

Conflicts:
	ChangeLog
	src/bin/auth/statistics.cc
	src/bin/auth/statistics.h
	src/lib/Makefile.am

ChangeLog

@@ -1,9 +1,474 @@
-TBD.	[func]		y-aharen
+339.	[func]		y-aharen
 	src/lib/statistics: Added statistics counter library for entire server
 	items and per zone items. Also, modified b10-auth to use it. It is
 	also intended to use in the other modules such as b10-resolver.
 	(Trac #510, git TBD)
 
+338.	[bug]		jinmei
+	b10-xfrin didn't check SOA serials of SOA and IXFR responses,
+	which resulted in unnecessary transfer or unexpected IXFR
+	timeouts (these issues were not overlooked but deferred to be
+	fixed until #1278 was completed).  Validation on responses to SOA
+	queries were tighten, too.
+	(Trac #1299, git 6ff03bb9d631023175df99248e8cc0cda586c30a)
+
+337.	[func]		tomek
+	libdhcp++: Support for DHCPv4 option that can store a single
+	address or a list of IPv4 addresses added. Support for END option
+	added.
+	(Trac #1350, git cc20ff993da1ddb1c6e8a98370438b45a2be9e0a)
+
+336.	[func]		jelte
+	libdns++ (and its python wrapper) now includes a class Serial, for 
+	SOA SERIAL comparison and addition. Operations on instances of this 
+	class follow the specification from RFC 1982. 
+	Rdata::SOA::getSerial() now returns values of this type (and not 
+	uint32_t).
+	(Trac #1278, git 2ae72d76c74f61a67590722c73ebbf631388acbd)
+
+335.	[bug]*		jelte
+	The DataSourceClientContainer class that dynamically loads 
+	datasource backend libraries no longer provides just a .so file name 
+	to its call to dlopen(), but passes it an absolute path. This means 
+	that it is no longer an system implementation detail that depends on 
+	[DY]LD_LIBRARY_PATH which file is chosen, should there be multiple 
+	options (for instance, when test-running a new build while a 
+	different version is installed).
+	These loadable libraries are also no longer installed in the default 
+	library path, but in a subdirectory of the libexec directory of the 
+	target ($prefix/libexec/[version]/backends).
+	This also removes the need to handle b10-xfin and b10-xfrout as 
+	'special' hardcoded components, and they are now started as regular 
+	components as dictated by the configuration of the boss process.
+	(Trac #1292, git 83ce13c2d85068a1bec015361e4ef8c35590a5d0)
+
+334.	[bug]		jinmei
+	b10-xfrout could potentially create an overflow response message
+	(exceeding the 64KB max) or could create unnecessarily small
+	messages.  The former was actually unlikely to happen due to the
+	effect of name compression, and the latter was marginal and at least
+	shouldn't cause an interoperability problem, but these were still
+	potential problems and were fixed.
+	(Trac #1389, git 3fdce88046bdad392bd89ea656ec4ac3c858ca2f)
+
+333.    [bug]		dvv
+	Solaris needs "-z now" to force non-lazy binding and prevent g++ static
+	initialization code from deadlocking.
+	(Trac #1439, git c789138250b33b6b08262425a08a2a0469d90433)
+
+332.    [bug]		vorner
+	C++ exceptions in the isc.dns.Rdata wrapper are now converted
+	to python ones instead of just aborting the interpretter.
+	(Trac #1407, git 5b64e839be2906b8950f5b1e42a3fadd72fca033)
+
+bind10-devel-20111128 released on November 28, 2011
+
+331.	[bug]		shane
+	Fixed a bug in data source library where a zone with more labels
+	than an out-of-bailiwick name server would cause an exception to
+	be raised.
+	(Trac #1430, git 81f62344db074bc5eea3aaf3682122fdec6451ad)
+
+330.	[bug]		jelte
+	Fixed a bug in b10-auth where it would sometimes fail because it
+	tried to check for queued msgq messages before the session was
+	fully running.
+	(git c35d0dde3e835fc5f0a78fcfcc8b76c74bc727ca)
+
+329.	[doc]		vorner, jreed
+	Document the bind10 run control configuration in guide and
+	manual page.
+	(Trac #1341, git c1171699a2b501321ab54207ad26e5da2b092d63)
+
+328.	[func]		jelte
+	b10-auth now passes IXFR requests on to b10-xfrout, and no longer
+	responds to them with NOTIMPL.
+	(Trac #1390, git ab3f90da16d31fc6833d869686e07729d9b8c135)
+
+327.	[func]		jinmei
+	b10-xfrout now supports IXFR.  (Right now there is no user
+	configurable parameter about this feature; b10-xfrout will
+	always respond to IXFR requests according to RFC1995).
+	(Trac #1371 and #1372, git 80c131f5b0763753d199b0fb9b51f10990bcd92b)
+
+326.	[build]*		jinmei
+	Added a check script for the SQLite3 schema version.  It will be
+	run at the beginning of 'make install', and if it detects an old
+	version of schema, installation will stop.  You'll then need to
+	upgrade the database file by following the error message.
+	(Trac #1404, git a435f3ac50667bcb76dca44b7b5d152f45432b57)
+
+325.	[func]		jinmei
+	Python isc.datasrc: added interfaces for difference management:
+	DataSourceClient.get_updater() now has the 'journaling' parameter
+	to enable storing diffs to the data source, and a new class
+	ZoneJournalReader was introduced to retrieve them, which can be
+	created by the new DataSourceClient.get_journal_reader() method.
+	(Trac #1333, git 3e19362bc1ba7dc67a87768e2b172c48b32417f5,
+	git 39def1d39c9543fc485eceaa5d390062edb97676)
+
+324.	[bug]		jinmei
+	Fixed reference leak in the isc.log Python module.  Most of all
+	BIND 10 Python programs had memory leak (even though the pace of
+	leak may be slow) due to this bug.
+	(Trac #1359, git 164d651a0e4c1059c71f56b52ea87ac72b7f6c77)
+
+323.	[bug]		jinmei
+	b10-xfrout incorrectly skipped adding TSIG RRs to some
+	intermediate responses (when TSIG is to be used for the
+	responses).  While RFC2845 optionally allows to skip intermediate
+	TSIGs (as long as the digest for the skipped part was included
+	in a later TSIG), the underlying TSIG API doesn't support this
+	mode of signing.
+	(Trac #1370, git 76fb414ea5257b639ba58ee336fae9a68998b30d)
+
+322.	[func]		jinmei
+	datasrc: Added C++ API for retrieving difference of two versions
+	of a zone.  A new ZoneJournalReader class was introduced for this
+	purpose, and a corresponding factory method was added to
+	DataSourceClient.
+	(Trac #1332, git c1138d13b2692fa3a4f2ae1454052c866d24e654)
+
+321.	[func]*		jinmei
+	b10-xfrin now installs IXFR differences into the underlying data
+	source (if it supports journaling) so that the stored differences
+	can be used for subsequent IXFR-out transactions.
+	Note: this is a backward incompatibility change for older sqlite3
+	database files.  They need to be upgraded to have a "diffs" table.
+	(Trac #1376, git 1219d81b49e51adece77dc57b5902fa1c6be1407)
+
+320.	[func]*		vorner
+	The --brittle switch was removed from the bind10 executable.
+	It didn't work after change #316 (Trac #213) and the same
+	effect can be accomplished by declaring all components as core.
+	(Trac #1340, git f9224368908dd7ba16875b0d36329cf1161193f0)
+
+319.	[func]		naokikambe
+	b10-stats-httpd was updated. In addition of the access to all
+	statistics items of all modules, the specified item or the items
+	of the specified module name can be accessed.  For example, the
+	URI requested by using the feature is showed as
+	"/bind10/statistics/xml/Auth" or
+	"/bind10/statistics/xml/Auth/queries.tcp". The list of all possible
+	module names and all possible item names can be showed in the
+	root document, whose URI is "/bind10/statistics/xml".  This change
+	is not only for the XML documents but also is for the XSD and
+	XSL documents.
+	(Trac #917, git b34bf286c064d44746ec0b79e38a6177d01e6956)
+
+318.	[func]		stephen
+	Add C++ API for accessing zone difference information in
+	database-based data sources.
+	(Trac #1330, git 78770f52c7f1e7268d99e8bfa8c61e889813bb33)
+
+317.	[func]		vorner
+	datasrc: the getUpdater method of DataSourceClient supports an
+	optional 'journaling' parameter to indicate the generated updater
+	to store diffs.  The database based derived class implements this
+	extension.
+	(Trac #1331, git 713160c9bed3d991a00b2ea5e7e3e7714d79625d)
+
+316.	[func]*		vorner
+	The configuration of what parts of the system run is more
+	flexible now.  Everything that should run must have an
+	entry in Boss/components.
+	(Trac #213, git 08e1873a3593b4fa06754654d22d99771aa388a6)
+
+315.	[func]		tomek
+	libdhcp: Support for DHCPv4 packet manipulation is now implemented.
+	All fixed fields are now supported. Generic support for DHCPv4
+	options is available (both parsing and assembly). There is no code
+	that uses this new functionality yet, so it is not usable directly
+	at this time. This code will be used by upcoming b10-dhcp4 daemon.
+	(Trac #1228, git 31d5a4f66b18cca838ca1182b9f13034066427a7)
+
+314.	[bug]		jelte
+	b10-xfrin would previously initiate incoming transfers upon 
+	receiving NOTIFY messages from any address (if the zone was 
+	known to b10-xfrin, and using the configured address). It now 
+	only starts a transfer if the source address from the NOTIFY 
+	packet matches the configured master address and port. This was 
+	really already fixed in release bind10-devel-20111014, but there 
+	were some deferred cleanups to add.
+	(Trac #1298, git 1177bfe30e17a76bea6b6447e14ae9be9e1ca8c2)
+
+313.	[func]		jinmei
+	datasrc: Added C++ API for adding zone differences to database
+	based data sources.  It's intended to be used for the support for
+	IXFR-in and dynamic update (so they can subsequently be retrieved
+	for IXFR-out).  The addRecordDiff method of the DatabaseAccessor
+	defines the interface, and a concrete implementation for SQLite3
+	was provided.
+	(Trac #1329, git 1aa233fab1d74dc776899df61181806679d14013)
+
+312.	[func]		jelte
+	Added an initial framework for doing system tests using the 
+	cucumber-based BDD tool Lettuce. A number of general steps are
+	included,  for instance running bind10 with specific
+	configurations, sending queries, and inspecting query answers. A
+	few very basic tests are included as well.
+	(Trac #1290, git 6b75c128bcdcefd85c18ccb6def59e9acedd4437)
+
+311.	[bug]		jelte
+	Fixed a bug in bindctl where tab-completion for names that
+	contain a hyphen resulted in unexpected behaviour, such as
+	appending the already-typed part again.
+	(Trac #1345, git f80ab7879cc29f875c40dde6b44e3796ac98d6da)
+
+310.	[bug]		jelte
+	Fixed a bug where bindctl could not set a value that is optional
+	and has no default, resulting in the error that the setting
+	itself was unknown. bindctl now correctly sees the setting and
+	is able to set it.
+	(Trac #1344, git 0e776c32330aee466073771600390ce74b959b38)
+
+309.	[bug]		jelte
+	Fixed a bug in bindctl where the removal of elements from a set
+	with default values was not stored, unless the set had been
+	modified in another way already.
+	(Trac #1343, git 25c802dd1c30580b94345e83eeb6a168ab329a33)
+
+308.	[build]		jelte
+	The configure script will now use pkg-config for finding
+	information about the Botan library. If pkg-config is unavailable,
+	or unaware of Botan, it will fall back to botan-config. It will
+	also use botan-config when a specific botan library directory is
+	given using the '--with-botan=' flag
+	(Trac #1194, git dc491833cf75ac1481ba1475795b0f266545013d)
+
+307.	[func]		vorner
+	When zone transfer in fails with IXFR, it is retried with AXFR
+	automatically.
+	(Trac #1279, git cd3588c9020d0310f949bfd053c4d3a4bd84ef88)
+
+306.	[bug]		stephen
+	Boss process now waits for the configuration manager to initialize
+	itself before continuing with startup.  This fixes a race condition
+	whereby the Boss could start the configuration manager and then
+	immediately start components that depended on that component being
+	fully initialized.
+	(Trac #1271, git 607cbae949553adac7e2a684fa25bda804658f61)
+
+305.	[bug]		jinmei
+	Python isc.dns, isc.datasrc, xfrin, xfrout: fixed reference leak
+	in Message.get_question(), Message.get_section(),
+	RRset.get_rdata(), and DataSourceClient.get_updater().
+	The leak caused severe memory leak in b10-xfrin, and (although no
+	one reported it) should have caused less visible leak in
+	b10-xfrout.  b10-xfrin had its own leak, which was also fixed.
+	(Trac #1028, git a72886e643864bb6f86ab47b115a55e0c7f7fcad)
+
+304.	[bug]		jelte
+	The run_bind10.sh test script now no longer runs processes from
+	an installed version of BIND 10, but will correctly use the
+	build tree paths.
+	(Trac #1246, git 1d43b46ab58077daaaf5cae3c6aa3e0eb76eb5d8)
+
+303.	[bug]		jinmei
+	Changed the installation path for the UNIX domain file used
+	for the communication between b10-auth and b10-xfrout to a
+	"@PACKAGE@" subdirectory (e.g. from /usr/local/var to
+	/usr/local/var/bind10-devel).  This should be transparent change
+	because this file is automatically created and cleaned up, but
+	if the old file somehow remains, it can now be safely removed.
+	(Trac #869, git 96e22f4284307b1d5f15e03837559711bb4f580c)
+
+302.	[bug]		jelte
+	msgq no longer crashes if the remote end is closed while msgq
+	tries to send data. It will now simply drop the message and close
+	the connection itself.
+	(Trac #1180, git 6e68b97b050e40e073f736d84b62b3e193dd870a)
+
+301.	[func]		stephen
+	Add system test for IXFR over TCP.
+	(Trac #1213, git 68ee3818bcbecebf3e6789e81ea79d551a4ff3e8)
+
+300.	[func]*		tomek
+	libdhcp: DHCP packet library was implemented. Currently it handles
+	packet reception, option parsing, option generation and output
+	packet building. Generic and specialized classes for several
+	DHCPv6 options (IA_NA, IAADDR, address-list) are available. A
+	simple code was added that leverages libdhcp. It is a skeleton
+	DHCPv6 server. It receives incoming SOLICIT and REQUEST messages
+	and responds with proper ADVERTISE and REPLY. Note that since
+	LeaseManager is not implemented, server assigns the same
+	hardcoded lease for every client. This change removes existing
+	DHCPv6 echo server as it was only a proof of concept code.
+	(Trac #1186, git 67ea6de047d4dbd63c25fe7f03f5d5cc2452ad7d)
+
+299.	[build]		jreed
+	Do not install the libfake_session, libtestutils, or libbench
+	libraries. They are used by tests within the source tree.
+	Convert all test-related makefiles to build test code at
+	regular make time to better work with test-driven development.
+	This reverts some of #1901. (The tests are ran using "make
+	check".)
+	(Trac #1286, git cee641fd3d12341d6bfce5a6fbd913e3aebc1e8e)
+
+bind10-devel-20111014 released on October 14, 2011
+
+298.	[doc]		jreed
+	Shorten README. Include plain text format of the Guide.
+	(git d1897d3, git 337198f)
+
+297.	[func]		dvv
+	Implement the SPF rrtype according to RFC4408.
+	(Trac #1140, git 146934075349f94ee27f23bf9ff01711b94e369e)
+
+296.	[build]		jreed
+	Do not install the unittest libraries. At this time, they
+	are not useful without source tree (and they may or may
+	not have googletest support). Also, convert several makefiles
+	to build tests at "check" time and not build time.
+	(Trac #1091, git 2adf4a90ad79754d52126e7988769580d20501c3)
+
+295.	[bug]		jinmei
+	__init__.py for isc.dns was installed in the wrong directory,
+	which would now make xfrin fail to start.  It was also bad
+	in that it replaced any existing __init__.py in th public
+	site-packages directory.  After applying this fix You may want to
+	check if the wrong init file is in the wrong place, in which
+	case it should be removed.
+	(Trac #1285, git af3b17472694f58b3d6a56d0baf64601b0f6a6a1)
+
+294.	[func]		jelte, jinmei, vorner
+	b10-xfrin now supports incoming IXFR.  See BIND 10 Guide for
+	how to configure it and operational notes.
+	(Trac #1212, multiple git merges)
+
+293.	[func]*		tomek
+	b10-dhcp6: Implemented DHCPv6 echo server. It joins DHCPv6
+	multicast groups and listens to incoming DHCPv6 client messages.
+	Received messages are then echoed back to clients. This
+	functionality is limited, but it can be used to test out client
+	resiliency to unexpected messages. Note that network interface
+	detection routines are not implemented yet, so interface name
+	and its address must be specified in interfaces.txt.
+	(Trac #878, git 3b1a604abf5709bfda7271fa94213f7d823de69d)
+
+292.	[func]		dvv
+	Implement the DLV rrtype according to RFC4431.
+	(Trac #1144, git d267c0511a07c41cd92e3b0b9ee9bf693743a7cf)
+
+291.	[func]		naokikambe
+	Statistics items are specified by each module's spec file.
+	Stats module can read these through the config manager. Stats
+	module and stats httpd report statistics data and statistics
+	schema by each module via both bindctl and HTTP/XML.
+	(Trac #928,#929,#930,#1175,
+	git 054699635affd9c9ecbe7a108d880829f3ba229e)
+
+290.	[func]		jinmei
+	libdns++/pydnspp: added an option parameter to the "from wire"
+	methods of the Message class.  One option is defined,
+	PRESERVE_ORDER, which specifies the parser to handle each RR
+	separately, preserving the order, and constructs RRsets in the
+	message sections so that each RRset contains only one RR.
+	(Trac #1258, git c874cb056e2a5e656165f3c160e1b34ccfe8b302)
+
+289.	[func]*		jinmei
+	b10-xfrout: ACLs for xfrout can now be configured per zone basis.
+	A per zone ACL is part of a more general zone configuration.  A
+	quick example for configuring an ACL for zone "example.com" that
+	rejects any transfer request for that zone is as follows:
+	> config add Xfrout/zone_config
+	> config set Xfrout/zone_config[0]/origin "example.com"
+	> config add Xfrout/zone_config[0]/transfer_acl
+	> config set Xfrout/zone_config[0]/transfer_acl[0] {"action": "REJECT"}
+	The previous global ACL (query_acl) was renamed to transfer_acl,
+	which now works as the default ACL.  Note: backward compatibility
+	is not provided, so an existing configuration using query_acl
+	needs to be updated by hand.
+	Note: the per zone configuration framework is a temporary
+	workaround.  It will eventually be redesigned as a system wide
+	configuration.
+	(Trac #1165, git 698176eccd5d55759fe9448b2c249717c932ac31)
+
+288.	[bug]		stephen
+	Fixed problem whereby the order in which component files appeared in
+	rdataclass.cc was system dependent, leading to problems on some
+	systems where data types were used before the header file in which
+	they were declared was included.
+	(Trac #1202, git 4a605525cda67bea8c43ca8b3eae6e6749797450)
+
+287.	[bug]*		jinmei
+	Python script files for log messages (xxx_messages.py) should have
+	been installed under the "isc" package.  This fix itself should
+	be a transparent change without affecting existing configurations
+	or other operational practices, but you may want to clean up the
+	python files from the common directly (such as "site-packages").
+	(Trac #1101, git 0eb576518f81c3758c7dbaa2522bd8302b1836b3)
+
+286.	[func]		ocean
+	libdns++: Implement the HINFO rrtype support according to RFC1034,
+	and RFC1035.
+	(Trac #1112, git 12d62d54d33fbb1572a1aa3089b0d547d02924aa)
+
+285.	[bug]		jelte
+	sqlite3 data source: fixed a race condition on initial startup,
+	when the database has not been initialized yet, and multiple
+	processes are trying to do so, resulting in one of them failing.
+	(Trac #326, git 5de6f9658f745e05361242042afd518b444d7466)
+
+284.	[bug]		jerry
+	b10-zonemgr: zonemgr will not terminate on empty zones, it will
+	log a warning and try to do zone transfer for them.
+	(Trac #1153, git 0a39659638fc68f60b95b102968d7d0ad75443ea)
+
+283.	[bug]		zhanglikun
+	Make stats and boss processes wait for answer messages from each
+	other in block mode to avoid orphan answer messages, add an internal
+	command "getstats" to boss process for getting statistics data from
+	boss.
+	(Trac #519, git 67d8e93028e014f644868fede3570abb28e5fb43)
+
+282.	[func]		ocean
+	libdns++: Implement the NAPTR rrtype according to RFC2915,
+	RFC2168 and RFC3403.
+	(Trac #1130, git 01d8d0f13289ecdf9996d6d5d26ac0d43e30549c)
+
+bind10-devel-20110819 released on August 19, 2011
+
+281.	[func]		jelte
+	Added a new type for configuration data: "named set". This allows for
+	similar configuration as the current "list" type, but with strings
+	instead of indices as identifiers. The intended use is for instance
+	/foo/zones/example.org/bar instead of /foo/zones[2]/bar. Currently
+	this new type is not in use yet.
+	(Trac #926, git 06aeefc4787c82db7f5443651f099c5af47bd4d6)
+
+280.	[func]		jerry
+	libdns++: Implement the MINFO rrtype according to RFC1035.
+	(Trac #1113, git 7a9a19d6431df02d48a7bc9de44f08d9450d3a37)
+
+279.	[func]		jerry
+	libdns++: Implement the AFSDB rrtype according to RFC1183.
+	(Trac #1114, git ce052cd92cd128ea3db5a8f154bd151956c2920c)
+
+278.	[doc]		jelte
+	Add logging configuration documentation to the guide.
+	(Trac #1011, git 2cc500af0929c1f268aeb6f8480bc428af70f4c4)
+
+277.	[func]		jerry
+	libdns++: Implement the SRV rrtype according to RFC2782.
+	(Trac #1128, git 5fd94aa027828c50e63ae1073d9d6708e0a9c223)
+
+276.	[func]		stephen
+	Although the top-level loggers are named after the program (e.g.
+	b10-auth, b10-resolver), allow the logger configuration to omit the
+	"b10-" prefix and use just the module name.
+	(Trac #1003, git a01cd4ac5a68a1749593600c0f338620511cae2d)
+
+275.	[func]		jinmei
+	Added support for TSIG key matching in ACLs.  The xfrout ACL can
+	now refer to TSIG key names using the "key" attribute.  For
+	example, the following specifies an ACL that allows zone transfer
+	if and only if the request is signed with a TSIG of a key name
+	"key.example":
+	> config set Xfrout/query_acl[0] {"action": "ACCEPT", \
+	                                  "key": "key.example"}
+	(Trac #1104, git 9b2e89cabb6191db86f88ee717f7abc4171fa979)
+
 274.	[bug]		naokikambe
 	add unittests for functions xml_handler, xsd_handler and xsl_handler
 	respectively to make sure their behaviors are correct, regardless of
@@ -11,7 +476,7 @@ TBD.	[func]		y-aharen
 	returns is str or byte.
 	(Trac #1021, git 486bf91e0ecc5fbecfe637e1e75ebe373d42509b)
 
-273.    [func]		vorner
+273.	[func]		vorner
 	It is possible to specify ACL for the xfrout module. It is in the ACL
 	configuration key and has the usual ACL syntax. It currently supports
 	only the source address. Default ACL accepts everything.
@@ -190,7 +655,7 @@ bind10-devel-20110705 released on July 05, 2011
 	(Trac #542, git 1aa773d84cd6431aa1483eb34a7f4204949a610f)
 
 243.	[func]*		feng
-	Add optional hmac algorithm SHA224/384/812.
+	Add optional hmac algorithm SHA224/384/512.
 	(Trac #782, git 77d792c9d7c1a3f95d3e6a8b721ac79002cd7db1)
 
 bind10-devel-20110519 released on May 19, 2011

Makefile.am

@@ -1,13 +1,17 @@
-SUBDIRS = doc src tests
+SUBDIRS = compatcheck doc src tests
 USE_LCOV=@USE_LCOV@
 LCOV=@LCOV@
 GENHTML=@GENHTML@
+DISTCHECK_GTEST_CONFIGURE_FLAG=@DISTCHECK_GTEST_CONFIGURE_FLAG@
 
 DISTCLEANFILES = config.report
 
 # When running distcheck target, do not install the configurations
 DISTCHECK_CONFIGURE_FLAGS = --disable-install-configurations
 
+# Use same --with-gtest flag if set
+DISTCHECK_CONFIGURE_FLAGS += $(DISTCHECK_GTEST_CONFIGURE_FLAG)
+
 clean-cpp-coverage:
 	@if [ $(USE_LCOV) = yes ] ; then \
 		$(LCOV) --directory . --zerocounters; \

README

@@ -1,3 +1,4 @@
+
 This is the source for the development version of BIND 10.
 
 BIND is the popular implementation of a DNS server, developer
@@ -8,10 +9,10 @@ for serving, maintaining, and developing DNS.
 BIND10-devel is new development leading up to the production
 BIND 10 release. It contains prototype code and experimental
 interfaces. Nevertheless it is ready to use now for testing the
-new BIND 10 infrastructure ideas. The Year 2 milestones of the
-five year plan are described here:
+new BIND 10 infrastructure ideas. The Year 3 goals of the five
+year plan are described here:
 
-	https://bind10.isc.org/wiki/Year2Milestones
+        http://bind10.isc.org/wiki/Year3Goals
 
 This release includes the bind10 master process, b10-msgq message
 bus, b10-auth authoritative DNS server (with SQLite3 and in-memory
@@ -21,12 +22,15 @@ AXFR inbound service, b10-xfrout outgoing AXFR service, b10-zonemgr
 secondary manager, b10-stats statistics collection and reporting
 daemon, b10-stats-httpd for HTTP access to XML-formatted stats,
 b10-host DNS lookup utility, and a new libdns++ library for C++
-with a python wrapper.
+with a python wrapper. BIND 10 also provides an experimental DHCPv6
+echo server, b10-dhcp6.
 
-Documentation is included and also available via the BIND 10
-website at http://bind10.isc.org/
+Documentation is included with the source. See doc/guide/bind10-guide.txt
+(or bind10-guide.html) for installation instructions.  The
+documentation is also available via the BIND 10 website at
+http://bind10.isc.org/
 
-The latest released source may be downloaded from:
+The latest released source tar file may be downloaded from:
 
         ftp://ftp.isc.org/isc/bind10/
 
@@ -40,15 +44,11 @@ Bugs may be reported as tickets via the developers website:
 
         http://bind10.isc.org/
 
-BUILDING
-
-See the Guide for detailed installation directions at
-doc/guide/bind10-guide.html.
-
-Simple build instructions:
+Simple build and installation instructions:
 
   ./configure
   make
+  make install
 
 If building from Git repository, run:
 
@@ -56,197 +56,11 @@ If building from Git repository, run:
 
 before running ./configure
 
-Requires autoconf 2.59 or newer.
-
-Use automake-1.11 or better for working Python 3.1 tests.
-Alternatively, you could manually specify an absolute path to python
-executable by the --with-pythonpath option of the configure script,
-e.g.,
-% ./configure --with-pythonpath=/usr/local/bin/python3.1
-
-Operating-System specific tips:
-
-- FreeBSD
-  You may need to install a python binding for sqlite3 by hand.  A
-  sample procedure is as follows:
-  - add the following to /etc/make.conf
-    PYTHON_VERSION=3.1
-  - build and install the python binding from ports, assuming the top
-    directory of the ports system is /usr/ports
-  % cd /usr/ports/databases/py-sqlite3/
-  % make
-  % sudo make install
-
-INSTALLATION
-
-Install with:
-
-  make install
-
-TESTS
-
-The tests use the googletests framework for C++. It is available
-from http://code.google.com/p/googletest/.  To enable the tests,
-configure BIND 10 with: 
-
-  ./configure --with-gtest
-
-Then run "make check" to run these tests.
-
-TEST COVERAGE
-
-Code coverage reports may be generated using make. These are
-based on running on the unit tests. The resulting reports are placed
-in coverage-cpp-html and coverage-python-html directories for C++
-and Python, respectively.
-
-The code coverage report for the C++ tests uses LCOV. It is available
-from http://ltp.sourceforge.net/. To generate the HTML report,
-first configure BIND 10 with:
- 
-  ./configure --with-lcov
-
-The code coverage report for the Python tests uses coverage.py (aka
-pycoverage). It is available from http://nedbatchelder.com/code/coverage/.
-To generate the HTML report, first configure BIND 10 with:
-
-  ./configure --with-pycoverage
-
-Doing code coverage tests:
-
-  make coverage
-	Does the clean, perform, and report targets for C++ and Python.
-
-  make clean-coverage
-	Zeroes the code coverage counters and removes the HTML reports
-	for C++ and Python.
-
-  make perform-coverage
-	Runs the C++ (using the googletests framework) and Python
-	tests.
+See the Guide for detailed installation directions at
+doc/guide/bind10-guide.txt.
 
-  make report-coverage
-	Generates the coverage reports in HTML for C++ and Python.
+For operating system specific tips see the wiki at:
 
-  make clean-cpp-coverage
-	Zeroes the code coverage counters and removes the HTML report
-	for the C++ tests.
+       http://bind10.isc.org/wiki/SystemSpecificNotes
 
-  make clean-python-coverage
-	Zeroes the code coverage counters and removes the HTML report
-	for the Python tests.
-
-  make report-cpp-coverage
-	Generates the coverage report in HTML for C++, excluding
-	some unrelated headers.  The HTML reports are placed in a
-	directory called coverage-cpp-html/.
-
-  make report-python-coverage
-	Generates the coverage report in HTML for Python. The HTML
-	reports are placed in a directory called coverage-python-html/.
-
-DEVELOPERS
-
-The generated run_*.sh scripts available in the src/bin directories
-are for running the code using the source tree.
-
-RUNNING
-
-You can start the BIND 10 processes by running bind10 which is
-installed to the sbin directory under the installation prefix.
-The default location is:
-
-  /usr/local/sbin/bind10
-
-For development work, you can also run the bind10 services from the
-source tree:
-
- ./src/bin/bind10/run_bind10.sh 
-
-(Which will use the modules and configurations also from the source
-tree.)
-
-CONFIGURATION
-
-Commands can be given through the bindctl tool.
-
-The server must be running for bindctl to work.
-
-The following configuration commands are available
-
-help: show the different command modules
-<module> help: show the commands for module
-<module> <command> help: show info for the command
-
-
-config show [identifier]: Show the currently set values. If no identifier is
-                          given, the current location is used. If a config
-                          option is a list or a map, the value is not
-                          shown directly, but must be requested separately.
-config go [identifier]:   Go to the given location within the configuration.
-config set [identifier] <value>: Set a configuration value.
-config unset [identifier]: Remove a value (reverts to default if the option
-                           is mandatory).
-config add [identifier] <value>: add a value to a list
-config remove [identifier] <value>: remove a value from a list 
-config revert:	Revert all changes that have not been committed
-config commit: Commit all changes
-config diff: Show the changes that have not been committed yet
-
-
-EXAMPLE SESSION
-
-~> bindctl
-["login success "] login as root
-> help
-BindCtl, verstion 0.1
-usage: <module name> <command name> [param1 = value1 [, param2 = value2]]
-Type Tab character to get the hint of module/command/paramters.
-Type "help(? h)" for help on bindctl.
-Type "<module_name> help" for help on the specific module.
-Type "<module_name> <command_name> help" for help on the specific command.
-
-Available module names: 
-	 help 	Get help for bindctl
-	 config 	Configuration commands
-	 Xfrin 	same here
-	 Auth 	same here
-	 Boss 	same here
-> config help
-Module  config 	Configuration commands 
-Available commands:
-	 help 	(Get help for module)
-	 show 	(Show configuration)
-	 add 	(Add entry to configuration list)
-	 remove 	(Remove entry from configuration list)
-	 set 	(Set a configuration value)
-	 unset 	(Unset a configuration value)
-	 diff 	(Show all local changes)
-	 revert 	(Revert all local changes)
-	 commit 	(Commit all local changes)
-	 go 	(Go to a specific configuration part)
-> config show
-Xfrin/	module	
-Auth/	module	
-Boss/	module	
-> config show Xfrin
-transfers_in:	10	integer	
-> config go Auth
-/Auth> config show
-database_file:	None	string	
-/Auth> config set database_file /tmp/bind10_zones.db
-/Auth> config commit
-/Auth> config go /
-> config show Auth/
-database_file:	/tmp/bind10_zones.db	string	
-> config diff
-{}
-> config set Auth/foobar
-Error: missing identifier or value
-> config set Auth/database_file foobar
-> config diff
-{'Auth': {'database_file': 'foobar'}}
-> config revert
-> config diff
-{}
-> quit
+Please see the wiki and the doc/ directory for various documentation.

compatcheck/Makefile.am

@@ -0,0 +1,8 @@
+noinst_SCRIPTS = sqlite3-difftbl-check.py
+
+# We're going to abuse install-data-local for a pre-install check.
+# This is to be considered a short term hack and is expected to be removed
+# in a near future version.
+install-data-local:
+	$(PYTHON) sqlite3-difftbl-check.py \
+	$(localstatedir)/$(PACKAGE)/zone.sqlite3

compatcheck/README

@@ -0,0 +1,5 @@
+This directory is a collection of compatibility checker programs.
+They will be run before any other installation attempts on 'make install'
+to see if the installation causes any substantial compatibility problems
+with existing configuratons.  If any checker program finds an issue,
+'make install' will stop at that point.

compatcheck/sqlite3-difftbl-check.py.in

@@ -0,0 +1,60 @@
+#!@PYTHON@
+
+# Copyright (C) 2011  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+import os, sqlite3, sys
+from optparse import OptionParser
+
+usage = 'usage: %prog [options] db_file'
+parser = OptionParser(usage=usage)
+parser.add_option("-u", "--upgrade", action="store_true",
+                  dest="upgrade", default=False,
+                  help="Upgrade the database file [default: %default]")
+(options, args) = parser.parse_args()
+if len(args) == 0:
+    parser.error('missing argument')
+
+db_file = args[0]
+
+# If the file doesn't exist, there's nothing to do
+if not os.path.exists(db_file):
+    sys.exit(0)
+
+conn = sqlite3.connect(db_file)
+cur = conn.cursor()
+try:
+    # This can be anything that works iff the "diffs" table exists
+    cur.execute('SELECT name FROM diffs DESC LIMIT 1')
+except sqlite3.OperationalError as ex:
+    # If it fails with 'no such table', create a new one or fail with
+    # warning depending on the --upgrade command line option.
+    if str(ex) == 'no such table: diffs':
+        if options.upgrade:
+            cur.execute('CREATE TABLE diffs (id INTEGER PRIMARY KEY, ' +
+                        'zone_id INTEGER NOT NULL, ' +
+                        'version INTEGER NOT NULL, ' +
+                        'operation INTEGER NOT NULL, ' +
+                        'name STRING NOT NULL COLLATE NOCASE, ' +
+                        'rrtype STRING NOT NULL COLLATE NOCASE, ' +
+                        'ttl INTEGER NOT NULL, rdata STRING NOT NULL)')
+        else:
+            sys.stdout.write('Found an older version of SQLite3 DB file: ' +
+                             db_file + '\n' + "Perform '" + os.getcwd() +
+                             "/sqlite3-difftbl-check.py --upgrade " +
+                             db_file + "'\n" +
+                             'before continuing install.\n')
+            sys.exit(1)
+conn.close()

configure.ac

@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.59])
-AC_INIT(bind10-devel, 20110519, bind10-dev@isc.org)
+AC_INIT(bind10-devel, 20111129, bind10-dev@isc.org)
 AC_CONFIG_SRCDIR(README)
 AM_INIT_AUTOMAKE
 AC_CONFIG_HEADERS([config.h])
@@ -12,6 +12,12 @@ AC_PROG_CXX
 
 # Libtool configuration
 #
+
+# libtool cannot handle spaces in paths, so exit early if there is one
+if [ test `echo $PWD | grep -c ' '` != "0"  ]; then
+    AC_MSG_ERROR([BIND 10 cannot be built in a directory that contains spaces, because of libtool limitations. Please change the directory name, or use a symbolic link that does not contain spaces.])
+fi
+
 # On FreeBSD (and probably some others), clang++ does not meet an autoconf
 # assumption in identifying libtool configuration regarding shared library:
 # the configure script will execute "$CC -shared $CFLAGS/$CXXFLAGS -v" and
@@ -90,6 +96,8 @@ case "$host" in
 	# Solaris requires special definitions to get some standard libraries
 	# (e.g. getopt(3)) available with common used header files.
 	CPPFLAGS="$CPPFLAGS -D_XPG4_2 -D__EXTENSIONS__"
+	# "now" binding is necessary to prevent deadlocks in C++ static initialization code
+	LDFLAGS="$LDFLAGS -z now"
 	;;
 *-apple-darwin*)
 	# libtool doesn't work perfectly with Darwin: libtool embeds the
@@ -101,6 +109,12 @@ case "$host" in
 	SET_ENV_LIBRARY_PATH=yes
 	ENV_LIBRARY_PATH=DYLD_LIBRARY_PATH
 	;;
+*-freebsd*)
+	SET_ENV_LIBRARY_PATH=yes
+	;;
+*-netbsd*)
+	SET_ENV_LIBRARY_PATH=yes
+	;;
 esac
 AM_CONDITIONAL(SET_ENV_LIBRARY_PATH, test $SET_ENV_LIBRARY_PATH = yes)
 AC_SUBST(SET_ENV_LIBRARY_PATH)
@@ -149,6 +163,16 @@ fi
 PYTHON_SITEPKG_DIR=${pyexecdir}
 AC_SUBST(PYTHON_SITEPKG_DIR)
 
+# This will be commonly used in various Makefile.am's that need to generate
+# python log messages.
+PYTHON_LOGMSGPKG_DIR="\$(top_builddir)/src/lib/python/isc/log_messages"
+AC_SUBST(PYTHON_LOGMSGPKG_DIR)
+
+# This is python package paths commonly used in python tests.  See
+# README of log_messages for why it's included.
+COMMON_PYTHON_PATH="\$(abs_top_builddir)/src/lib/python/isc/log_messages:\$(abs_top_srcdir)/src/lib/python:\$(abs_top_builddir)/src/lib/python"
+AC_SUBST(COMMON_PYTHON_PATH)
+
 # Check for python development environments
 if test -x ${PYTHON}-config; then
 	PYTHON_INCLUDES=`${PYTHON}-config --includes`
@@ -270,6 +294,8 @@ B10_CXXFLAGS="-Wall -Wextra -Wwrite-strings -Woverloaded-virtual -Wno-sign-compa
 case "$host" in
 *-solaris*)
 	MULTITHREADING_FLAG=-pthreads
+	# In Solaris, IN6ADDR_ANY_INIT and IN6ADDR_LOOPBACK_INIT need -Wno-missing-braces
+	B10_CXXFLAGS="$B10_CXXFLAGS -Wno-missing-braces"
 	;;
 *)
 	MULTITHREADING_FLAG=-pthread
@@ -419,7 +445,7 @@ AC_ARG_WITH([botan],
   AC_HELP_STRING([--with-botan=PATH],
     [specify exact directory of Botan library]),
     [botan_path="$withval"])
-if test "${botan_path}" == "no" ; then
+if test "${botan_path}" = "no" ; then
     AC_MSG_ERROR([Need botan for libcryptolink])
 fi
 if test "${botan_path}" != "yes" ; then
@@ -429,42 +455,65 @@ if test "${botan_path}" != "yes" ; then
         AC_MSG_ERROR([${botan_path}/bin/botan-config not found])
     fi
 else
+    # First see if pkg-config knows of it.
+    # Unfortunately, the botan.pc files have their minor version in them
+    # too, so we need to try them one by one
+    BOTAN_CONFIG=""
+    AC_PATH_PROG([PKG_CONFIG], [pkg-config])
+    if test "$PKG_CONFIG" != "" ; then
+        BOTAN_VERSIONS="botan-1.10 botan-1.9 botan-1.8"
+        for version in $BOTAN_VERSIONS; do
+            AC_MSG_CHECKING([Checking botan version with pkg-config $version])
+            
+            if [ $PKG_CONFIG --exists ${version} ]; then
+                AC_MSG_RESULT([found])
+                BOTAN_CONFIG="$PKG_CONFIG ${version}"
+                break
+            else
+                AC_MSG_RESULT([not found])
+            fi
+        done
+    fi
+    # If we had no pkg-config, or it didn't know about botan, use botan-config
+    if test "$BOTAN_CONFIG" = "" ; then
         AC_PATH_PROG([BOTAN_CONFIG], [botan-config])
+    fi
 fi
 
-if test -x "${BOTAN_CONFIG}" ; then
-    BOTAN_LDFLAGS=`${BOTAN_CONFIG} --libs`
-    # We expect botan-config --libs to contain -L<path_to_libbotan>, but
-    # this is not always the case.  As a heuristics workaround we add
-    # -L`botan-config --prefix/lib` in this case.  Same for BOTAN_INCLUDES
-    # (but using include instead of lib) below.
+BOTAN_LDFLAGS=`${BOTAN_CONFIG} --libs`
+BOTAN_INCLUDES=`${BOTAN_CONFIG} --cflags`
+
+# We expect botan-config --libs to contain -L<path_to_libbotan>, but
+# this is not always the case.  As a heuristics workaround we add
+# -L`botan-config --prefix/lib` in this case.  Same for BOTAN_INCLUDES
+# (but using include instead of lib) below.
+if [ $BOTAN_CONFIG --prefix >/dev/null 2>&1 ] ; then
     echo ${BOTAN_LDFLAGS} | grep -- -L > /dev/null || \
-	    BOTAN_LDFLAGS="-L`${BOTAN_CONFIG} --prefix`/lib ${BOTAN_LDFLAGS}"
-    BOTAN_INCLUDES=`${BOTAN_CONFIG} --cflags`
+        BOTAN_LDFLAGS="-L`${BOTAN_CONFIG} --prefix`/lib ${BOTAN_LDFLAGS}"
     echo ${BOTAN_INCLUDES} | grep -- -I > /dev/null || \
-	    BOTAN_INCLUDES="-I`${BOTAN_CONFIG} --prefix`/include ${BOTAN_INCLUDES}"
-    # See python_rpath for some info on why we do this
-    if test $rpath_available = yes; then
-        BOTAN_RPATH=
-        for flag in ${BOTAN_LDFLAGS}; do
-                BOTAN_RPATH="${BOTAN_RPATH} `echo $flag | sed -ne 's/^\(\-L\)/-R/p'`"
-        done
-	AC_SUBST(BOTAN_RPATH)
-
-	# According to the libtool manual, it should be sufficient if we
-	# specify the "-R libdir" in our wrapper library of botan (no other
-	# programs will need libbotan directly); "libdir" should be added to
-	# the program's binary image.  But we've seen in our build environments
-	# that (some versions of?) libtool doesn't propagate -R as documented,
-	# and it caused a linker error at run time.  To work around this, we
-	# also add the rpath to the global LDFLAGS.
-        LDFLAGS="$BOTAN_RPATH $LDFLAGS"
-    fi
-
-    AC_SUBST(BOTAN_LDFLAGS)
-    AC_SUBST(BOTAN_INCLUDES)
+        BOTAN_INCLUDES="-I`${BOTAN_CONFIG} --prefix`/include ${BOTAN_INCLUDES}"
+fi
+# See python_rpath for some info on why we do this
+if test $rpath_available = yes; then
+    BOTAN_RPATH=
+    for flag in ${BOTAN_LDFLAGS}; do
+            BOTAN_RPATH="${BOTAN_RPATH} `echo $flag | sed -ne 's/^\(\-L\)/-R/p'`"
+    done
+AC_SUBST(BOTAN_RPATH)
+
+# According to the libtool manual, it should be sufficient if we
+# specify the "-R libdir" in our wrapper library of botan (no other
+# programs will need libbotan directly); "libdir" should be added to
+# the program's binary image.  But we've seen in our build environments
+# that (some versions of?) libtool doesn't propagate -R as documented,
+# and it caused a linker error at run time.  To work around this, we
+# also add the rpath to the global LDFLAGS.
+    LDFLAGS="$BOTAN_RPATH $LDFLAGS"
 fi
 
+AC_SUBST(BOTAN_LDFLAGS)
+AC_SUBST(BOTAN_INCLUDES)
+
 CPPFLAGS_SAVED=$CPPFLAGS
 CPPFLAGS="$BOTAN_INCLUDES $CPPFLAGS"
 LDFLAGS_SAVED="$LDFLAGS"
@@ -492,7 +541,7 @@ AC_ARG_WITH([log4cplus],
   AC_HELP_STRING([--with-log4cplus=PATH],
     [specify exact directory of log4cplus library and headers]),
     [log4cplus_path="$withval"])
-if test "${log4cplus_path}" == "no" ; then
+if test "${log4cplus_path}" = "no" ; then
     AC_MSG_ERROR([Need log4cplus])
 elif test "${log4cplus_path}" != "yes" ; then
   LOG4CPLUS_INCLUDES="-I${log4cplus_path}/include"
@@ -632,6 +681,7 @@ fi
 #
 if test "$gtest_path" != "no"
 then
+	DISTCHECK_GTEST_CONFIGURE_FLAG="--with-gtest=\"$gtest_path\""
 	if test "$gtest_path" != "yes"; then
 		GTEST_PATHS=$gtest_path
 		if test -x "${gtest_path}/bin/gtest-config" ; then
@@ -672,8 +722,10 @@ else
 	GTEST_INCLUDES=
 	GTEST_LDFLAGS=
 	GTEST_LDADD=
+	DISTCHECK_GTEST_CONFIGURE_FLAG=
 fi
 AM_CONDITIONAL(HAVE_GTEST, test $gtest_path != "no")
+AC_SUBST(DISTCHECK_GTEST_CONFIGURE_FLAG)
 AC_SUBST(GTEST_INCLUDES)
 AC_SUBST(GTEST_LDFLAGS)
 AC_SUBST(GTEST_LDADD)
@@ -749,6 +801,8 @@ fi
 #
 AC_PATH_PROGS(PERL, perl5 perl)
 AC_SUBST(PERL)
+AC_PATH_PROGS(AWK, gawk awk)
+AC_SUBST(AWK)
 
 AC_ARG_ENABLE(man, [AC_HELP_STRING([--enable-man],
   [regenerate man pages [default=no]])], enable_man=yes, enable_man=no)
@@ -764,6 +818,7 @@ AM_CONDITIONAL(INSTALL_CONFIGURATIONS, test x$install_configurations = xyes || t
 AC_CONFIG_FILES([Makefile
                  doc/Makefile
                  doc/guide/Makefile
+                 compatcheck/Makefile
                  src/Makefile
                  src/bin/Makefile
                  src/bin/bind10/Makefile
@@ -793,19 +848,13 @@ AC_CONFIG_FILES([Makefile
                  src/bin/sockcreator/tests/Makefile
                  src/bin/xfrin/Makefile
                  src/bin/xfrin/tests/Makefile
+                 src/bin/xfrin/tests/testdata/Makefile
                  src/bin/xfrout/Makefile
                  src/bin/xfrout/tests/Makefile
                  src/bin/zonemgr/Makefile
                  src/bin/zonemgr/tests/Makefile
                  src/bin/stats/Makefile
                  src/bin/stats/tests/Makefile
-                 src/bin/stats/tests/isc/Makefile
-                 src/bin/stats/tests/isc/cc/Makefile
-                 src/bin/stats/tests/isc/config/Makefile
-                 src/bin/stats/tests/isc/util/Makefile
-                 src/bin/stats/tests/isc/log/Makefile
-                 src/bin/stats/tests/testdata/Makefile
-                 src/bin/stats/tests/http/Makefile
                  src/bin/usermgr/Makefile
                  src/bin/tests/Makefile
                  src/lib/Makefile
@@ -826,17 +875,24 @@ AC_CONFIG_FILES([Makefile
                  src/lib/python/isc/util/tests/Makefile
                  src/lib/python/isc/datasrc/Makefile
                  src/lib/python/isc/datasrc/tests/Makefile
+                 src/lib/python/isc/dns/Makefile
                  src/lib/python/isc/cc/Makefile
                  src/lib/python/isc/cc/tests/Makefile
                  src/lib/python/isc/config/Makefile
                  src/lib/python/isc/config/tests/Makefile
                  src/lib/python/isc/log/Makefile
                  src/lib/python/isc/log/tests/Makefile
+                 src/lib/python/isc/log_messages/Makefile
+                 src/lib/python/isc/log_messages/work/Makefile
                  src/lib/python/isc/net/Makefile
                  src/lib/python/isc/net/tests/Makefile
                  src/lib/python/isc/notify/Makefile
                  src/lib/python/isc/notify/tests/Makefile
                  src/lib/python/isc/testutils/Makefile
+                 src/lib/python/isc/bind10/Makefile
+                 src/lib/python/isc/bind10/tests/Makefile
+                 src/lib/python/isc/xfrin/Makefile
+                 src/lib/python/isc/xfrin/tests/Makefile
                  src/lib/config/Makefile
                  src/lib/config/tests/Makefile
                  src/lib/config/tests/testdata/Makefile
@@ -848,10 +904,13 @@ AC_CONFIG_FILES([Makefile
                  src/lib/dns/python/Makefile
                  src/lib/dns/python/tests/Makefile
                  src/lib/dns/benchmarks/Makefile
+                 src/lib/dhcp/Makefile
+                 src/lib/dhcp/tests/Makefile
                  src/lib/exceptions/Makefile
                  src/lib/exceptions/tests/Makefile
                  src/lib/datasrc/Makefile
                  src/lib/datasrc/tests/Makefile
+                 src/lib/datasrc/tests/testdata/Makefile
                  src/lib/xfr/Makefile
                  src/lib/log/Makefile
                  src/lib/log/compiler/Makefile
@@ -869,6 +928,7 @@ AC_CONFIG_FILES([Makefile
                  src/lib/util/Makefile
                  src/lib/util/io/Makefile
                  src/lib/util/unittests/Makefile
+                 src/lib/util/python/Makefile
                  src/lib/util/pyunittests/Makefile
                  src/lib/util/tests/Makefile
                  src/lib/acl/Makefile
@@ -882,6 +942,7 @@ AC_CONFIG_FILES([Makefile
                  tests/tools/badpacket/tests/Makefile
                ])
 AC_OUTPUT([doc/version.ent
+           compatcheck/sqlite3-difftbl-check.py
            src/bin/cfgmgr/b10-cfgmgr.py
            src/bin/cfgmgr/tests/b10-cfgmgr_test.py
            src/bin/cmdctl/cmdctl.py
@@ -904,7 +965,7 @@ AC_OUTPUT([doc/version.ent
            src/bin/zonemgr/run_b10-zonemgr.sh
            src/bin/stats/stats.py
            src/bin/stats/stats_httpd.py
-           src/bin/bind10/bind10.py
+           src/bin/bind10/bind10_src.py
            src/bin/bind10/run_bind10.sh
            src/bin/bind10/tests/bind10_test.py
            src/bin/bindctl/run_bindctl.sh
@@ -928,11 +989,12 @@ AC_OUTPUT([doc/version.ent
            src/lib/python/isc/cc/tests/cc_test
            src/lib/python/isc/notify/tests/notify_out_test
            src/lib/python/isc/log/tests/log_console.py
+           src/lib/python/isc/log_messages/work/__init__.py
            src/lib/dns/gen-rdatacode.py
            src/lib/python/bind10_config.py
-           src/lib/dns/tests/testdata/gen-wiredata.py
            src/lib/cc/session_config.h.pre
            src/lib/cc/tests/session_unittests_config.h
+           src/lib/datasrc/datasrc_config.h.pre
            src/lib/log/tests/console_test.sh
            src/lib/log/tests/destination_test.sh
            src/lib/log/tests/init_logger_test.sh
@@ -940,12 +1002,28 @@ AC_OUTPUT([doc/version.ent
            src/lib/log/tests/severity_test.sh
            src/lib/log/tests/tempdir.h
            src/lib/util/python/mkpywrapper.py
+           src/lib/util/python/gen_wiredata.py
            src/lib/server_common/tests/data_path.h
+           tests/lettuce/setup_intree_bind10.sh
            tests/system/conf.sh
+           tests/system/run.sh
            tests/system/glue/setup.sh
            tests/system/glue/nsx1/b10-config.db
            tests/system/bindctl/nsx1/b10-config.db.template
+           tests/system/ixfr/db.example.n0
+           tests/system/ixfr/db.example.n2
+           tests/system/ixfr/db.example.n2.refresh
+           tests/system/ixfr/db.example.n4
+           tests/system/ixfr/db.example.n6
+           tests/system/ixfr/ixfr_init.sh
+           tests/system/ixfr/b10-config.db
+           tests/system/ixfr/common_tests.sh
+           tests/system/ixfr/in-1/setup.sh
+           tests/system/ixfr/in-2/setup.sh
+           tests/system/ixfr/in-3/setup.sh
+           tests/system/ixfr/in-4/setup.sh
           ], [
+           chmod +x compatcheck/sqlite3-difftbl-check.py
            chmod +x src/bin/cmdctl/run_b10-cmdctl.sh
            chmod +x src/bin/xfrin/run_b10-xfrin.sh
            chmod +x src/bin/xfrout/run_b10-xfrout.sh
@@ -964,15 +1042,22 @@ AC_OUTPUT([doc/version.ent
            chmod +x src/bin/msgq/run_msgq.sh
            chmod +x src/bin/msgq/tests/msgq_test
            chmod +x src/lib/dns/gen-rdatacode.py
-           chmod +x src/lib/dns/tests/testdata/gen-wiredata.py
            chmod +x src/lib/log/tests/console_test.sh
            chmod +x src/lib/log/tests/destination_test.sh
            chmod +x src/lib/log/tests/init_logger_test.sh
            chmod +x src/lib/log/tests/local_file_test.sh
            chmod +x src/lib/log/tests/severity_test.sh
            chmod +x src/lib/util/python/mkpywrapper.py
+           chmod +x src/lib/util/python/gen_wiredata.py
            chmod +x src/lib/python/isc/log/tests/log_console.py
            chmod +x tests/system/conf.sh
+           chmod +x tests/system/run.sh
+           chmod +x tests/system/ixfr/ixfr_init.sh
+           chmod +x tests/system/ixfr/common_tests.sh
+           chmod +x tests/system/ixfr/in-1/setup.sh
+           chmod +x tests/system/ixfr/in-2/setup.sh
+           chmod +x tests/system/ixfr/in-3/setup.sh
+           chmod +x tests/system/ixfr/in-4/setup.sh
           ])
 AC_OUTPUT
 

doc/Doxyfile

@@ -568,13 +568,13 @@ WARN_LOGFILE           =
 # directories like "/usr/src/myproject". Separate the files or directories
 # with spaces.
 
-INPUT                  = ../src/lib/cc ../src/lib/config \
-    ../src/lib/cryptolink ../src/lib/dns ../src/lib/datasrc \
+INPUT                  = ../src/lib/exceptions ../src/lib/cc \
+    ../src/lib/config ../src/lib/cryptolink ../src/lib/dns ../src/lib/datasrc \
     ../src/bin/auth ../src/bin/resolver ../src/lib/bench ../src/lib/log \
     ../src/lib/log/compiler ../src/lib/asiolink/ ../src/lib/nsas \
     ../src/lib/testutils ../src/lib/cache ../src/lib/server_common/ \
     ../src/bin/sockcreator/ ../src/lib/util/ \
-    ../src/lib/resolve ../src/lib/acl
+    ../src/lib/resolve ../src/lib/acl ../src/bin/dhcp6 ../src/lib/dhcp
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is

doc/guide/Makefile.am

@@ -1,5 +1,5 @@
 EXTRA_DIST = bind10-guide.css
-EXTRA_DIST += bind10-guide.xml bind10-guide.html
+EXTRA_DIST += bind10-guide.xml bind10-guide.html bind10-guide.txt
 EXTRA_DIST += bind10-messages.xml bind10-messages.html
 
 # This is not a "man" manual, but reuse this for now for docbook.
@@ -15,6 +15,11 @@ bind10-guide.html: bind10-guide.xml
 		http://docbook.sourceforge.net/release/xsl/current/html/docbook.xsl \
 		$(srcdir)/bind10-guide.xml
 
+HTML2TXT = elinks -dump -no-numbering -no-references
+
+bind10-guide.txt: bind10-guide.html
+	$(HTML2TXT) $(srcdir)/bind10-guide.html > $@
+
 bind10-messages.html: bind10-messages.xml
 	xsltproc --novalid --xinclude --nonet \
 		--path $(top_builddir)/doc \

ext/asio/asio/impl/error_code.ipp

@@ -11,6 +11,9 @@
 #ifndef ASIO_IMPL_ERROR_CODE_IPP
 #define ASIO_IMPL_ERROR_CODE_IPP
 
+// strerror() needs <cstring>
+#include <cstring>
+
 #if defined(_MSC_VER) && (_MSC_VER >= 1200)
 # pragma once
 #endif // defined(_MSC_VER) && (_MSC_VER >= 1200)

src/bin/auth/Makefile.am

@@ -50,12 +50,19 @@ b10_auth_SOURCES += command.cc command.h
 b10_auth_SOURCES += common.h common.cc
 b10_auth_SOURCES += statistics.cc statistics.h
 b10_auth_SOURCES += main.cc
+# This is a temporary workaround for #1206, where the InMemoryClient has been
+# moved to an ldopened library. We could add that library to LDADD, but that
+# is nonportable. When #1207 is done this becomes moot anyway, and the
+# specific workaround is not needed anymore, so we can then remove this
+# line again.
+b10_auth_SOURCES += ${top_srcdir}/src/lib/datasrc/memory_datasrc.cc
 
 nodist_b10_auth_SOURCES = auth_messages.h auth_messages.cc
 EXTRA_DIST += auth_messages.mes
 
 b10_auth_LDADD =  $(top_builddir)/src/lib/datasrc/libdatasrc.la
 b10_auth_LDADD += $(top_builddir)/src/lib/dns/libdns++.la
+b10_auth_LDADD += $(top_builddir)/src/lib/util/libutil.la
 b10_auth_LDADD += $(top_builddir)/src/lib/config/libcfgclient.la
 b10_auth_LDADD += $(top_builddir)/src/lib/cc/libcc.la
 b10_auth_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la

src/bin/auth/auth.spec.pre.in

@@ -122,6 +122,24 @@
           }
         ]
       }
+    ],
+    "statistics": [
+      {
+        "item_name": "queries.tcp",
+        "item_type": "integer",
+        "item_optional": false,
+        "item_default": 0,
+        "item_title": "Queries TCP ",
+        "item_description": "A number of total query counts which all auth servers receive over TCP since they started initially"
+      },
+      {
+        "item_name": "queries.udp",
+        "item_type": "integer",
+        "item_optional": false,
+        "item_default": 0,
+        "item_title": "Queries UDP",
+        "item_description": "A number of total query counts which all auth servers receive over UDP since they started initially"
+      }
     ]
   }
 }

src/bin/auth/auth_config.cc

@@ -107,7 +107,7 @@ DatasourcesConfig::commit() {
     // server implementation details, and isn't scalable wrt the number of
     // data source types, and should eventually be improved.
     // Currently memory data source for class IN is the only possibility.
-    server_.setMemoryDataSrc(RRClass::IN(), AuthSrv::MemoryDataSrcPtr());
+    server_.setInMemoryClient(RRClass::IN(), AuthSrv::InMemoryClientPtr());
 
     BOOST_FOREACH(shared_ptr<AuthConfigParser> datasrc_config, datasources_) {
         datasrc_config->commit();
@@ -125,12 +125,12 @@ public:
     {}
     virtual void build(ConstElementPtr config_value);
     virtual void commit() {
-        server_.setMemoryDataSrc(rrclass_, memory_datasrc_);
+        server_.setInMemoryClient(rrclass_, memory_client_);
     }
 private:
     AuthSrv& server_;
     RRClass rrclass_;
-    AuthSrv::MemoryDataSrcPtr memory_datasrc_;
+    AuthSrv::InMemoryClientPtr memory_client_;
 };
 
 void
@@ -143,8 +143,8 @@ MemoryDatasourceConfig::build(ConstElementPtr config_value) {
     // We'd eventually optimize building zones (in case of reloading) by
     // selectively loading fresh zones.  Right now we simply check the
     // RR class is supported by the server implementation.
-    server_.getMemoryDataSrc(rrclass_);
-    memory_datasrc_ = AuthSrv::MemoryDataSrcPtr(new MemoryDataSrc());
+    server_.getInMemoryClient(rrclass_);
+    memory_client_ = AuthSrv::InMemoryClientPtr(new InMemoryClient());
 
     ConstElementPtr zones_config = config_value->get("zones");
     if (!zones_config) {
@@ -163,9 +163,10 @@ MemoryDatasourceConfig::build(ConstElementPtr config_value) {
             isc_throw(AuthConfigError, "Missing zone file for zone: "
                       << origin->str());
         }
-        shared_ptr<MemoryZone> new_zone(new MemoryZone(rrclass_,
+        shared_ptr<InMemoryZoneFinder> zone_finder(new
+                                                   InMemoryZoneFinder(rrclass_,
             Name(origin->stringValue())));
-        const result::Result result = memory_datasrc_->addZone(new_zone);
+        const result::Result result = memory_client_->addZone(zone_finder);
         if (result == result::EXIST) {
             isc_throw(AuthConfigError, "zone "<< origin->str()
                       << " already exists");
@@ -177,7 +178,7 @@ MemoryDatasourceConfig::build(ConstElementPtr config_value) {
          * need the load method to be split into some kind of build and
          * commit/abort parts.
          */
-        new_zone->load(file->stringValue());
+        zone_finder->load(file->stringValue());
     }
 }
 

src/bin/auth/auth_log.h

@@ -28,19 +28,19 @@ namespace auth {
 /// output.
 
 // Debug messages indicating normal startup are logged at this debug level.
-const int DBG_AUTH_START = 10;
+const int DBG_AUTH_START = DBGLVL_START_SHUT;
 
 // Debug level used to log setting information (such as configuration changes).
-const int DBG_AUTH_OPS = 30;
+const int DBG_AUTH_OPS = DBGLVL_COMMAND;
 
 // Trace detailed operations, including errors raised when processing invalid
 // packets.  (These are not logged at severities of WARN or higher for fear
 // that a set of deliberately invalid packets set to the authoritative server
 // could overwhelm the logging.)
-const int DBG_AUTH_DETAIL = 50;
+const int DBG_AUTH_DETAIL = DBGLVL_TRACE_BASIC;
 
 // This level is used to log the contents of packets received and sent.
-const int DBG_AUTH_MESSAGES = 70;
+const int DBG_AUTH_MESSAGES = DBGLVL_TRACE_DETAIL_DATA;
 
 /// Define the logger for the "auth" module part of b10-auth.  We could define
 /// a logger in each file, but we would want to define a common name to avoid

src/bin/auth/auth_messages.mes

@@ -63,7 +63,7 @@ datebase data source, listing the file that is being accessed.
 
 % AUTH_DNS_SERVICES_CREATED DNS services created
 This is a debug message indicating that the component that will handling
-incoming queries for the authoritiative server (DNSServices) has been
+incoming queries for the authoritative server (DNSServices) has been
 successfully created. It is issued during server startup is an indication
 that the initialization is proceeding normally.
 
@@ -74,7 +74,7 @@ reason for the failure is given in the message.) The server will drop the
 packet.
 
 % AUTH_LOAD_TSIG loading TSIG keys
-This is a debug message indicating that the authoritiative server
+This is a debug message indicating that the authoritative server
 has requested the keyring holding TSIG keys from the configuration
 database. It is issued during server startup is an indication that the
 initialization is proceeding normally.
@@ -141,8 +141,8 @@ encountered an internal error whilst processing a received packet:
 the cause of the error is included in the message.
 
 The server will return a SERVFAIL error code to the sender of the packet.
-However, this message indicates a potential error in the server.
-Please open a bug ticket for this issue.
+This message indicates a potential error in the server.  Please open a
+bug ticket for this issue.
 
 % AUTH_RECEIVED_COMMAND command '%1' received
 This is a debug message issued when the authoritative server has received
@@ -209,7 +209,7 @@ channel.  It is issued during server startup is an indication that the
 initialization is proceeding normally.
 
 % AUTH_STATS_COMMS communication error in sending statistics data: %1
-An error was encountered when the authoritiative server tried to send data
+An error was encountered when the authoritative server tried to send data
 to the statistics daemon. The message includes additional information
 describing the reason for the failure.
 
@@ -257,4 +257,6 @@ request. The zone manager component has been informed of the request,
 but has returned an error response (which is included in the message). The
 NOTIFY request will not be honored.
 
-
+% AUTH_INVALID_STATISTICS_DATA invalid specification of statistics data specified
+An error was encountered when the authoritiative server specified
+statistics data which is invalid for the auth specification file.

src/bin/auth/auth_srv.cc

@@ -91,9 +91,9 @@ public:
     bool processNormalQuery(const IOMessage& io_message, MessagePtr message,
                             OutputBufferPtr buffer,
                             auto_ptr<TSIGContext> tsig_context);
-    bool processAxfrQuery(const IOMessage& io_message, MessagePtr message,
-                          OutputBufferPtr buffer,
-                          auto_ptr<TSIGContext> tsig_context);
+    bool processXfrQuery(const IOMessage& io_message, MessagePtr message,
+                         OutputBufferPtr buffer,
+                         auto_ptr<TSIGContext> tsig_context);
     bool processNotify(const IOMessage& io_message, MessagePtr message,
                        OutputBufferPtr buffer,
                        auto_ptr<TSIGContext> tsig_context);
@@ -108,8 +108,8 @@ public:
     AbstractSession* xfrin_session_;
 
     /// In-memory data source.  Currently class IN only for simplicity.
-    const RRClass memory_datasrc_class_;
-    AuthSrv::MemoryDataSrcPtr memory_datasrc_;
+    const RRClass memory_client_class_;
+    AuthSrv::InMemoryClientPtr memory_client_;
 
     /// Hot spot cache
     isc::datasrc::HotCache cache_;
@@ -125,6 +125,10 @@ public:
 
     /// The TSIG keyring
     const shared_ptr<TSIGKeyRing>* keyring_;
+
+    /// Bind the ModuleSpec object in config_session_ with
+    /// isc:config::ModuleSpec::validateStatistics.
+    void registerStatisticsValidator();
 private:
     std::string db_file_;
 
@@ -139,13 +143,16 @@ private:
 
     /// Increment query counter
     void incCounter(const int protocol);
+
+    // validateStatistics
+    bool validateStatistics(isc::data::ConstElementPtr data) const;
 };
 
 AuthSrvImpl::AuthSrvImpl(const bool use_cache,
                          AbstractXfroutClient& xfrout_client) :
     config_session_(NULL),
     xfrin_session_(NULL),
-    memory_datasrc_class_(RRClass::IN()),
+    memory_client_class_(RRClass::IN()),
     statistics_timer_(io_service_),
     counters_(),
     keyring_(NULL),
@@ -212,8 +219,9 @@ class ConfigChecker : public SimpleCallback {
 public:
     ConfigChecker(AuthSrv* srv) : server_(srv) {}
     virtual void operator()(const IOMessage&) const {
-        if (server_->getConfigSession()->hasQueuedMsgs()) {
-            server_->getConfigSession()->checkCommand();
+        ModuleCCSession* cfg_session = server_->getConfigSession();
+        if (cfg_session != NULL && cfg_session->hasQueuedMsgs()) {
+            cfg_session->checkCommand();
         }
     }
 private:
@@ -317,6 +325,7 @@ AuthSrv::setXfrinSession(AbstractSession* xfrin_session) {
 void
 AuthSrv::setConfigSession(ModuleCCSession* config_session) {
     impl_->config_session_ = config_session;
+    impl_->registerStatisticsValidator();
 }
 
 void
@@ -329,34 +338,34 @@ AuthSrv::getConfigSession() const {
     return (impl_->config_session_);
 }
 
-AuthSrv::MemoryDataSrcPtr
-AuthSrv::getMemoryDataSrc(const RRClass& rrclass) {
+AuthSrv::InMemoryClientPtr
+AuthSrv::getInMemoryClient(const RRClass& rrclass) {
     // XXX: for simplicity, we only support the IN class right now.
-    if (rrclass != impl_->memory_datasrc_class_) {
+    if (rrclass != impl_->memory_client_class_) {
         isc_throw(InvalidParameter,
                   "Memory data source is not supported for RR class "
                   << rrclass);
     }
-    return (impl_->memory_datasrc_);
+    return (impl_->memory_client_);
 }
 
 void
-AuthSrv::setMemoryDataSrc(const isc::dns::RRClass& rrclass,
-                          MemoryDataSrcPtr memory_datasrc)
+AuthSrv::setInMemoryClient(const isc::dns::RRClass& rrclass,
+                           InMemoryClientPtr memory_client)
 {
     // XXX: see above
-    if (rrclass != impl_->memory_datasrc_class_) {
+    if (rrclass != impl_->memory_client_class_) {
         isc_throw(InvalidParameter,
                   "Memory data source is not supported for RR class "
                   << rrclass);
-    } else if (!impl_->memory_datasrc_ && memory_datasrc) {
+    } else if (!impl_->memory_client_ && memory_client) {
         LOG_DEBUG(auth_logger, DBG_AUTH_OPS, AUTH_MEM_DATASRC_ENABLED)
                   .arg(rrclass);
-    } else if (impl_->memory_datasrc_ && !memory_datasrc) {
+    } else if (impl_->memory_client_ && !memory_client) {
         LOG_DEBUG(auth_logger, DBG_AUTH_OPS, AUTH_MEM_DATASRC_DISABLED)
                   .arg(rrclass);
     }
-    impl_->memory_datasrc_ = memory_datasrc;
+    impl_->memory_client_ = memory_client;
 }
 
 uint32_t
@@ -464,10 +473,11 @@ AuthSrv::processMessage(const IOMessage& io_message, MessagePtr message,
         ConstQuestionPtr question = *message->beginQuestion();
         const RRType &qtype = question->getType();
         if (qtype == RRType::AXFR()) {
-            sendAnswer = impl_->processAxfrQuery(io_message, message, buffer,
-                                                 tsig_context);
+            sendAnswer = impl_->processXfrQuery(io_message, message, buffer,
+                                                tsig_context);
         } else if (qtype == RRType::IXFR()) {
-            makeErrorMessage(message, buffer, Rcode::NOTIMP(), tsig_context);
+            sendAnswer = impl_->processXfrQuery(io_message, message, buffer,
+                                                tsig_context);
         } else {
             sendAnswer = impl_->processNormalQuery(io_message, message, buffer,
                                                    tsig_context);
@@ -505,10 +515,10 @@ AuthSrvImpl::processNormalQuery(const IOMessage& io_message, MessagePtr message,
         // If a memory data source is configured call the separate
         // Query::process()
         const ConstQuestionPtr question = *message->beginQuestion();
-        if (memory_datasrc_ && memory_datasrc_class_ == question->getClass()) {
+        if (memory_client_ && memory_client_class_ == question->getClass()) {
             const RRType& qtype = question->getType();
             const Name& qname = question->getName();
-            auth::Query(*memory_datasrc_, qname, qtype, *message).process();
+            auth::Query(*memory_client_, qname, qtype, *message).process();
         } else {
             datasrc::Query query(*message, cache_, dnssec_ok);
             data_sources_.doQuery(query);
@@ -535,9 +545,9 @@ AuthSrvImpl::processNormalQuery(const IOMessage& io_message, MessagePtr message,
 }
 
 bool
-AuthSrvImpl::processAxfrQuery(const IOMessage& io_message, MessagePtr message,
-                              OutputBufferPtr buffer,
-                              auto_ptr<TSIGContext> tsig_context)
+AuthSrvImpl::processXfrQuery(const IOMessage& io_message, MessagePtr message,
+                             OutputBufferPtr buffer,
+                             auto_ptr<TSIGContext> tsig_context)
 {
     // Increment query counter.
     incCounter(io_message.getSocket().getProtocol());
@@ -670,6 +680,22 @@ AuthSrvImpl::incCounter(const int protocol) {
     }
 }
 
+void
+AuthSrvImpl::registerStatisticsValidator() {
+    counters_.registerStatisticsValidator(
+        boost::bind(&AuthSrvImpl::validateStatistics, this, _1));
+}
+
+bool
+AuthSrvImpl::validateStatistics(isc::data::ConstElementPtr data) const {
+    if (config_session_ == NULL) {
+        return (false);
+    }
+    return (
+        config_session_->getModuleSpec().validateStatistics(
+            data, true));
+}
+
 ConstElementPtr
 AuthSrvImpl::setDbFile(ConstElementPtr config) {
     ConstElementPtr answer = isc::config::createAnswer();

src/bin/auth/auth_srv.h

@@ -17,7 +17,7 @@
 
 #include <string>
 
-// For MemoryDataSrcPtr below.  This should be a temporary definition until
+// For InMemoryClientPtr below.  This should be a temporary definition until
 // we reorganize the data source framework.
 #include <boost/shared_ptr.hpp>
 
@@ -39,7 +39,7 @@
 
 namespace isc {
 namespace datasrc {
-class MemoryDataSrc;
+class InMemoryClient;
 }
 namespace xfr {
 class AbstractXfroutClient;
@@ -133,7 +133,7 @@ public:
     /// If there is a data source installed, it will be replaced with the
     /// new one.
     ///
-    /// In the current implementation, the SQLite data source and MemoryDataSrc
+    /// In the current implementation, the SQLite data source and InMemoryClient
     /// are assumed.
     /// We can enable memory data source and get the path of SQLite database by
     /// the \c config parameter.  If we disabled memory data source, the SQLite
@@ -233,16 +233,16 @@ public:
     ///
     void setXfrinSession(isc::cc::AbstractSession* xfrin_session);
 
-    /// A shared pointer type for \c MemoryDataSrc.
+    /// A shared pointer type for \c InMemoryClient.
     ///
     /// This is defined inside the \c AuthSrv class as it's supposed to be
     /// a short term interface until we integrate the in-memory and other
     /// data source frameworks.
-    typedef boost::shared_ptr<isc::datasrc::MemoryDataSrc> MemoryDataSrcPtr;
+    typedef boost::shared_ptr<isc::datasrc::InMemoryClient> InMemoryClientPtr;
 
-    /// An immutable shared pointer type for \c MemoryDataSrc.
-    typedef boost::shared_ptr<const isc::datasrc::MemoryDataSrc>
-    ConstMemoryDataSrcPtr;
+    /// An immutable shared pointer type for \c InMemoryClient.
+    typedef boost::shared_ptr<const isc::datasrc::InMemoryClient>
+    ConstInMemoryClientPtr;
 
     /// Returns the in-memory data source configured for the \c AuthSrv,
     /// if any.
@@ -260,11 +260,11 @@ public:
     /// \param rrclass The RR class of the requested in-memory data source.
     /// \return A pointer to the in-memory data source, if configured;
     /// otherwise NULL.
-    MemoryDataSrcPtr getMemoryDataSrc(const isc::dns::RRClass& rrclass);
+    InMemoryClientPtr getInMemoryClient(const isc::dns::RRClass& rrclass);
 
     /// Sets or replaces the in-memory data source of the specified RR class.
     ///
-    /// As noted in \c getMemoryDataSrc(), some RR classes may not be
+    /// As noted in \c getInMemoryClient(), some RR classes may not be
     /// supported, in which case an exception of class \c InvalidParameter
     /// will be thrown.
     /// This method never throws an exception otherwise.
@@ -275,9 +275,9 @@ public:
     /// in-memory data source.
     ///
     /// \param rrclass The RR class of the in-memory data source to be set.
-    /// \param memory_datasrc A (shared) pointer to \c MemoryDataSrc to be set.
-    void setMemoryDataSrc(const isc::dns::RRClass& rrclass,
-                          MemoryDataSrcPtr memory_datasrc);
+    /// \param memory_datasrc A (shared) pointer to \c InMemoryClient to be set.
+    void setInMemoryClient(const isc::dns::RRClass& rrclass,
+                           InMemoryClientPtr memory_client);
 
     /// \brief Set the communication session with Statistics.
     ///

src/bin/auth/b10-auth.8

@@ -2,12 +2,12 @@
 .\"     Title: b10-auth
 .\"    Author: [FIXME: author] [see http://docbook.sf.net/el/author]
 .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
-.\"      Date: March 8, 2011
+.\"      Date: August 11, 2011
 .\"    Manual: BIND10
 .\"    Source: BIND10
 .\"  Language: English
 .\"
-.TH "B10\-AUTH" "8" "March 8, 2011" "BIND10" "BIND10"
+.TH "B10\-AUTH" "8" "August 11, 2011" "BIND10" "BIND10"
 .\" -----------------------------------------------------------------
 .\" * set default formatting
 .\" -----------------------------------------------------------------
@@ -70,18 +70,6 @@ defines the path to the SQLite3 zone file when using the sqlite datasource\&. Th
 /usr/local/var/bind10\-devel/zone\&.sqlite3\&.
 .PP
 
-\fIlisten_on\fR
-is a list of addresses and ports for
-\fBb10\-auth\fR
-to listen on\&. The list items are the
-\fIaddress\fR
-string and
-\fIport\fR
-number\&. By default,
-\fBb10\-auth\fR
-listens on port 53 on the IPv6 (::) and IPv4 (0\&.0\&.0\&.0) wildcard addresses\&.
-.PP
-
 \fIdatasources\fR
 configures data sources\&. The list items include:
 \fItype\fR
@@ -114,6 +102,18 @@ In this development version, currently this is only used for the memory data sou
 .RE
 .PP
 
+\fIlisten_on\fR
+is a list of addresses and ports for
+\fBb10\-auth\fR
+to listen on\&. The list items are the
+\fIaddress\fR
+string and
+\fIport\fR
+number\&. By default,
+\fBb10\-auth\fR
+listens on port 53 on the IPv6 (::) and IPv4 (0\&.0\&.0\&.0) wildcard addresses\&.
+.PP
+
 \fIstatistics\-interval\fR
 is the timer interval in seconds for
 \fBb10\-auth\fR
@@ -164,6 +164,25 @@ immediately\&.
 \fBshutdown\fR
 exits
 \fBb10\-auth\fR\&. (Note that the BIND 10 boss process will restart this service\&.)
+.SH "STATISTICS DATA"
+.PP
+The statistics data collected by the
+\fBb10\-stats\fR
+daemon include:
+.PP
+auth\&.queries\&.tcp
+.RS 4
+Total count of queries received by the
+\fBb10\-auth\fR
+server over TCP since startup\&.
+.RE
+.PP
+auth\&.queries\&.udp
+.RS 4
+Total count of queries received by the
+\fBb10\-auth\fR
+server over UDP since startup\&.
+.RE
 .SH "FILES"
 .PP
 

src/bin/auth/b10-auth.xml

@@ -20,7 +20,7 @@
 <refentry>
 
   <refentryinfo>
-    <date>March 8, 2011</date>
+    <date>August 11, 2011</date>
   </refentryinfo>
 
   <refmeta>
@@ -132,15 +132,6 @@
     </para>
 
     <para>
-      <varname>listen_on</varname> is a list of addresses and ports for
-      <command>b10-auth</command> to listen on.
-      The list items are the <varname>address</varname> string
-      and <varname>port</varname> number.
-      By default, <command>b10-auth</command> listens on port 53
-      on the IPv6 (::) and IPv4 (0.0.0.0) wildcard addresses.
-    </para>
-
-    <para>
       <varname>datasources</varname> configures data sources.
       The list items include:
       <varname>type</varname> to optionally choose the data source type
@@ -165,6 +156,15 @@
     </para>
 
     <para>
+      <varname>listen_on</varname> is a list of addresses and ports for
+      <command>b10-auth</command> to listen on.
+      The list items are the <varname>address</varname> string
+      and <varname>port</varname> number.
+      By default, <command>b10-auth</command> listens on port 53
+      on the IPv6 (::) and IPv4 (0.0.0.0) wildcard addresses.
+    </para>
+
+    <para>
       <varname>statistics-interval</varname> is the timer interval
       in seconds for <command>b10-auth</command> to share its
       statistics information to
@@ -209,6 +209,34 @@
   </refsect1>
 
   <refsect1>
+    <title>STATISTICS DATA</title>
+
+    <para>
+      The statistics data collected by the <command>b10-stats</command>
+      daemon include:
+    </para>
+
+    <variablelist>
+
+      <varlistentry>
+        <term>auth.queries.tcp</term>
+        <listitem><simpara>Total count of queries received by the
+          <command>b10-auth</command> server over TCP since startup.
+        </simpara></listitem>
+      </varlistentry>
+
+      <varlistentry>
+        <term>auth.queries.udp</term>
+        <listitem><simpara>Total count of queries received by the
+          <command>b10-auth</command> server over UDP since startup.
+        </simpara></listitem>
+      </varlistentry>
+
+    </variablelist>
+
+  </refsect1>
+
+  <refsect1>
     <title>FILES</title>
     <para>
       <filename>/usr/local/var/bind10-devel/zone.sqlite3</filename>

src/bin/auth/benchmarks/Makefile.am

@@ -13,10 +13,17 @@ query_bench_SOURCES += ../auth_srv.h ../auth_srv.cc
 query_bench_SOURCES += ../auth_config.h ../auth_config.cc
 query_bench_SOURCES += ../statistics.h ../statistics.cc
 query_bench_SOURCES += ../auth_log.h ../auth_log.cc
+# This is a temporary workaround for #1206, where the InMemoryClient has been
+# moved to an ldopened library. We could add that library to LDADD, but that
+# is nonportable. When #1207 is done this becomes moot anyway, and the
+# specific workaround is not needed anymore, so we can then remove this
+# line again.
+query_bench_SOURCES += ${top_srcdir}/src/lib/datasrc/memory_datasrc.cc
 
 nodist_query_bench_SOURCES = ../auth_messages.h ../auth_messages.cc
 
 query_bench_LDADD = $(top_builddir)/src/lib/dns/libdns++.la
+query_bench_LDADD += $(top_builddir)/src/lib/util/libutil.la
 query_bench_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la
 query_bench_LDADD += $(top_builddir)/src/lib/bench/libbench.la
 query_bench_LDADD += $(top_builddir)/src/lib/datasrc/libdatasrc.la

src/bin/auth/command.cc

@@ -136,19 +136,21 @@ public:
         // that doesn't block other server operations.
         // TODO: we may (should?) want to check the "last load time" and
         // the timestamp of the file and skip loading if the file isn't newer.
-        shared_ptr<MemoryZone> newzone(new MemoryZone(oldzone->getClass(),
-                                                      oldzone->getOrigin()));
-        newzone->load(oldzone->getFileName());
-        oldzone->swap(*newzone);
+        shared_ptr<InMemoryZoneFinder> zone_finder(
+            new InMemoryZoneFinder(old_zone_finder->getClass(),
+                                   old_zone_finder->getOrigin()));
+        zone_finder->load(old_zone_finder->getFileName());
+        old_zone_finder->swap(*zone_finder);
         LOG_DEBUG(auth_logger, DBG_AUTH_OPS, AUTH_LOAD_ZONE)
-                  .arg(newzone->getOrigin()).arg(newzone->getClass());
+                  .arg(zone_finder->getOrigin()).arg(zone_finder->getClass());
     }
 
 private:
-    shared_ptr<MemoryZone> oldzone; // zone to be updated with the new file.
+    // zone finder to be updated with the new file.
+    shared_ptr<InMemoryZoneFinder> old_zone_finder;
 
     // A helper private method to parse and validate command parameters.
-    // On success, it sets 'oldzone' to the zone to be updated.
+    // On success, it sets 'old_zone_finder' to the zone to be updated.
     // It returns true if everything is okay; and false if the command is
     // valid but there's no need for further process.
     bool validate(AuthSrv& server, isc::data::ConstElementPtr args) {
@@ -176,7 +178,7 @@ private:
         const RRClass zone_class = class_elem ?
             RRClass(class_elem->stringValue()) : RRClass::IN();
 
-        AuthSrv::MemoryDataSrcPtr datasrc(server.getMemoryDataSrc(zone_class));
+        AuthSrv::InMemoryClientPtr datasrc(server.getInMemoryClient(zone_class));
         if (datasrc == NULL) {
             isc_throw(AuthCommandError, "Memory data source is disabled");
         }
@@ -188,13 +190,14 @@ private:
         const Name origin(origin_elem->stringValue());
 
         // Get the current zone
-        const MemoryDataSrc::FindResult result = datasrc->findZone(origin);
+        const InMemoryClient::FindResult result = datasrc->findZone(origin);
         if (result.code != result::SUCCESS) {
             isc_throw(AuthCommandError, "Zone " << origin <<
                       " is not found in data source");
         }
 
-        oldzone = boost::dynamic_pointer_cast<MemoryZone>(result.zone);
+        old_zone_finder = boost::dynamic_pointer_cast<InMemoryZoneFinder>(
+            result.zone_finder);
 
         return (true);
     }

src/bin/auth/common.cc

@@ -12,22 +12,25 @@
 // OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 // PERFORMANCE OF THIS SOFTWARE.
 
+#include <string>
+
 #include <auth/common.h>
 #include <auth/spec_config.h>
 #include <stdlib.h>
 
 using std::string;
 
-string getXfroutSocketPath() {
+string
+getXfroutSocketPath() {
     if (getenv("B10_FROM_BUILD") != NULL) {
-        if (getenv("B10_FROM_SOURCE_LOCALSTATEDIR")) {
+        if (getenv("B10_FROM_SOURCE_LOCALSTATEDIR") != NULL) {
             return (string(getenv("B10_FROM_SOURCE_LOCALSTATEDIR")) +
                     "/auth_xfrout_conn");
         } else {
             return (string(getenv("B10_FROM_BUILD")) + "/auth_xfrout_conn");
         }
     } else {
-        if (getenv("BIND10_XFROUT_SOCKET_FILE")) {
+        if (getenv("BIND10_XFROUT_SOCKET_FILE") != NULL) {
             return (getenv("BIND10_XFROUT_SOCKET_FILE"));
         } else {
             return (UNIX_SOCKET_FILE);

src/bin/auth/query.cc

@@ -12,6 +12,7 @@
 // OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 // PERFORMANCE OF THIS SOFTWARE.
 
+#include <algorithm>            // for std::max
 #include <vector>
 #include <boost/foreach.hpp>
 
@@ -19,7 +20,7 @@
 #include <dns/rcode.h>
 #include <dns/rdataclass.h>
 
-#include <datasrc/memory_datasrc.h>
+#include <datasrc/client.h>
 
 #include <auth/query.h>
 
@@ -31,24 +32,24 @@ namespace isc {
 namespace auth {
 
 void
-Query::getAdditional(const Zone& zone, const RRset& rrset) const {
+Query::addAdditional(ZoneFinder& zone, const RRset& rrset) {
     RdataIteratorPtr rdata_iterator(rrset.getRdataIterator());
     for (; !rdata_iterator->isLast(); rdata_iterator->next()) {
         const Rdata& rdata(rdata_iterator->getCurrent());
         if (rrset.getType() == RRType::NS()) {
             // Need to perform the search in the "GLUE OK" mode.
             const generic::NS& ns = dynamic_cast<const generic::NS&>(rdata);
-            findAddrs(zone, ns.getNSName(), Zone::FIND_GLUE_OK);
+            addAdditionalAddrs(zone, ns.getNSName(), ZoneFinder::FIND_GLUE_OK);
         } else if (rrset.getType() == RRType::MX()) {
             const generic::MX& mx(dynamic_cast<const generic::MX&>(rdata));
-            findAddrs(zone, mx.getMXName());
+            addAdditionalAddrs(zone, mx.getMXName());
         }
     }
 }
 
 void
-Query::findAddrs(const Zone& zone, const Name& qname,
-                 const Zone::FindOptions options) const
+Query::addAdditionalAddrs(ZoneFinder& zone, const Name& qname,
+                          const ZoneFinder::FindOptions options)
 {
     // Out of zone name
     NameComparisonResult result = zone.getOrigin().compare(qname);
@@ -66,32 +67,33 @@ Query::findAddrs(const Zone& zone, const Name& qname,
 
     // Find A rrset
     if (qname_ != qname || qtype_ != RRType::A()) {
-        Zone::FindResult a_result = zone.find(qname, RRType::A(), NULL,
-                                              options);
-        if (a_result.code == Zone::SUCCESS) {
+        ZoneFinder::FindResult a_result = zone.find(qname, RRType::A(), NULL,
+                                                    options | dnssec_opt_);
+        if (a_result.code == ZoneFinder::SUCCESS) {
             response_.addRRset(Message::SECTION_ADDITIONAL,
-                    boost::const_pointer_cast<RRset>(a_result.rrset));
+                    boost::const_pointer_cast<RRset>(a_result.rrset), dnssec_);
         }
     }
 
     // Find AAAA rrset
     if (qname_ != qname || qtype_ != RRType::AAAA()) {
-        Zone::FindResult aaaa_result =
-            zone.find(qname, RRType::AAAA(), NULL, options);
-        if (aaaa_result.code == Zone::SUCCESS) {
+        ZoneFinder::FindResult aaaa_result =
+            zone.find(qname, RRType::AAAA(), NULL, options | dnssec_opt_);
+        if (aaaa_result.code == ZoneFinder::SUCCESS) {
             response_.addRRset(Message::SECTION_ADDITIONAL,
-                    boost::const_pointer_cast<RRset>(aaaa_result.rrset));
+                    boost::const_pointer_cast<RRset>(aaaa_result.rrset),
+                    dnssec_);
         }
     }
 }
 
 void
-Query::putSOA(const Zone& zone) const {
-    Zone::FindResult soa_result(zone.find(zone.getOrigin(),
-        RRType::SOA()));
-    if (soa_result.code != Zone::SUCCESS) {
+Query::addSOA(ZoneFinder& finder) {
+    ZoneFinder::FindResult soa_result(finder.find(finder.getOrigin(),
+        RRType::SOA(), NULL, dnssec_opt_));
+    if (soa_result.code != ZoneFinder::SUCCESS) {
         isc_throw(NoSOA, "There's no SOA record in zone " <<
-            zone.getOrigin().toText());
+            finder.getOrigin().toText());
     } else {
         /*
          * FIXME:
@@ -99,34 +101,140 @@ Query::putSOA(const Zone& zone) const {
          * to insist.
          */
         response_.addRRset(Message::SECTION_AUTHORITY,
-            boost::const_pointer_cast<RRset>(soa_result.rrset));
+            boost::const_pointer_cast<RRset>(soa_result.rrset), dnssec_);
     }
 }
 
+// Note: unless the data source client implementation or the zone content
+// is broken, 'nsec' should be a valid NSEC RR.  Likewise, the call to
+// find() in this method should result in NXDOMAIN and an NSEC RR that proves
+// the non existent of matching wildcard.  If these assumptions aren't met
+// due to a buggy data source implementation or a broken zone, we'll let
+// underlying libdns++ modules throw an exception, which would result in
+// either an SERVFAIL response or just ignoring the query.  We at least prevent
+// a complete crash due to such broken behavior.
 void
-Query::getAuthAdditional(const Zone& zone) const {
+Query::addNXDOMAINProof(ZoneFinder& finder, ConstRRsetPtr nsec) {
+    if (nsec->getRdataCount() == 0) {
+        isc_throw(BadNSEC, "NSEC for NXDOMAIN is empty");
+    }
+
+    // Add the NSEC proving NXDOMAIN to the authority section.
+    response_.addRRset(Message::SECTION_AUTHORITY,
+                       boost::const_pointer_cast<RRset>(nsec), dnssec_);
+
+    // Next, identify the best possible wildcard name that would match
+    // the query name.  It's the longer common suffix with the qname
+    // between the owner or the next domain of the NSEC that proves NXDOMAIN,
+    // prefixed by the wildcard label, "*".  For example, for query name
+    // a.b.example.com, if the NXDOMAIN NSEC is
+    // b.example.com. NSEC c.example.com., the longer suffix is b.example.com.,
+    // and the best possible wildcard is *.b.example.com.  If the NXDOMAIN
+    // NSEC is a.example.com. NSEC c.b.example.com., the longer suffix
+    // is the next domain of the NSEC, and we get the same wildcard name.
+    const int qlabels = qname_.getLabelCount();
+    const int olabels = qname_.compare(nsec->getName()).getCommonLabels();
+    const int nlabels = qname_.compare(
+        dynamic_cast<const generic::NSEC&>(nsec->getRdataIterator()->
+                                           getCurrent()).
+        getNextName()).getCommonLabels();
+    const int common_labels = std::max(olabels, nlabels);
+    const Name wildname(Name("*").concatenate(qname_.split(qlabels -
+                                                           common_labels)));
+
+    // Confirm the wildcard doesn't exist (this should result in NXDOMAIN;
+    // otherwise we shouldn't have got NXDOMAIN for the original query in
+    // the first place).
+    const ZoneFinder::FindResult fresult = finder.find(wildname,
+                                                       RRType::NSEC(), NULL,
+                                                       dnssec_opt_);
+    if (fresult.code != ZoneFinder::NXDOMAIN || !fresult.rrset ||
+        fresult.rrset->getRdataCount() == 0) {
+        isc_throw(BadNSEC, "Unexpected result for wildcard NXDOMAIN proof");
+    }
+
+    // Add the (no-) wildcard proof only when it's different from the NSEC
+    // that proves NXDOMAIN; sometimes they can be the same.
+    // Note: name comparison is relatively expensive.  When we are at the
+    // stage of performance optimization, we should consider optimizing this
+    // for some optimized data source implementations.
+    if (nsec->getName() != fresult.rrset->getName()) {
+        response_.addRRset(Message::SECTION_AUTHORITY,
+                           boost::const_pointer_cast<RRset>(fresult.rrset),
+                           dnssec_);
+    }
+}
+
+void
+Query::addWildcardProof(ZoneFinder& finder) {
+    // The query name shouldn't exist in the zone if there were no wildcard
+    // substitution.  Confirm that by specifying NO_WILDCARD.  It should result
+    // in NXDOMAIN and an NSEC RR that proves it should be returned.
+    const ZoneFinder::FindResult fresult =
+        finder.find(qname_, RRType::NSEC(), NULL,
+                    dnssec_opt_ | ZoneFinder::NO_WILDCARD);
+    if (fresult.code != ZoneFinder::NXDOMAIN || !fresult.rrset ||
+        fresult.rrset->getRdataCount() == 0) {
+        isc_throw(BadNSEC, "Unexpected result for wildcard proof");
+    }
+    response_.addRRset(Message::SECTION_AUTHORITY,
+                       boost::const_pointer_cast<RRset>(fresult.rrset),
+                       dnssec_);
+}
+
+void
+Query::addWildcardNXRRSETProof(ZoneFinder& finder, ConstRRsetPtr nsec) {
+    // There should be one NSEC RR which was found in the zone to prove
+    // that there is not matched <QNAME,QTYPE> via wildcard expansion.
+    if (nsec->getRdataCount() == 0) {
+        isc_throw(BadNSEC, "NSEC for WILDCARD_NXRRSET is empty");
+    }
+    // Add this NSEC RR to authority section.
+    response_.addRRset(Message::SECTION_AUTHORITY,
+                      boost::const_pointer_cast<RRset>(nsec), dnssec_);
+    
+    const ZoneFinder::FindResult fresult =
+        finder.find(qname_, RRType::NSEC(), NULL,
+                    dnssec_opt_ | ZoneFinder::NO_WILDCARD);
+    if (fresult.code != ZoneFinder::NXDOMAIN || !fresult.rrset ||
+        fresult.rrset->getRdataCount() == 0) {
+        isc_throw(BadNSEC, "Unexpected result for no match QNAME proof");
+    }
+   
+    if (nsec->getName() != fresult.rrset->getName()) {
+        // one NSEC RR proves wildcard_nxrrset that no matched QNAME.
+        response_.addRRset(Message::SECTION_AUTHORITY,
+                           boost::const_pointer_cast<RRset>(fresult.rrset),
+                           dnssec_);
+    }
+}
+    
+void
+Query::addAuthAdditional(ZoneFinder& finder) {
     // Fill in authority and addtional sections.
-    Zone::FindResult ns_result = zone.find(zone.getOrigin(), RRType::NS());
+    ZoneFinder::FindResult ns_result = finder.find(finder.getOrigin(),
+                                                   RRType::NS(), NULL,
+                                                   dnssec_opt_);
     // zone origin name should have NS records
-    if (ns_result.code != Zone::SUCCESS) {
+    if (ns_result.code != ZoneFinder::SUCCESS) {
         isc_throw(NoApexNS, "There's no apex NS records in zone " <<
-                zone.getOrigin().toText());
+                finder.getOrigin().toText());
     } else {
         response_.addRRset(Message::SECTION_AUTHORITY,
-            boost::const_pointer_cast<RRset>(ns_result.rrset));
+            boost::const_pointer_cast<RRset>(ns_result.rrset), dnssec_);
         // Handle additional for authority section
-        getAdditional(zone, *ns_result.rrset);
+        addAdditional(finder, *ns_result.rrset);
     }
 }
 
 void
-Query::process() const {
+Query::process() {
     bool keep_doing = true;
     const bool qtype_is_any = (qtype_ == RRType::ANY());
 
     response_.setHeaderFlag(Message::HEADERFLAG_AA, false);
-    const MemoryDataSrc::FindResult result =
-        memory_datasrc_.findZone(qname_);
+    const DataSourceClient::FindResult result =
+        datasrc_client_.findZone(qname_);
 
     // If we have no matching authoritative zone for the query name, return
     // REFUSED.  In short, this is to be compatible with BIND 9, but the
@@ -138,6 +246,7 @@ Query::process() const {
         response_.setRcode(Rcode::REFUSED());
         return;
     }
+    ZoneFinder& zfinder = *result.zone_finder;
 
     // Found a zone which is the nearest ancestor to QNAME, set the AA bit
     response_.setHeaderFlag(Message::HEADERFLAG_AA);
@@ -145,14 +254,14 @@ Query::process() const {
     while (keep_doing) {
         keep_doing = false;
         std::auto_ptr<RRsetList> target(qtype_is_any ? new RRsetList : NULL);
-        const Zone::FindResult db_result(result.zone->find(qname_, qtype_,
-            target.get()));
-
+        const ZoneFinder::FindResult db_result(
+            zfinder.find(qname_, qtype_, target.get(), dnssec_opt_));
         switch (db_result.code) {
-            case Zone::DNAME: {
+            case ZoneFinder::DNAME: {
                 // First, put the dname into the answer
                 response_.addRRset(Message::SECTION_ANSWER,
-                    boost::const_pointer_cast<RRset>(db_result.rrset));
+                    boost::const_pointer_cast<RRset>(db_result.rrset),
+                    dnssec_);
                 /*
                  * Empty DNAME should never get in, as it is impossible to
                  * create one in master file.
@@ -188,10 +297,11 @@ Query::process() const {
                     qname_.getLabelCount() -
                     db_result.rrset->getName().getLabelCount()).
                     concatenate(dname.getDname())));
-                response_.addRRset(Message::SECTION_ANSWER, cname);
+                response_.addRRset(Message::SECTION_ANSWER, cname, dnssec_);
                 break;
             }
-            case Zone::CNAME:
+            case ZoneFinder::CNAME:
+            case ZoneFinder::WILDCARD_CNAME:
                 /*
                  * We don't do chaining yet. Therefore handling a CNAME is
                  * mostly the same as handling SUCCESS, but we didn't get
@@ -202,48 +312,84 @@ Query::process() const {
                  * So, just put it there.
                  */
                 response_.addRRset(Message::SECTION_ANSWER,
-                    boost::const_pointer_cast<RRset>(db_result.rrset));
+                    boost::const_pointer_cast<RRset>(db_result.rrset),
+                    dnssec_);
+
+                // If the answer is a result of wildcard substitution,
+                // add a proof that there's no closer name.
+                if (dnssec_ && db_result.code == ZoneFinder::WILDCARD_CNAME) {
+                    addWildcardProof(*result.zone_finder);
+                }
                 break;
-            case Zone::SUCCESS:
+            case ZoneFinder::SUCCESS:
+            case ZoneFinder::WILDCARD:
                 if (qtype_is_any) {
                     // If quety type is ANY, insert all RRs under the domain
                     // into answer section.
                     BOOST_FOREACH(RRsetPtr rrset, *target) {
-                        response_.addRRset(Message::SECTION_ANSWER, rrset);
+                        response_.addRRset(Message::SECTION_ANSWER, rrset,
+                                           dnssec_);
                         // Handle additional for answer section
-                        getAdditional(*result.zone, *rrset.get());
+                        addAdditional(*result.zone_finder, *rrset.get());
                     }
                 } else {
                     response_.addRRset(Message::SECTION_ANSWER,
-                        boost::const_pointer_cast<RRset>(db_result.rrset));
+                        boost::const_pointer_cast<RRset>(db_result.rrset),
+                        dnssec_);
                     // Handle additional for answer section
-                    getAdditional(*result.zone, *db_result.rrset);
+                    addAdditional(*result.zone_finder, *db_result.rrset);
                 }
                 // If apex NS records haven't been provided in the answer
                 // section, insert apex NS records into the authority section
                 // and AAAA/A RRS of each of the NS RDATA into the additional
                 // section.
-                if (qname_ != result.zone->getOrigin() ||
-                    db_result.code != Zone::SUCCESS ||
+                if (qname_ != result.zone_finder->getOrigin() ||
+                    db_result.code != ZoneFinder::SUCCESS ||
                     (qtype_ != RRType::NS() && !qtype_is_any))
                 {
-                    getAuthAdditional(*result.zone);
+                    addAuthAdditional(*result.zone_finder);
+                }
+
+                // If the answer is a result of wildcard substitution,
+                // add a proof that there's no closer name.
+                if (dnssec_ && db_result.code == ZoneFinder::WILDCARD) {
+                    addWildcardProof(*result.zone_finder);
                 }
                 break;
-            case Zone::DELEGATION:
+            case ZoneFinder::DELEGATION:
                 response_.setHeaderFlag(Message::HEADERFLAG_AA, false);
                 response_.addRRset(Message::SECTION_AUTHORITY,
-                    boost::const_pointer_cast<RRset>(db_result.rrset));
-                getAdditional(*result.zone, *db_result.rrset);
+                    boost::const_pointer_cast<RRset>(db_result.rrset),
+                    dnssec_);
+                addAdditional(*result.zone_finder, *db_result.rrset);
                 break;
-            case Zone::NXDOMAIN:
-                // Just empty answer with SOA in authority section
+            case ZoneFinder::NXDOMAIN:
                 response_.setRcode(Rcode::NXDOMAIN());
-                putSOA(*result.zone);
+                addSOA(*result.zone_finder);
+                if (dnssec_ && db_result.rrset) {
+                    addNXDOMAINProof(zfinder, db_result.rrset);
+                }
+                break;
+            case ZoneFinder::NXRRSET:
+                addSOA(*result.zone_finder);
+                if (dnssec_ && db_result.rrset) {
+                    response_.addRRset(Message::SECTION_AUTHORITY,
+                                       boost::const_pointer_cast<RRset>(
+                                           db_result.rrset),
+                                       dnssec_);
+                }
+                break;
+            case ZoneFinder::WILDCARD_NXRRSET:
+                addSOA(*result.zone_finder);
+                if (dnssec_ && db_result.rrset) {
+                    addWildcardNXRRSETProof(zfinder, db_result.rrset);
+                }
                 break;
-            case Zone::NXRRSET:
-                // Just empty answer with SOA in authority section
-                putSOA(*result.zone);
+            default:
+                // This is basically a bug of the data source implementation,
+                // but could also happen in the middle of development where
+                // we try to add a new result code.
+                isc_throw(isc::NotImplemented, "Unknown result code");
                 break;
         }
     }

src/bin/auth/query.h

@@ -26,7 +26,7 @@ class RRset;
 }
 
 namespace datasrc {
-class MemoryDataSrc;
+class DataSourceClient;
 }
 
 namespace auth {
@@ -36,10 +36,8 @@ namespace auth {
 ///
 /// Many of the design details for this class are still in flux.
 /// We'll revisit and update them as we add more functionality, for example:
-/// - memory_datasrc parameter of the constructor.  It is a data source that
-///   uses in memory dedicated backend.
 /// - as a related point, we may have to pass the RR class of the query.
-///   in the initial implementation the RR class is an attribute of memory
+///   in the initial implementation the RR class is an attribute of
 ///   datasource and omitted.  It's not clear if this assumption holds with
 ///   generic data sources.  On the other hand, it will help keep
 ///   implementation simpler, and we might rather want to modify the design
@@ -51,7 +49,7 @@ namespace auth {
 ///   separate attribute setter.
 /// - likewise, we'll eventually need to do per zone access control, for which
 ///   we need querier's information such as its IP address.
-/// - memory_datasrc and response may better be parameters to process() instead
+/// - datasrc_client and response may better be parameters to process() instead
 ///   of the constructor.
 ///
 /// <b>Note:</b> The class name is intentionally the same as the one used in
@@ -71,10 +69,33 @@ private:
     /// Adds a SOA of the zone into the authority zone of response_.
     /// Can throw NoSOA.
     ///
-    void putSOA(const isc::datasrc::Zone& zone) const;
+    void addSOA(isc::datasrc::ZoneFinder& finder);
 
+    /// Add NSEC RRs that prove an NXDOMAIN result.
+    ///
+    /// This corresponds to Section 3.1.3.2 of RFC 4035.
+    void addNXDOMAINProof(isc::datasrc::ZoneFinder& finder,
+                          isc::dns::ConstRRsetPtr nsec);
+
+    /// Add NSEC RRs that prove a wildcard answer is the best one.
+    ///
+    /// This corresponds to Section 3.1.3.3 of RFC 4035.
+    void addWildcardProof(isc::datasrc::ZoneFinder& finder);
+
+    /// \brief Adds one NSEC RR proved no matched QNAME,one NSEC RR proved no
+    /// matched <QNAME,QTYPE> through wildcard extension.
+    ///
+    /// Add NSEC RRs that prove an WILDCARD_NXRRSET result.
+    /// This corresponds to Section 3.1.3.4 of RFC 4035.
+    /// \param finder The ZoneFinder through which the authority data for the
+    /// query is to be found.
+    /// \param nsec The RRset (NSEC RR) which proved that there is no matched 
+    /// <QNAME,QTTYPE>.
+    void addWildcardNXRRSETProof(isc::datasrc::ZoneFinder& finder,
+                                 isc::dns::ConstRRsetPtr nsec);
+    
     /// \brief Look up additional data (i.e., address records for the names
-    /// included in NS or MX records).
+    /// included in NS or MX records) and add them to the additional section.
     ///
     /// Note: Any additional data which has already been provided in the
     /// answer section (i.e., if the original query happend to be for the
@@ -83,12 +104,12 @@ private:
     /// This method may throw a exception because its underlying methods may
     /// throw exceptions.
     ///
-    /// \param zone The Zone wherein the additional data to the query is bo be
-    /// found.
+    /// \param zone The ZoneFinder through which the additional data for the
+    /// query is to be found.
     /// \param rrset The RRset (i.e., NS or MX rrset) which require additional
     /// processing.
-    void getAdditional(const isc::datasrc::Zone& zone,
-                       const isc::dns::RRset& rrset) const;
+    void addAdditional(isc::datasrc::ZoneFinder& zone,
+                       const isc::dns::RRset& rrset);
 
     /// \brief Find address records for a specified name.
     ///
@@ -102,18 +123,19 @@ private:
     /// The glue records must exactly match the name in the NS RDATA, without
     /// CNAME or wildcard processing.
     ///
-    /// \param zone The \c Zone wherein the address records is to be found.
+    /// \param zone The \c ZoneFinder through which the address records is to
+    /// be found.
     /// \param qname The name in rrset RDATA.
     /// \param options The search options.
-    void findAddrs(const isc::datasrc::Zone& zone,
-                   const isc::dns::Name& qname,
-                   const isc::datasrc::Zone::FindOptions options
-                   = isc::datasrc::Zone::FIND_DEFAULT) const;
+    void addAdditionalAddrs(isc::datasrc::ZoneFinder& zone,
+                            const isc::dns::Name& qname,
+                            const isc::datasrc::ZoneFinder::FindOptions options
+                            = isc::datasrc::ZoneFinder::FIND_DEFAULT);
 
-    /// \brief Look up \c Zone's NS and address records for the NS RDATA
-    /// (domain name) for authoritative answer.
+    /// \brief Look up a zone's NS RRset and their address records for an
+    /// authoritative answer, and add them to the additional section.
     ///
-    /// On returning an authoritative answer, insert the \c Zone's NS into the
+    /// On returning an authoritative answer, insert a zone's NS into the
     /// authority section and AAAA/A RRs of each of the NS RDATA into the
     /// additional section.
     ///
@@ -126,25 +148,29 @@ private:
     /// include AAAA/A RRs under a zone cut in additional section. (BIND 9
     /// excludes under-cut RRs; NSD include them.)
     ///
-    /// \param zone The \c Zone wherein the additional data to the query is to
-    /// be found.
-    void getAuthAdditional(const isc::datasrc::Zone& zone) const;
+    /// \param finder The \c ZoneFinder through which the NS and additional
+    /// data for the query are to be found.
+    void addAuthAdditional(isc::datasrc::ZoneFinder& finder);
 
 public:
     /// Constructor from query parameters.
     ///
     /// This constructor never throws an exception.
     ///
-    /// \param memory_datasrc The memory datasource wherein the answer to the query is
+    /// \param datasrc_client The datasource wherein the answer to the query is
     /// to be found.
     /// \param qname The query name
     /// \param qtype The RR type of the query
     /// \param response The response message to store the answer to the query.
-    Query(const isc::datasrc::MemoryDataSrc& memory_datasrc,
+    /// \param dnssec If the answer should include signatures and NSEC/NSEC3 if
+    ///     possible.
+    Query(const isc::datasrc::DataSourceClient& datasrc_client,
           const isc::dns::Name& qname, const isc::dns::RRType& qtype,
-          isc::dns::Message& response) :
-        memory_datasrc_(memory_datasrc), qname_(qname), qtype_(qtype),
-        response_(response)
+          isc::dns::Message& response, bool dnssec = false) :
+        datasrc_client_(datasrc_client), qname_(qname), qtype_(qtype),
+        response_(response), dnssec_(dnssec),
+        dnssec_opt_(dnssec ?  isc::datasrc::ZoneFinder::FIND_DNSSEC :
+                    isc::datasrc::ZoneFinder::FIND_DEFAULT)
     {}
 
     /// Process the query.
@@ -157,7 +183,7 @@ public:
     /// successful search would result in adding a corresponding RRset to
     /// the answer section of the response.
     ///
-    /// If no matching zone is found in the memory datasource, the RCODE of
+    /// If no matching zone is found in the datasource, the RCODE of
     /// SERVFAIL will be set in the response.
     /// <b>Note:</b> this is different from the error code that BIND 9 returns
     /// by default when it's configured as an authoritative-only server (and
@@ -173,7 +199,7 @@ public:
     /// This might throw BadZone or any of its specific subclasses, but that
     /// shouldn't happen in real-life (as BadZone means wrong data, it should
     /// have been rejected upon loading).
-    void process() const;
+    void process();
 
     /// \short Bad zone data encountered.
     ///
@@ -207,11 +233,24 @@ public:
         {}
     };
 
+    /// An invalid result is given when a valid NSEC is expected
+    ///
+    // This can only happen when the underlying data source implementation or
+    /// the zone is broken.  By throwing an exception we treat such cases
+    /// as SERVFAIL.
+    struct BadNSEC : public BadZone {
+        BadNSEC(const char* file, size_t line, const char* what) :
+            BadZone(file, line, what)
+        {}
+    };
+
 private:
-    const isc::datasrc::MemoryDataSrc& memory_datasrc_;
+    const isc::datasrc::DataSourceClient& datasrc_client_;
     const isc::dns::Name& qname_;
     const isc::dns::RRType& qtype_;
     isc::dns::Message& response_;
+    const bool dnssec_;
+    const isc::datasrc::ZoneFinder::FindOptions dnssec_opt_;
 };
 
 }

src/bin/auth/spec_config.h.pre.in

@@ -1,16 +1,16 @@
-// Copyright (C) 2009  Internet Systems Consortium, Inc. ("ISC")
-//
-// Permission to use, copy, modify, and/or distribute this software for any
-// purpose with or without fee is hereby granted, provided that the above
-// copyright notice and this permission notice appear in all copies.
-//
-// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
-// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
-// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
-// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
-// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
-// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
-// PERFORMANCE OF THIS SOFTWARE.
-
-#define AUTH_SPECFILE_LOCATION "@prefix@/share/@PACKAGE@/auth.spec"
-#define UNIX_SOCKET_FILE "@@LOCALSTATEDIR@@/auth_xfrout_conn"
+// Copyright (C) 2009  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#define AUTH_SPECFILE_LOCATION "@prefix@/share/@PACKAGE@/auth.spec"
+#define UNIX_SOCKET_FILE "@@LOCALSTATEDIR@@/@PACKAGE@/auth_xfrout_conn"

src/bin/auth/statistics.cc

@@ -44,12 +44,15 @@ public:
              const AuthCounters::PerZoneCounterType type);
     bool submitStatistics() const;
     void setStatisticsSession(isc::cc::AbstractSession* statistics_session);
+    void registerStatisticsValidator
+    (AuthCounters::validator_type validator);
     // Currently for testing purpose only
     uint64_t getCounter(const AuthCounters::ServerCounterType type) const;
 private:
     Counter server_counter_;
     CounterDictionary per_zone_counter_;
     isc::cc::AbstractSession* statistics_session_;
+    AuthCounters::validator_type validator_;
 };
 
 AuthCountersImpl::AuthCountersImpl() :
@@ -86,16 +89,25 @@ AuthCountersImpl::submitStatistics() const {
     }
     std::stringstream statistics_string;
     statistics_string << "{\"command\": [\"set\","
-                      <<   "{ \"stats_data\": "
-                      <<     "{ \"auth.queries.udp\": "
+                      <<   "{ \"owner\": \"Auth\","
+                      <<   "  \"data\":"
+                      <<     "{ \"queries.udp\": "
                       <<     server_counter_.get(AuthCounters::SERVER_UDP_QUERY)
-                      <<     ", \"auth.queries.tcp\": "
+                      <<     ", \"queries.tcp\": "
                       <<     server_counter_.get(AuthCounters::SERVER_TCP_QUERY)
                       <<   " }"
                       <<   "}"
                       << "]}";
     isc::data::ConstElementPtr statistics_element =
         isc::data::Element::fromJSON(statistics_string);
+    // validate the statistics data before send
+    if (validator_) {
+        if (!validator_(
+                statistics_element->get("command")->get(1)->get("data"))) {
+            LOG_ERROR(auth_logger, AUTH_INVALID_STATISTICS_DATA);
+            return (false);
+        }
+    }
     try {
         // group_{send,recv}msg() can throw an exception when encountering
         // an error, and group_recvmsg() will throw an exception on timeout.
@@ -124,6 +136,13 @@ AuthCountersImpl::setStatisticsSession
     statistics_session_ = statistics_session;
 }
 
+void
+AuthCountersImpl::registerStatisticsValidator
+    (AuthCounters::validator_type validator)
+{
+    validator_ = validator;
+}
+
 // Currently for testing purpose only
 uint64_t
 AuthCountersImpl::getCounter(const AuthCounters::ServerCounterType type) const {
@@ -156,3 +175,10 @@ uint64_t
 AuthCounters::getCounter(const AuthCounters::ServerCounterType type) const {
     return (impl_->getCounter(type));
 }
+
+void
+AuthCounters::registerStatisticsValidator
+    (AuthCounters::validator_type validator) const
+{
+    return (impl_->registerStatisticsValidator(validator));
+}

src/bin/auth/statistics.h

@@ -83,7 +83,7 @@ public:
     ///
     /// \throw std::out_of_range \a type is unknown.
     ///
-    /// usage: counter.inc(CounterType::COUNTER_UDP_QUERY);
+    /// usage: counter.inc(AuthCounters::SERVER_UDP_QUERY);
     /// 
     void inc(const ServerCounterType type);
 
@@ -137,6 +137,26 @@ public:
     /// \return the value of the counter specified by \a type.
     ///
     uint64_t getCounter(const AuthCounters::ServerCounterType type) const;
+
+    /// \brief A type of validation function for the specification in
+    /// isc::config::ModuleSpec.
+    ///
+    /// This type might be useful for not only statistics
+    /// specificatoin but also for config_data specification and for
+    /// commnad.
+    ///
+    typedef boost::function<bool(const isc::data::ConstElementPtr&)>
+    validator_type;
+
+    /// \brief Register a function type of the statistics validation
+    /// function for AuthCounters.
+    ///
+    /// This method never throws an exception.
+    ///
+    /// \param validator A function type of the validation of
+    /// statistics specification.
+    ///
+    void registerStatisticsValidator(AuthCounters::validator_type validator) const;
 };
 
 #endif // __STATISTICS_H

src/bin/auth/tests/Makefile.am

@@ -37,6 +37,13 @@ run_unittests_SOURCES += query_unittest.cc
 run_unittests_SOURCES += change_user_unittest.cc
 run_unittests_SOURCES += statistics_unittest.cc
 run_unittests_SOURCES += run_unittests.cc
+# This is a temporary workaround for #1206, where the InMemoryClient has been
+# moved to an ldopened library. We could add that library to LDADD, but that
+# is nonportable. When #1207 is done this becomes moot anyway, and the
+# specific workaround is not needed anymore, so we can then remove this
+# line again.
+run_unittests_SOURCES += ${top_srcdir}/src/lib/datasrc/memory_datasrc.cc
+
 
 nodist_run_unittests_SOURCES = ../auth_messages.h ../auth_messages.cc
 
@@ -47,6 +54,7 @@ run_unittests_LDADD += $(SQLITE_LIBS)
 run_unittests_LDADD += $(top_builddir)/src/lib/testutils/libtestutils.la
 run_unittests_LDADD +=  $(top_builddir)/src/lib/datasrc/libdatasrc.la
 run_unittests_LDADD +=  $(top_builddir)/src/lib/dns/libdns++.la
+run_unittests_LDADD += $(top_builddir)/src/lib/util/libutil.la
 run_unittests_LDADD += $(top_builddir)/src/lib/asiodns/libasiodns.la
 run_unittests_LDADD += $(top_builddir)/src/lib/asiolink/libasiolink.la
 run_unittests_LDADD += $(top_builddir)/src/lib/config/libcfgclient.la

src/bin/auth/tests/auth_srv_unittest.cc

@@ -229,7 +229,8 @@ TEST_F(AuthSrvTest, AXFROverUDP) {
 TEST_F(AuthSrvTest, AXFRSuccess) {
     EXPECT_FALSE(xfrout.isConnected());
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::AXFR());
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::AXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     // On success, the AXFR query has been passed to a separate process,
     // so we shouldn't have to respond.
@@ -245,7 +246,8 @@ TEST_F(AuthSrvTest, TSIGSigned) {
     const TSIGKey key("key:c2VjcmV0Cg==:hmac-sha1");
     TSIGContext context(key);
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("version.bind"), RRClass::CH(), RRType::TXT());
+                                       Name("version.bind"), RRClass::CH(),
+                                       RRType::TXT());
     createRequestPacket(request_message, IPPROTO_UDP, &context);
 
     // Run the message through the server
@@ -278,7 +280,8 @@ TEST_F(AuthSrvTest, TSIGSignedBadKey) {
     TSIGKey key("key:c2VjcmV0Cg==:hmac-sha1");
     TSIGContext context(key);
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("version.bind"), RRClass::CH(), RRType::TXT());
+                                       Name("version.bind"), RRClass::CH(),
+                                       RRType::TXT());
     createRequestPacket(request_message, IPPROTO_UDP, &context);
 
     // Process the message, but use a different key there
@@ -309,7 +312,8 @@ TEST_F(AuthSrvTest, TSIGBadSig) {
     TSIGKey key("key:c2VjcmV0Cg==:hmac-sha1");
     TSIGContext context(key);
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("version.bind"), RRClass::CH(), RRType::TXT());
+                                       Name("version.bind"), RRClass::CH(),
+                                       RRType::TXT());
     createRequestPacket(request_message, IPPROTO_UDP, &context);
 
     // Process the message, but use a different key there
@@ -375,7 +379,8 @@ TEST_F(AuthSrvTest, AXFRConnectFail) {
     EXPECT_FALSE(xfrout.isConnected()); // check prerequisite
     xfrout.disableConnect();
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::AXFR());
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::AXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -388,7 +393,8 @@ TEST_F(AuthSrvTest, AXFRSendFail) {
     // first send a valid query, making the connection with the xfr process
     // open.
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::AXFR());
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::AXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(xfrout.isConnected());
@@ -397,7 +403,8 @@ TEST_F(AuthSrvTest, AXFRSendFail) {
     parse_message->clear(Message::PARSE);
     response_obuffer->clear();
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::AXFR());
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::AXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -414,7 +421,66 @@ TEST_F(AuthSrvTest, AXFRDisconnectFail) {
     xfrout.disableSend();
     xfrout.disableDisconnect();
     UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::AXFR());
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::AXFR());
+    createRequestPacket(request_message, IPPROTO_TCP);
+    EXPECT_THROW(server.processMessage(*io_message, parse_message,
+                                       response_obuffer, &dnsserv),
+                 XfroutError);
+    EXPECT_TRUE(xfrout.isConnected());
+    // XXX: we need to re-enable disconnect.  otherwise an exception would be
+    // thrown via the destructor of the server.
+    xfrout.enableDisconnect();
+}
+
+TEST_F(AuthSrvTest, IXFRConnectFail) {
+    EXPECT_FALSE(xfrout.isConnected()); // check prerequisite
+    xfrout.disableConnect();
+    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::IXFR());
+    createRequestPacket(request_message, IPPROTO_TCP);
+    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
+    EXPECT_TRUE(dnsserv.hasAnswer());
+    headerCheck(*parse_message, default_qid, Rcode::SERVFAIL(),
+                opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
+    EXPECT_FALSE(xfrout.isConnected());
+}
+
+TEST_F(AuthSrvTest, IXFRSendFail) {
+    // first send a valid query, making the connection with the xfr process
+    // open.
+    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::IXFR());
+    createRequestPacket(request_message, IPPROTO_TCP);
+    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
+    EXPECT_TRUE(xfrout.isConnected());
+
+    xfrout.disableSend();
+    parse_message->clear(Message::PARSE);
+    response_obuffer->clear();
+    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::IXFR());
+    createRequestPacket(request_message, IPPROTO_TCP);
+    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
+    EXPECT_TRUE(dnsserv.hasAnswer());
+    headerCheck(*parse_message, default_qid, Rcode::SERVFAIL(),
+                opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
+
+    // The connection should have been closed due to the send failure.
+    EXPECT_FALSE(xfrout.isConnected());
+}
+
+TEST_F(AuthSrvTest, IXFRDisconnectFail) {
+    // In our usage disconnect() shouldn't fail.  So we'll see the exception
+    // should it be thrown.
+    xfrout.disableSend();
+    xfrout.disableDisconnect();
+    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
+                                       Name("example.com"), RRClass::IN(),
+                                       RRType::IXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     EXPECT_THROW(server.processMessage(*io_message, parse_message,
                                        response_obuffer, &dnsserv),
@@ -426,8 +492,9 @@ TEST_F(AuthSrvTest, AXFRDisconnectFail) {
 }
 
 TEST_F(AuthSrvTest, notify) {
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -458,8 +525,9 @@ TEST_F(AuthSrvTest, notify) {
 
 TEST_F(AuthSrvTest, notifyForCHClass) {
     // Same as the previous test, but for the CH RRClass.
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::CH(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::CH(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -487,8 +555,9 @@ TEST_F(AuthSrvTest, notifyEmptyQuestion) {
 }
 
 TEST_F(AuthSrvTest, notifyMultiQuestions) {
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     // add one more SOA question
     request_message.addQuestion(Question(Name("example.com"), RRClass::IN(),
                                          RRType::SOA()));
@@ -501,8 +570,9 @@ TEST_F(AuthSrvTest, notifyMultiQuestions) {
 }
 
 TEST_F(AuthSrvTest, notifyNonSOAQuestion) {
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::NS());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::NS());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -513,8 +583,9 @@ TEST_F(AuthSrvTest, notifyNonSOAQuestion) {
 
 TEST_F(AuthSrvTest, notifyWithoutAA) {
     // implicitly leave the AA bit off.  our implementation will accept it.
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
     EXPECT_TRUE(dnsserv.hasAnswer());
@@ -523,8 +594,9 @@ TEST_F(AuthSrvTest, notifyWithoutAA) {
 }
 
 TEST_F(AuthSrvTest, notifyWithErrorRcode) {
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     request_message.setRcode(Rcode::SERVFAIL());
     createRequestPacket(request_message, IPPROTO_UDP);
@@ -537,8 +609,9 @@ TEST_F(AuthSrvTest, notifyWithErrorRcode) {
 TEST_F(AuthSrvTest, notifyWithoutSession) {
     server.setXfrinSession(NULL);
 
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
 
@@ -551,8 +624,9 @@ TEST_F(AuthSrvTest, notifyWithoutSession) {
 TEST_F(AuthSrvTest, notifySendFail) {
     notify_session.disableSend();
 
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
 
@@ -563,8 +637,9 @@ TEST_F(AuthSrvTest, notifySendFail) {
 TEST_F(AuthSrvTest, notifyReceiveFail) {
     notify_session.disableReceive();
 
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -574,8 +649,9 @@ TEST_F(AuthSrvTest, notifyReceiveFail) {
 TEST_F(AuthSrvTest, notifyWithBogusSessionMessage) {
     notify_session.setMessage(Element::fromJSON("{\"foo\": 1}"));
 
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -586,8 +662,9 @@ TEST_F(AuthSrvTest, notifyWithSessionMessageError) {
     notify_session.setMessage(
         Element::fromJSON("{\"result\": [1, \"FAIL\"]}"));
 
-    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(), default_qid,
-                         Name("example.com"), RRClass::IN(), RRType::SOA());
+    UnitTestUtil::createRequestMessage(request_message, Opcode::NOTIFY(),
+                                       default_qid, Name("example.com"),
+                                       RRClass::IN(), RRType::SOA());
     request_message.setHeaderFlag(Message::HEADERFLAG_AA);
     createRequestPacket(request_message, IPPROTO_UDP);
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
@@ -651,17 +728,17 @@ TEST_F(AuthSrvTest, updateConfigFail) {
                 QR_FLAG | AA_FLAG, 1, 1, 1, 0);
 }
 
-TEST_F(AuthSrvTest, updateWithMemoryDataSrc) {
+TEST_F(AuthSrvTest, updateWithInMemoryClient) {
     // Test configuring memory data source.  Detailed test cases are covered
     // in the configuration tests.  We only check the AuthSrv interface here.
 
     // By default memory data source isn't enabled
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
     updateConfig(&server,
                  "{\"datasources\": [{\"type\": \"memory\"}]}", true);
     // after successful configuration, we should have one (with empty zoneset).
-    ASSERT_NE(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
-    EXPECT_EQ(0, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    ASSERT_NE(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
+    EXPECT_EQ(0, server.getInMemoryClient(rrclass)->getZoneCount());
 
     // The memory data source is empty, should return REFUSED rcode.
     createDataFromFile("examplequery_fromWire.wire");
@@ -672,7 +749,7 @@ TEST_F(AuthSrvTest, updateWithMemoryDataSrc) {
                 opcode.getCode(), QR_FLAG, 1, 0, 0, 0);
 }
 
-TEST_F(AuthSrvTest, chQueryWithMemoryDataSrc) {
+TEST_F(AuthSrvTest, chQueryWithInMemoryClient) {
     // Configure memory data source for class IN
     updateConfig(&server, "{\"datasources\": "
                  "[{\"class\": \"IN\", \"type\": \"memory\"}]}", true);
@@ -737,12 +814,28 @@ TEST_F(AuthSrvTest, queryCounterTCPAXFR) {
                          Name("example.com"), RRClass::IN(), RRType::AXFR());
     createRequestPacket(request_message, IPPROTO_TCP);
     // On success, the AXFR query has been passed to a separate process,
-    // so we shouldn't have to respond.
+    // so auth itself shouldn't respond.
     server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
+    EXPECT_FALSE(dnsserv.hasAnswer());
     // After processing TCP AXFR query, the counter should be 1.
     EXPECT_EQ(1, server.getCounter(AuthCounters::SERVER_TCP_QUERY));
 }
 
+// Submit TCP IXFR query and check query counter
+TEST_F(AuthSrvTest, queryCounterTCPIXFR) {
+    // The counter should be initialized to 0.
+    EXPECT_EQ(0, server.getCounter(AuthCounters::COUNTER_TCP_QUERY));
+    UnitTestUtil::createRequestMessage(request_message, opcode, default_qid,
+                         Name("example.com"), RRClass::IN(), RRType::IXFR());
+    createRequestPacket(request_message, IPPROTO_TCP);
+    // On success, the IXFR query has been passed to a separate process,
+    // so auth itself shouldn't respond.
+    server.processMessage(*io_message, parse_message, response_obuffer, &dnsserv);
+    EXPECT_FALSE(dnsserv.hasAnswer());
+    // After processing TCP IXFR query, the counter should be 1.
+    EXPECT_EQ(1, server.getCounter(AuthCounters::COUNTER_TCP_QUERY));
+}
+
 // class for queryCounterUnexpected test
 // getProtocol() returns IPPROTO_IP
 class DummyUnknownSocket : public IOSocket {

src/bin/auth/tests/command_unittest.cc

@@ -60,7 +60,6 @@ protected:
     MockSession statistics_session;
     MockXfroutClient xfrout;
     AuthSrv server;
-    AuthSrv::ConstMemoryDataSrcPtr memory_datasrc;
     ConstElementPtr result;
     int rcode;
 public:
@@ -110,18 +109,18 @@ TEST_F(AuthCommandTest, shutdown) {
 // zones, and checks the zones are correctly loaded.
 void
 zoneChecks(AuthSrv& server) {
-    EXPECT_TRUE(server.getMemoryDataSrc(RRClass::IN()));
-    EXPECT_EQ(Zone::SUCCESS, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test1.example")).zone->
+    EXPECT_TRUE(server.getInMemoryClient(RRClass::IN()));
+    EXPECT_EQ(ZoneFinder::SUCCESS, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test1.example")).zone_finder->
               find(Name("ns.test1.example"), RRType::A()).code);
-    EXPECT_EQ(Zone::NXRRSET, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test1.example")).zone->
+    EXPECT_EQ(ZoneFinder::NXRRSET, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test1.example")).zone_finder->
               find(Name("ns.test1.example"), RRType::AAAA()).code);
-    EXPECT_EQ(Zone::SUCCESS, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test2.example")).zone->
+    EXPECT_EQ(ZoneFinder::SUCCESS, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test2.example")).zone_finder->
               find(Name("ns.test2.example"), RRType::A()).code);
-    EXPECT_EQ(Zone::NXRRSET, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test2.example")).zone->
+    EXPECT_EQ(ZoneFinder::NXRRSET, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test2.example")).zone_finder->
               find(Name("ns.test2.example"), RRType::AAAA()).code);
 }
 
@@ -147,21 +146,21 @@ configureZones(AuthSrv& server) {
 
 void
 newZoneChecks(AuthSrv& server) {
-    EXPECT_TRUE(server.getMemoryDataSrc(RRClass::IN()));
-    EXPECT_EQ(Zone::SUCCESS, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test1.example")).zone->
+    EXPECT_TRUE(server.getInMemoryClient(RRClass::IN()));
+    EXPECT_EQ(ZoneFinder::SUCCESS, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test1.example")).zone_finder->
               find(Name("ns.test1.example"), RRType::A()).code);
     // now test1.example should have ns/AAAA
-    EXPECT_EQ(Zone::SUCCESS, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test1.example")).zone->
+    EXPECT_EQ(ZoneFinder::SUCCESS, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test1.example")).zone_finder->
               find(Name("ns.test1.example"), RRType::AAAA()).code);
 
     // test2.example shouldn't change
-    EXPECT_EQ(Zone::SUCCESS, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test2.example")).zone->
+    EXPECT_EQ(ZoneFinder::SUCCESS, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test2.example")).zone_finder->
               find(Name("ns.test2.example"), RRType::A()).code);
-    EXPECT_EQ(Zone::NXRRSET, server.getMemoryDataSrc(RRClass::IN())->
-              findZone(Name("ns.test2.example")).zone->
+    EXPECT_EQ(ZoneFinder::NXRRSET, server.getInMemoryClient(RRClass::IN())->
+              findZone(Name("ns.test2.example")).zone_finder->
               find(Name("ns.test2.example"), RRType::AAAA()).code);
 }
 

src/bin/auth/tests/config_unittest.cc

@@ -57,12 +57,12 @@ protected:
 
 TEST_F(AuthConfigTest, datasourceConfig) {
     // By default, we don't have any in-memory data source.
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
     configureAuthServer(server, Element::fromJSON(
                             "{\"datasources\": [{\"type\": \"memory\"}]}"));
     // after successful configuration, we should have one (with empty zoneset).
-    ASSERT_NE(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
-    EXPECT_EQ(0, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    ASSERT_NE(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
+    EXPECT_EQ(0, server.getInMemoryClient(rrclass)->getZoneCount());
 }
 
 TEST_F(AuthConfigTest, databaseConfig) {
@@ -82,7 +82,7 @@ TEST_F(AuthConfigTest, versionConfig) {
 }
 
 TEST_F(AuthConfigTest, exceptionGuarantee) {
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
     // This configuration contains an invalid item, which will trigger
     // an exception.
     EXPECT_THROW(configureAuthServer(
@@ -92,7 +92,7 @@ TEST_F(AuthConfigTest, exceptionGuarantee) {
                          " \"no_such_config_var\": 1}")),
                  AuthConfigError);
     // The server state shouldn't change
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
 }
 
 TEST_F(AuthConfigTest, exceptionConversion) {
@@ -154,22 +154,22 @@ protected:
 TEST_F(MemoryDatasrcConfigTest, addZeroDataSrc) {
     parser->build(Element::fromJSON("[]"));
     parser->commit();
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
 }
 
 TEST_F(MemoryDatasrcConfigTest, addEmpty) {
     // By default, we don't have any in-memory data source.
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
     parser->build(Element::fromJSON("[{\"type\": \"memory\"}]"));
     parser->commit();
-    EXPECT_EQ(0, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(0, server.getInMemoryClient(rrclass)->getZoneCount());
 }
 
 TEST_F(MemoryDatasrcConfigTest, addZeroZone) {
     parser->build(Element::fromJSON("[{\"type\": \"memory\","
                                     "  \"zones\": []}]"));
     parser->commit();
-    EXPECT_EQ(0, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(0, server.getInMemoryClient(rrclass)->getZoneCount());
 }
 
 TEST_F(MemoryDatasrcConfigTest, addOneZone) {
@@ -179,10 +179,10 @@ TEST_F(MemoryDatasrcConfigTest, addOneZone) {
                       "               \"file\": \"" TEST_DATA_DIR
                       "/example.zone\"}]}]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(1, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(1, server.getInMemoryClient(rrclass)->getZoneCount());
     // Check it actually loaded something
-    EXPECT_EQ(Zone::SUCCESS, server.getMemoryDataSrc(rrclass)->findZone(
-        Name("ns.example.com.")).zone->find(Name("ns.example.com."),
+    EXPECT_EQ(ZoneFinder::SUCCESS, server.getInMemoryClient(rrclass)->findZone(
+        Name("ns.example.com.")).zone_finder->find(Name("ns.example.com."),
         RRType::A()).code);
 }
 
@@ -199,7 +199,7 @@ TEST_F(MemoryDatasrcConfigTest, addMultiZones) {
                       "               \"file\": \"" TEST_DATA_DIR
                       "/example.net.zone\"}]}]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(3, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(3, server.getInMemoryClient(rrclass)->getZoneCount());
 }
 
 TEST_F(MemoryDatasrcConfigTest, replace) {
@@ -209,9 +209,9 @@ TEST_F(MemoryDatasrcConfigTest, replace) {
                       "               \"file\": \"" TEST_DATA_DIR
                       "/example.zone\"}]}]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(1, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(1, server.getInMemoryClient(rrclass)->getZoneCount());
     EXPECT_EQ(isc::datasrc::result::SUCCESS,
-              server.getMemoryDataSrc(rrclass)->findZone(
+              server.getInMemoryClient(rrclass)->findZone(
                   Name("example.com")).code);
 
     // create a new parser, and install a new set of configuration.  It
@@ -227,9 +227,9 @@ TEST_F(MemoryDatasrcConfigTest, replace) {
                       "               \"file\": \"" TEST_DATA_DIR
                       "/example.net.zone\"}]}]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(2, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(2, server.getInMemoryClient(rrclass)->getZoneCount());
     EXPECT_EQ(isc::datasrc::result::NOTFOUND,
-              server.getMemoryDataSrc(rrclass)->findZone(
+              server.getInMemoryClient(rrclass)->findZone(
                   Name("example.com")).code);
 }
 
@@ -241,9 +241,9 @@ TEST_F(MemoryDatasrcConfigTest, exception) {
                       "               \"file\": \"" TEST_DATA_DIR
                       "/example.zone\"}]}]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(1, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(1, server.getInMemoryClient(rrclass)->getZoneCount());
     EXPECT_EQ(isc::datasrc::result::SUCCESS,
-              server.getMemoryDataSrc(rrclass)->findZone(
+              server.getInMemoryClient(rrclass)->findZone(
                   Name("example.com")).code);
 
     // create a new parser, and try to load something. It will throw,
@@ -262,9 +262,9 @@ TEST_F(MemoryDatasrcConfigTest, exception) {
     // commit it
 
     // The original should be untouched
-    EXPECT_EQ(1, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(1, server.getInMemoryClient(rrclass)->getZoneCount());
     EXPECT_EQ(isc::datasrc::result::SUCCESS,
-              server.getMemoryDataSrc(rrclass)->findZone(
+              server.getInMemoryClient(rrclass)->findZone(
                   Name("example.com")).code);
 }
 
@@ -275,13 +275,13 @@ TEST_F(MemoryDatasrcConfigTest, remove) {
                       "               \"file\": \"" TEST_DATA_DIR
                       "/example.zone\"}]}]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(1, server.getMemoryDataSrc(rrclass)->getZoneCount());
+    EXPECT_EQ(1, server.getInMemoryClient(rrclass)->getZoneCount());
 
     delete parser;
     parser = createAuthConfigParser(server, "datasources"); 
     EXPECT_NO_THROW(parser->build(Element::fromJSON("[]")));
     EXPECT_NO_THROW(parser->commit());
-    EXPECT_EQ(AuthSrv::MemoryDataSrcPtr(), server.getMemoryDataSrc(rrclass));
+    EXPECT_EQ(AuthSrv::InMemoryClientPtr(), server.getInMemoryClient(rrclass));
 }
 
 TEST_F(MemoryDatasrcConfigTest, adDuplicateZones) {

src/bin/auth/tests/statistics_unittest.cc

@@ -16,6 +16,8 @@
 
 #include <gtest/gtest.h>
 
+#include <boost/bind.hpp>
+
 #include <cc/data.h>
 #include <cc/session.h>
 
@@ -76,6 +78,13 @@ protected:
     }
     MockSession statistics_session_;
     AuthCounters counters;
+    // no need to be inherited from the original class here.
+    class MockModuleSpec {
+    public:
+        bool validateStatistics(ConstElementPtr, const bool valid) const
+            { return (valid); }
+    };
+    MockModuleSpec module_spec_;
 };
 
 void
@@ -180,7 +189,7 @@ TEST_F(AuthCountersTest, submitStatisticsWithException) {
     statistics_session_.setThrowSessionTimeout(false);
 }
 
-TEST_F(AuthCountersTest, submitStatistics) {
+TEST_F(AuthCountersTest, submitStatisticsWithoutValidator) {
     // Submit statistics data.
     // Validate if it submits correct data.
 
@@ -200,12 +209,69 @@ TEST_F(AuthCountersTest, submitStatistics) {
     // Command is "set".
     EXPECT_EQ("set", statistics_session_.sent_msg->get("command")
                          ->get(0)->stringValue());
+    EXPECT_EQ("Auth", statistics_session_.sent_msg->get("command")
+                         ->get(1)->get("owner")->stringValue());
     ConstElementPtr statistics_data = statistics_session_.sent_msg
                                           ->get("command")->get(1)
-                                          ->get("stats_data");
+                                          ->get("data");
     // UDP query counter is 2 and TCP query counter is 1.
-    EXPECT_EQ(2, statistics_data->get("auth.queries.udp")->intValue());
-    EXPECT_EQ(1, statistics_data->get("auth.queries.tcp")->intValue());
+    EXPECT_EQ(2, statistics_data->get("queries.udp")->intValue());
+    EXPECT_EQ(1, statistics_data->get("queries.tcp")->intValue());
 }
 
+TEST_F(AuthCountersTest, submitStatisticsWithValidator) {
+
+    //a validator for the unittest
+    AuthCounters::validator_type validator;
+    ConstElementPtr el;
+
+    // Submit statistics data with correct statistics validator.
+    validator = boost::bind(
+        &AuthCountersTest::MockModuleSpec::validateStatistics,
+        &module_spec_, _1, true);
+
+    EXPECT_TRUE(validator(el));
+
+    // register validator to AuthCounters
+    counters.registerStatisticsValidator(validator);
+
+    // Counters should be initialized to 0.
+    EXPECT_EQ(0, counters.getCounter(AuthCounters::SERVER_UDP_QUERY));
+    EXPECT_EQ(0, counters.getCounter(AuthCounters::SERVER_TCP_QUERY));
+
+    // UDP query counter is set to 2.
+    counters.inc(AuthCounters::SERVER_UDP_QUERY);
+    counters.inc(AuthCounters::SERVER_UDP_QUERY);
+    // TCP query counter is set to 1.
+    counters.inc(AuthCounters::SERVER_TCP_QUERY);
+
+    // checks the value returned by submitStatistics
+    EXPECT_TRUE(counters.submitStatistics());
+
+    // Destination is "Stats".
+    EXPECT_EQ("Stats", statistics_session_.msg_destination);
+    // Command is "set".
+    EXPECT_EQ("set", statistics_session_.sent_msg->get("command")
+                         ->get(0)->stringValue());
+    EXPECT_EQ("Auth", statistics_session_.sent_msg->get("command")
+                         ->get(1)->get("owner")->stringValue());
+    ConstElementPtr statistics_data = statistics_session_.sent_msg
+                                          ->get("command")->get(1)
+                                          ->get("data");
+    // UDP query counter is 2 and TCP query counter is 1.
+    EXPECT_EQ(2, statistics_data->get("queries.udp")->intValue());
+    EXPECT_EQ(1, statistics_data->get("queries.tcp")->intValue());
+
+    // Submit statistics data with incorrect statistics validator.
+    validator = boost::bind(
+        &AuthCountersTest::MockModuleSpec::validateStatistics,
+        &module_spec_, _1, false);
+
+    EXPECT_FALSE(validator(el));
+
+    counters.registerStatisticsValidator(validator);
+
+    // checks the value returned by submitStatistics
+    EXPECT_FALSE(counters.submitStatistics());
+}
 }

src/bin/auth/tests/testdata/Makefile.am

@@ -23,4 +23,4 @@ EXTRA_DIST += example.com
 EXTRA_DIST += example.sqlite3
 
 .spec.wire:
-	$(abs_top_builddir)/src/lib/dns/tests/testdata/gen-wiredata.py -o $@ $<
+	$(PYTHON) $(top_builddir)/src/lib/util/python/gen_wiredata.py -o $@ $<

src/bin/bind10/Makefile.am

@@ -1,10 +1,16 @@
 SUBDIRS = . tests
 
 sbin_SCRIPTS = bind10
-CLEANFILES = bind10 bind10.pyc bind10_messages.py bind10_messages.pyc
+CLEANFILES = bind10 bind10_src.pyc
+CLEANFILES += $(PYTHON_LOGMSGPKG_DIR)/work/bind10_messages.py
+CLEANFILES += $(PYTHON_LOGMSGPKG_DIR)/work/bind10_messages.pyc
 
 pkglibexecdir = $(libexecdir)/@PACKAGE@
-pyexec_DATA = bind10_messages.py
+
+nodist_pylogmessage_PYTHON = $(PYTHON_LOGMSGPKG_DIR)/work/bind10_messages.py
+pylogmessagedir = $(pyexecdir)/isc/log_messages/
+
+noinst_SCRIPTS = run_bind10.sh
 
 bind10dir = $(pkgdatadir)
 bind10_DATA = bob.spec
@@ -20,13 +26,15 @@ bind10.8: bind10.xml
 
 endif
 
-bind10_messages.py: bind10_messages.mes
-	$(top_builddir)/src/lib/log/compiler/message -p $(top_srcdir)/src/bin/bind10/bind10_messages.mes
+$(PYTHON_LOGMSGPKG_DIR)/work/bind10_messages.py : bind10_messages.mes
+	$(top_builddir)/src/lib/log/compiler/message \
+	-d $(PYTHON_LOGMSGPKG_DIR)/work -p $(srcdir)/bind10_messages.mes
 
 # this is done here since configure.ac AC_OUTPUT doesn't expand exec_prefix
-bind10: bind10.py
+bind10: bind10_src.py $(PYTHON_LOGMSGPKG_DIR)/work/bind10_messages.py
 	$(SED) -e "s|@@PYTHONPATH@@|@pyexecdir@|" \
-	       -e "s|@@LIBEXECDIR@@|$(pkglibexecdir)|" bind10.py >$@
+	       -e "s|@@LIBDIR@@|$(libdir)|" \
+	       -e "s|@@LIBEXECDIR@@|$(pkglibexecdir)|" bind10_src.py >$@
 	chmod a+x $@
 
 pytest:

src/bin/bind10/TODO

@@ -1,19 +1,13 @@
 - Read msgq configuration from configuration manager (Trac #213)
   https://bind10.isc.org/ticket/213
 - Provide more administrator options:
-  - Get process list
   - Get information on a process (returns list of times started & stopped, 
     plus current information such as PID)
-  - Add a component (not necessary for parking lot, but...)
   - Stop a component
   - Force-stop a component
 - Mechanism to wait for child to start before continuing
-- Way to ask a child to die politely 
-- Start statistics daemon
-- Statistics interaction (?)
 - Use .spec file to define comands
 - Rename "c-channel" stuff to msgq for clarity
-- Use logger
 - Reply to shutdown message?
 - Some sort of group creation so termination signals can be sent to
   children of children processes (if any)

src/bin/bind10/bind10.xml

@@ -2,7 +2,7 @@
                "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
 	       [<!ENTITY mdash "&#8212;">]>
 <!--
- - Copyright (C) 2010  Internet Systems Consortium, Inc. ("ISC")
+ - Copyright (C) 2010-2011  Internet Systems Consortium, Inc. ("ISC")
  -
  - Permission to use, copy, modify, and/or distribute this software for any
  - purpose with or without fee is hereby granted, provided that the above
@@ -20,7 +20,7 @@
 <refentry>
 
   <refentryinfo>
-    <date>March 31, 2011</date>
+    <date>November 23, 2011</date>
   </refentryinfo>
 
   <refmeta>
@@ -50,7 +50,7 @@
       <arg><option>-p <replaceable>data_path</replaceable></option></arg>
       <arg><option>-u <replaceable>user</replaceable></option></arg>
       <arg><option>-v</option></arg>
-      <arg><option>--brittle</option></arg>
+      <arg><option>-w <replaceable>wait_time</replaceable></option></arg>
       <arg><option>--cmdctl-port</option> <replaceable>port</replaceable></arg>
       <arg><option>--config-file</option> <replaceable>config-filename</replaceable></arg>
       <arg><option>--data-path</option> <replaceable>directory</replaceable></arg>
@@ -60,6 +60,7 @@
       <arg><option>--pretty-name <replaceable>name</replaceable></option></arg>
       <arg><option>--user <replaceable>user</replaceable></option></arg>
       <arg><option>--verbose</option></arg>
+      <arg><option>--wait <replaceable>wait_time</replaceable></option></arg>
     </cmdsynopsis>
   </refsynopsisdiv>
 
@@ -90,20 +91,6 @@
 
       <varlistentry>
         <term>
-          <option>--brittle</option>
-        </term>
-        <listitem>
-          <para>
-	    Shutdown if any of the child processes of
-	    <command>bind10</command> exit.  This is intended to
-	    help developers debug the server, and should not be
-	    used in production.
-          </para>
-        </listitem>
-      </varlistentry>
-
-      <varlistentry>
-        <term>
           <option>-c</option> <replaceable>config-filename</replaceable>,
           <option>--config-file</option> <replaceable>config-filename</replaceable>
         </term>
@@ -211,12 +198,246 @@ The default is the basename of ARG 0.
         </listitem>
       </varlistentry>
 
+      <varlistentry>
+        <term><option>-w</option> <replaceable>wait_time</replaceable>, <option>--wait</option> <replaceable>wait_time</replaceable></term>
+        <listitem>
+          <para>Sets the amount of time that BIND 10 will wait for
+          the configuration manager (a key component of BIND 10) to
+          initialize itself before abandoning the start up and
+          terminating with an error.  The wait_time is specified in
+          seconds and has a default value of 10.
+          </para>
+        </listitem>
+      </varlistentry>
+
     </variablelist>
   </refsect1>
 
 <!--
 TODO: configuration section
 -->
+
+  <refsect1>
+    <title>CONFIGURATION AND COMMANDS</title>
+
+    <para>
+      The configuration provides settings for components for
+      <command>bind10</command> to manage under
+      <varname>/Boss/components/</varname>.
+      The default elements are:
+    </para>
+
+    <itemizedlist>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-auth</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-cmdctl</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/setuid</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-stats</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-stats-httpd</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-xfrin</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-xfrout</varname> </para>
+      </listitem>
+
+      <listitem>
+        <para> <varname>/Boss/components/b10-zonemgr</varname> </para>
+      </listitem>
+
+    </itemizedlist>
+
+    <para>
+      (Note that the startup of <command>b10-sockcreator</command>,
+      <command>b10-cfgmgr</command>, and <command>b10-msgq</command>
+      is not configurable. It is hardcoded and <command>bind10</command>
+      will not run without them.)
+    </para>
+
+    <para>
+      These named sets (listed above) contain the following settings:
+    </para>
+
+    <variablelist>
+
+      <varlistentry>
+        <term><varname>address</varname></term>
+        <listitem>
+	  <para>The name used for communicating to it on the message
+	  bus.</para>
+<!-- NOTE: vorner said:
+These can be null, because the components are special ones, and
+the special class there already knows the address. It is (I hope)
+explained in the guide. I'd like to get rid of the special components
+sometime and I'd like it to teach to guess the address.
+-->
+        </listitem>
+      </varlistentry>
+
+      <varlistentry>
+        <term><varname>kind</varname></term>
+        <listitem>
+          <para>
+            This defines how required a component is.
+            The possible settings for <varname>kind</varname> are:
+            <varname>core</varname> (system won't start if it won't
+            start and <command>bind10</command> will shutdown if
+            a <quote>core</quote> component crashes),
+            <varname>dispensable</varname> (<command>bind10</command>
+            will restart failing component),
+            and
+	    <varname>needed</varname> (<command>bind10</command>
+	    will shutdown if component won't initially start, but
+	    if crashes later, it will attempt to restart).
+            This setting is required.
+<!-- TODO: formatting -->
+          </para>
+        </listitem>
+      </varlistentry>
+
+<!--
+TODO: currently not used
+      <varlistentry>
+        <term> <varname>params</varname> </term>
+        <listitem>
+          <para>
+list
+</para>
+        </listitem>
+      </varlistentry>
+-->
+
+      <varlistentry>
+        <term> <varname>priority</varname> </term>
+        <listitem>
+          <para>This is an integer. <command>bind10</command>
+            will start the components with largest priority numbers first.
+          </para>
+        </listitem>
+      </varlistentry>
+
+      <varlistentry>
+          <term> <varname>process</varname> </term>
+        <listitem>
+          <para>This is the filename of the executable to be started.
+            If not defined, then <command>bind10</command> will
+            use the component name instead.
+          </para>
+        </listitem>
+      </varlistentry>
+
+      <varlistentry>
+          <term> <varname>special</varname> </term>
+        <listitem>
+          <para>
+            This defines if the component is started a special
+            way.
+<!--
+TODO: document this ... but maybe some of these will be removed
+once we get rid of some using switches for components?
+
+auth
+cfgmgr
+cmdctl
+msgq
+resolver
+setuid
+sockcreator
+xfrin
+-->
+
+</para>
+        </listitem>
+      </varlistentry>
+
+    </variablelist>
+
+<!-- TODO: formating -->
+    <para>
+      The <varname>Boss</varname> configuration commands are:
+    </para>
+<!-- TODO: let's just let bind10 be known as bind10 and not Boss -->
+
+    <para>
+      <command>getstats</command> tells <command>bind10</command>
+      to send its statistics data to the <command>b10-stats</command>
+      daemon.
+      This is an internal command and not exposed to the administrator.
+<!-- not defined in spec -->
+<!-- TODO: explain difference with sendstat -->
+    </para>
+
+    <para>
+      <command>ping</command> is used to check the connection with the
+      <command>bind10</command> daemon.
+      It returns the text <quote>pong</quote>.
+    </para>
+
+    <para>
+      <command>sendstats</command> tells <command>bind10</command>
+      to send its statistics data to the <command>b10-stats</command>
+      daemon immediately.
+<!-- TODO: compare with internal command getstats? -->
+    </para>
+
+    <para>
+      <command>show_processes</command> lists the current processes
+      managed by <command>bind10</command>.
+      The output is an array in JSON format containing the process
+      ID and the name for each.
+<!-- TODO: what is name? -->
+<!-- TODO: change to JSON object format? -->
+<!-- TODO: ticket #1406 -->
+    </para>
+
+    <para>
+      <command>shutdown</command> tells <command>bind10</command>
+      to shutdown the BIND 10 servers.
+      It will tell each process it manages to shutdown and, when
+      complete, <command>bind10</command> will exit.
+    </para>
+
+  </refsect1>
+
+  <refsect1>
+    <title>STATISTICS DATA</title>
+
+    <para>
+      The statistics data collected by the <command>b10-stats</command>
+      daemon include:
+    </para>
+
+    <variablelist>
+
+      <varlistentry>
+        <term>bind10.boot_time</term>
+        <listitem><para>
+          The date and time that the <command>bind10</command>
+          process started.
+          This is represented in ISO 8601 format.
+        </para></listitem>
+      </varlistentry>
+
+    </variablelist>
+
+  </refsect1>
+
 <!--
   <refsect1>
     <title>FILES</title>

src/bin/bind10/bind10_messages.mes

@@ -20,54 +20,100 @@ The boss process is starting up and will now check if the message bus
 daemon is already running. If so, it will not be able to start, as it
 needs a dedicated message bus.
 
-% BIND10_CONFIGURATION_START_AUTH start authoritative server: %1
-This message shows whether or not the authoritative server should be
-started according to the configuration.
-
-% BIND10_CONFIGURATION_START_RESOLVER start resolver: %1
-This message shows whether or not the resolver should be
-started according to the configuration.
+% BIND10_INVALID_STATISTICS_DATA invalid specification of statistics data specified
+An error was encountered when the boss module specified
+statistics data which is invalid for the boss specification file.
+
+% BIND10_COMPONENT_FAILED component %1 (pid %2) failed with %3 exit status
+The process terminated, but the bind10 boss didn't expect it to, which means
+it must have failed.
+
+% BIND10_COMPONENT_RESTART component %1 is about to restart
+The named component failed previously and we will try to restart it to provide
+as flawless service as possible, but it should be investigated what happened,
+as it could happen again.
+
+% BIND10_COMPONENT_START component %1 is starting
+The named component is about to be started by the boss process.
+
+% BIND10_COMPONENT_START_EXCEPTION component %1 failed to start: %2
+An exception (mentioned in the message) happened during the startup of the
+named component. The componet is not considered started and further actions
+will be taken about it.
+
+% BIND10_COMPONENT_STOP component %1 is being stopped
+A component is about to be asked to stop willingly by the boss.
+
+% BIND10_COMPONENT_UNSATISFIED component %1 is required to run and failed
+A component failed for some reason (see previous messages). It is either a core
+component or needed component that was just started. In any case, the system
+can't continue without it and will terminate.
+
+% BIND10_CONFIGURATOR_BUILD building plan '%1' -> '%2'
+A debug message. This indicates that the configurator is building a plan
+how to change configuration from the older one to newer one. This does no
+real work yet, it just does the planning what needs to be done.
+
+% BIND10_CONFIGURATOR_PLAN_INTERRUPTED configurator plan interrupted, only %1 of %2 done
+There was an exception during some planned task. The plan will not continue and
+only some tasks of the plan were completed. The rest is aborted. The exception
+will be propagated.
+
+% BIND10_CONFIGURATOR_RECONFIGURE reconfiguring running components
+A different configuration of which components should be running is being
+installed. All components that are no longer needed will be stopped and
+newly introduced ones started. This happens at startup, when the configuration
+is read the first time, or when an operator changes configuration of the boss.
+
+% BIND10_CONFIGURATOR_RUN running plan of %1 tasks
+A debug message. The configurator is about to execute a plan of actions it
+computed previously.
+
+% BIND10_CONFIGURATOR_START bind10 component configurator is starting up
+The part that cares about starting and stopping the right component from the
+boss process is starting up. This happens only once at the startup of the
+boss process. It will start the basic set of processes now (the ones boss
+needs to read the configuration), the rest will be started after the
+configuration is known.
+
+% BIND10_CONFIGURATOR_STOP bind10 component configurator is shutting down
+The part that cares about starting and stopping processes in the boss is
+shutting down. All started components will be shut down now (more precisely,
+asked to terminate by their own, if they fail to comply, other parts of
+the boss process will try to force them).
+
+% BIND10_CONFIGURATOR_TASK performing task %1 on %2
+A debug message. The configurator is about to perform one task of the plan it
+is currently executing on the named component.
 
 % BIND10_INVALID_USER invalid user: %1
 The boss process was started with the -u option, to drop root privileges
 and continue running as the specified user, but the user is unknown.
 
+% BIND10_KILLING_ALL_PROCESSES killing all started processes
+The boss module was not able to start every process it needed to start
+during startup, and will now kill the processes that did get started.
+
 % BIND10_KILL_PROCESS killing process %1
 The boss module is sending a kill signal to process with the given name,
 as part of the process of killing all started processes during a failed
 startup, as described for BIND10_KILLING_ALL_PROCESSES
 
-% BIND10_KILLING_ALL_PROCESSES killing all started processes
-The boss module was not able to start every process it needed to start
-during startup, and will now kill the processes that did get started.
-
 % BIND10_MSGQ_ALREADY_RUNNING msgq daemon already running, cannot start
 There already appears to be a message bus daemon running. Either an
 old process was not shut down correctly, and needs to be killed, or
 another instance of BIND10, with the same msgq domain socket, is
 running, which needs to be stopped.
 
-% BIND10_MSGQ_DAEMON_ENDED b10-msgq process died, shutting down
-The message bus daemon has died. This is a fatal error, since it may
-leave the system in an inconsistent state. BIND10 will now shut down.
-
 % BIND10_MSGQ_DISAPPEARED msgq channel disappeared
 While listening on the message bus channel for messages, it suddenly
 disappeared. The msgq daemon may have died. This might lead to an
 inconsistent state of the system, and BIND 10 will now shut down.
 
-% BIND10_PROCESS_ENDED_NO_EXIT_STATUS process %1 (PID %2) died: exit status not available
-The given process ended unexpectedly, but no exit status is
-available. See BIND10_PROCESS_ENDED_WITH_EXIT_STATUS for a longer
-description.
-
-% BIND10_PROCESS_ENDED_WITH_EXIT_STATUS process %1 (PID %2) terminated, exit status = %3
-The given process ended unexpectedly with the given exit status.
-Depending on which module it was, it may simply be restarted, or it
-may be a problem that will cause the boss module to shut down too.
-The latter happens if it was the message bus daemon, which, if it has
-died suddenly, may leave the system in an inconsistent state. BIND10
-will also shut down now if it has been run with --brittle.
+% BIND10_PROCESS_ENDED process %2 of %1 ended with status %3
+This indicates a process started previously terminated. The process id
+and component owning the process are indicated, as well as the exit code.
+This doesn't distinguish if the process was supposed to terminate or not.
 
 % BIND10_READING_BOSS_CONFIGURATION reading boss configuration
 The boss process is starting up, and will now process the initial
@@ -103,6 +149,9 @@ The boss module is sending a SIGKILL signal to the given process.
 % BIND10_SEND_SIGTERM sending SIGTERM to %1 (PID %2)
 The boss module is sending a SIGTERM signal to the given process.
 
+% BIND10_SETUID setting UID to %1
+The boss switches the user it runs as to the given UID.
+
 % BIND10_SHUTDOWN stopping the server
 The boss process received a command or signal telling it to shut down.
 It will send a shutdown command to each process. The processes that do
@@ -113,12 +162,48 @@ it shall send SIGKILL signals to the processes still alive.
 All child processes have been stopped, and the boss process will now
 stop itself.
 
-% BIND10_START_AS_NON_ROOT starting %1 as a user, not root. This might fail.
-The given module is being started or restarted without root privileges.
-If the module needs these privileges, it may have problems starting.
-Note that this issue should be resolved by the pending 'socket-creator'
-process; once that has been implemented, modules should not need root
-privileges anymore. See tickets #800 and #801 for more information.
+% BIND10_SOCKCREATOR_BAD_CAUSE unknown error cause from socket creator: %1
+The socket creator reported an error when creating a socket. But the function
+which failed is unknown (not one of 'S' for socket or 'B' for bind).
+
+% BIND10_SOCKCREATOR_BAD_RESPONSE unknown response for socket request: %1
+The boss requested a socket from the creator, but the answer is unknown. This
+looks like a programmer error.
+
+% BIND10_SOCKCREATOR_EOF eof while expecting data from socket creator
+There should be more data from the socket creator, but it closed the socket.
+It probably crashed.
+
+% BIND10_SOCKCREATOR_INIT initializing socket creator parser
+The boss module initializes routines for parsing the socket creator
+protocol.
+
+% BIND10_SOCKCREATOR_KILL killing the socket creator
+The socket creator is being terminated the aggressive way, by sending it
+sigkill. This should not happen usually.
+
+% BIND10_SOCKCREATOR_TERMINATE terminating socket creator
+The boss module sends a request to terminate to the socket creator.
+
+% BIND10_SOCKCREATOR_TRANSPORT_ERROR transport error when talking to the socket creator: %1
+Either sending or receiving data from the socket creator failed with the given
+error. The creator probably crashed or some serious OS-level problem happened,
+as the communication happens only on local host.
+
+% BIND10_SOCKET_CREATED successfully created socket %1
+The socket creator successfully created and sent a requested socket, it has
+the given file number.
+
+% BIND10_SOCKET_ERROR error on %1 call in the creator: %2/%3
+The socket creator failed to create the requested socket. It failed on the
+indicated OS API function with given error.
+
+% BIND10_SOCKET_GET requesting socket [%1]:%2 of type %3 from the creator
+The boss forwards a request for a socket to the socket creator.
+
+% BIND10_STARTED_CC started configuration/command session
+Debug message given when BIND 10 has successfull started the object that
+handles configuration and commands.
 
 % BIND10_STARTED_PROCESS started %1
 The given process has successfully been started.
@@ -129,6 +214,10 @@ The given process has successfully been started, and has the given PID.
 % BIND10_STARTING starting BIND10: %1
 Informational message on startup that shows the full version.
 
+% BIND10_STARTING_CC starting configuration/command session
+Informational message given when BIND 10 is starting the session object
+that handles configuration and commands.
+
 % BIND10_STARTING_PROCESS starting process %1
 The boss module is starting the given process.
 
@@ -147,6 +236,32 @@ All modules have been successfully started, and BIND 10 is now running.
 There was a fatal error when BIND10 was trying to start. The error is
 shown, and BIND10 will now shut down.
 
+% BIND10_STARTUP_UNEXPECTED_MESSAGE unrecognised startup message %1
+During the startup process, a number of messages are exchanged between the
+Boss process and the processes it starts.  This error is output when a
+message received by the Boss process is recognised as being of the
+correct format but is unexpected.  It may be that processes are starting
+of sequence.
+
+% BIND10_STARTUP_UNRECOGNISED_MESSAGE unrecognised startup message %1
+During the startup process, a number of messages are exchanged between the
+Boss process and the processes it starts.  This error is output when a
+message received by the Boss process is not recognised.
+
+% BIND10_START_AS_NON_ROOT_AUTH starting b10-auth as a user, not root. This might fail.
+The authoritative server is being started or restarted without root privileges.
+If the module needs these privileges, it may have problems starting.
+Note that this issue should be resolved by the pending 'socket-creator'
+process; once that has been implemented, modules should not need root
+privileges anymore. See tickets #800 and #801 for more information.
+
+% BIND10_START_AS_NON_ROOT_RESOLVER starting b10-resolver as a user, not root. This might fail.
+The resolver is being started or restarted without root privileges.
+If the module needs these privileges, it may have problems starting.
+Note that this issue should be resolved by the pending 'socket-creator'
+process; once that has been implemented, modules should not need root
+privileges anymore. See tickets #800 and #801 for more information.
+
 % BIND10_STOP_PROCESS asking %1 to shut down
 The boss module is sending a shutdown command to the given module over
 the message channel.
@@ -155,3 +270,13 @@ the message channel.
 An unknown child process has exited. The PID is printed, but no further
 action will be taken by the boss process.
 
+% BIND10_WAIT_CFGMGR waiting for configuration manager process to initialize
+The configuration manager process is so critical to operation of BIND 10
+that after starting it, the Boss module will wait for it to initialize
+itself before continuing.  This debug message is produced during the
+wait and may be output zero or more times depending on how long it takes
+the configuration manager to start up.  The total length of time Boss
+will wait for the configuration manager before reporting an error is
+set with the command line --wait switch, which has a default value of
+ten seconds.
+

src/bin/bind10/bind10.py.in

@@ -44,10 +44,12 @@ import os
 # installed on the system
 if "B10_FROM_SOURCE" in os.environ:
     SPECFILE_LOCATION = os.environ["B10_FROM_SOURCE"] + "/src/bin/bind10/bob.spec"
+    ADD_LIBEXEC_PATH = False
 else:
     PREFIX = "@prefix@"
     DATAROOTDIR = "@datarootdir@"
     SPECFILE_LOCATION = "@datadir@/@PACKAGE@/bob.spec".replace("${datarootdir}", DATAROOTDIR).replace("${prefix}", PREFIX)
+    ADD_LIBEXEC_PATH = True
     
 import subprocess
 import signal
@@ -61,20 +63,23 @@ from optparse import OptionParser, OptionValueError
 import io
 import pwd
 import posix
+import copy
 
 import isc.cc
 import isc.util.process
 import isc.net.parse
 import isc.log
-from bind10_messages import *
+from isc.log_messages.bind10_messages import *
+import isc.bind10.component
+import isc.bind10.special_component
 
 isc.log.init("b10-boss")
 logger = isc.log.Logger("boss")
 
 # Pending system-wide debug level definitions, the ones we
 # use here are hardcoded for now
-DBG_PROCESS = 10
-DBG_COMMANDS = 30
+DBG_PROCESS = logger.DBGLVL_TRACE_BASIC
+DBG_COMMANDS = logger.DBGLVL_TRACE_DETAIL
 
 # Assign this process some longer name
 isc.util.process.rename(sys.argv[0])
@@ -84,54 +89,9 @@ isc.util.process.rename(sys.argv[0])
 # number, and the overall BIND 10 version number (set in configure.ac).
 VERSION = "bind10 20110223 (BIND 10 @PACKAGE_VERSION@)"
 
-# This is for bind10.boottime of stats module
+# This is for boot_time of Boss
 _BASETIME = time.gmtime()
 
-class RestartSchedule:
-    """
-Keeps state when restarting something (in this case, a process).
-
-When a process dies unexpectedly, we need to restart it. However, if 
-it fails to restart for some reason, then we should not simply keep
-restarting it at high speed.
-
-A more sophisticated algorithm can be developed, but for now we choose
-a simple set of rules:
-
-  * If a process was been running for >=10 seconds, we restart it
-    right away.
-  * If a process was running for <10 seconds, we wait until 10 seconds
-    after it was started.
-
-To avoid programs getting into lockstep, we use a normal distribution
-to avoid being restarted at exactly 10 seconds."""
-
-    def __init__(self, restart_frequency=10.0):
-        self.restart_frequency = restart_frequency
-        self.run_start_time = None
-        self.run_stop_time = None
-        self.restart_time = None
-    
-    def set_run_start_time(self, when=None):
-        if when is None:
-            when = time.time()
-        self.run_start_time = when
-        sigma = self.restart_frequency * 0.05
-        self.restart_time = when + random.normalvariate(self.restart_frequency, 
-                                                        sigma)
-
-    def set_run_stop_time(self, when=None):
-        """We don't actually do anything with stop time now, but it 
-        might be useful for future algorithms."""
-        if when is None:
-            when = time.time()
-        self.run_stop_time = when
-
-    def get_restart_time(self, when=None):
-        if when is None:
-            when = time.time()
-        return max(when, self.restart_time)
-
 class ProcessInfoError(Exception): pass
 
 class ProcessInfo:
@@ -146,7 +106,6 @@ class ProcessInfo:
         self.env = env
         self.dev_null_stdout = dev_null_stdout
         self.dev_null_stderr = dev_null_stderr
-        self.restart_schedule = RestartSchedule()
         self.uid = uid
         self.username = username
         self.process = None
@@ -183,9 +142,9 @@ class ProcessInfo:
         # Environment variables for the child process will be a copy of those
         # of the boss process with any additional specific variables given
         # on construction (self.env).
-        spawn_env = os.environ
+        spawn_env = copy.deepcopy(os.environ)
         spawn_env.update(self.env)
-        if 'B10_FROM_SOURCE' not in os.environ:
+        if ADD_LIBEXEC_PATH:
             spawn_env['PATH'] = "@@LIBEXECDIR@@:" + spawn_env['PATH']
         self.process = subprocess.Popen(self.args,
                                         stdin=subprocess.PIPE,
@@ -195,7 +154,6 @@ class ProcessInfo:
                                         env=spawn_env,
                                         preexec_fn=self._preexec_work)
         self.pid = self.process.pid
-        self.restart_schedule.set_run_start_time()
 
     # spawn() and respawn() are the same for now, but in the future they
     # may have different functionality
@@ -207,12 +165,14 @@ class ProcessInfo:
 
 class CChannelConnectError(Exception): pass
 
+class ProcessStartError(Exception): pass
+
 class BoB:
     """Boss of BIND class."""
     
     def __init__(self, msgq_socket_file=None, data_path=None,
     config_filename=None, nocache=False, verbose=False, setuid=None,
-    username=None, cmdctl_port=None, brittle=False):
+    username=None, cmdctl_port=None, wait_time=10):
         """
             Initialize the Boss of BIND. This is a singleton (only one can run).
         
@@ -220,26 +180,30 @@ class BoB:
             msgq process listens on.  If verbose is True, then the boss reports
             what it is doing.
 
-            Data path and config filename are passed trough to config manager
+            Data path and config filename are passed through to config manager
             (if provided) and specify the config file to be used.
 
             The cmdctl_port is passed to cmdctl and specify on which port it
             should listen.
+
+            wait_time controls the amount of time (in seconds) that Boss waits
+            for selected processes to initialize before continuing with the
+            initialization.  Currently this is only the configuration manager.
         """
         self.cc_session = None
         self.ccs = None
-        self.cfg_start_auth = True
-        self.cfg_start_resolver = False
-        self.cfg_start_dhcp6 = False
-        self.cfg_start_dhcp4 = False
-        self.started_auth_family = False
-        self.started_resolver_family = False
         self.curproc = None
-        self.dead_processes = {}
         self.msgq_socket_file = msgq_socket_file
         self.nocache = nocache
-        self.processes = {}
-        self.expected_shutdowns = {}
+        self.component_config = {}
+        # Some time in future, it may happen that a single component has
+        # multple processes. If so happens, name "components" may be
+        # inapropriate. But as the code isn't probably completely ready
+        # for it, we leave it at components for now.
+        self.components = {}
+        # Simply list of components that died and need to wait for a
+        # restart. Components manage their own restart schedule now
+        self.components_to_restart = []
         self.runnable = False
         self.uid = setuid
         self.username = username
@@ -247,64 +211,76 @@ class BoB:
         self.data_path = data_path
         self.config_filename = config_filename
         self.cmdctl_port = cmdctl_port
-        self.brittle = brittle
+        self.wait_time = wait_time
+        self._component_configurator = isc.bind10.component.Configurator(self,
+            isc.bind10.special_component.get_specials())
+        # The priorities here make them start in the correct order. First
+        # the socket creator (which would drop root privileges by then),
+        # then message queue and after that the config manager (which uses
+        # the config manager)
+        self.__core_components = {
+            'sockcreator': {
+                'kind': 'core',
+                'special': 'sockcreator',
+                'priority': 200
+            },
+            'msgq': {
+                'kind': 'core',
+                'special': 'msgq',
+                'priority': 199
+            },
+            'cfgmgr': {
+                'kind': 'core',
+                'special': 'cfgmgr',
+                'priority': 198
+            }
+        }
+        self.__started = False
+        self.exitcode = 0
+
+        # If -v was set, enable full debug logging.
+        if self.verbose:
+            logger.set_severity("DEBUG", 99)
+
+    def __propagate_component_config(self, config):
+        comps = dict(config)
+        # Fill in the core components, so they stay alive
+        for comp in self.__core_components:
+            if comp in comps:
+                raise Exception(comp + " is core component managed by " +
+                                "bind10 boss, do not set it")
+            comps[comp] = self.__core_components[comp]
+        # Update the configuration
+        self._component_configurator.reconfigure(comps)
 
     def config_handler(self, new_config):
         # If this is initial update, don't do anything now, leave it to startup
         if not self.runnable:
             return
-        # Now we declare few functions used only internally here. Besides the
-        # benefit of not polluting the name space, they are closures, so we
-        # don't need to pass some variables
-        def start_stop(name, started, start, stop):
-            if not'start_' + name in new_config:
-                return
-            if new_config['start_' + name]:
-                if not started:
-                    if self.uid is not None:
-                        logger.info(BIND10_START_AS_NON_ROOT, name)
-                    start()
-            else:
-                stop()
-        # These four functions are passed to start_stop (smells like functional
-        # programming little bit)
-        def resolver_on():
-            self.start_resolver(self.c_channel_env)
-            self.started_resolver_family = True
-        def resolver_off():
-            self.stop_resolver()
-            self.started_resolver_family = False
-        def auth_on():
-            self.start_auth(self.c_channel_env)
-            self.start_xfrout(self.c_channel_env)
-            self.start_xfrin(self.c_channel_env)
-            self.start_zonemgr(self.c_channel_env)
-            self.started_auth_family = True
-        def auth_off():
-            self.stop_zonemgr()
-            self.stop_xfrin()
-            self.stop_xfrout()
-            self.stop_auth()
-            self.started_auth_family = False
-
-        # The real code of the config handler function follows here
         logger.debug(DBG_COMMANDS, BIND10_RECEIVED_NEW_CONFIGURATION,
                      new_config)
-        start_stop('resolver', self.started_resolver_family, resolver_on,
-            resolver_off)
-        start_stop('auth', self.started_auth_family, auth_on, auth_off)
-
-        answer = isc.config.ccsession.create_answer(0)
-        return answer
+        try:
+            if 'components' in new_config:
+                self.__propagate_component_config(new_config['components'])
+            return isc.config.ccsession.create_answer(0)
+        except Exception as e:
+            return isc.config.ccsession.create_answer(1, str(e))
 
     def get_processes(self):
-        pids = list(self.processes.keys())
+        pids = list(self.components.keys())
         pids.sort()
         process_list = [ ]
         for pid in pids:
-            process_list.append([pid, self.processes[pid].name])
+            process_list.append([pid, self.components[pid].name()])
         return process_list
 
+    def _get_stats_data(self):
+        return { "owner": "Boss",
+                 "data": { 'boot_time':
+                               time.strftime('%Y-%m-%dT%H:%M:%SZ', _BASETIME)
+                           }
+                 }
+
     def command_handler(self, command, args):
         logger.debug(DBG_COMMANDS, BIND10_RECEIVED_COMMAND, command)
         answer = isc.config.ccsession.create_answer(1, "command not implemented")
@@ -314,15 +290,26 @@ class BoB:
             if command == "shutdown":
                 self.runnable = False
                 answer = isc.config.ccsession.create_answer(0)
+            elif command == "getstats":
+                answer = isc.config.ccsession.create_answer(0, self._get_stats_data())
             elif command == "sendstats":
                 # send statistics data to the stats daemon immediately
-                cmd = isc.config.ccsession.create_command(
-                    'set', { "stats_data": {
-                            'bind10.boot_time': time.strftime('%Y-%m-%dT%H:%M:%SZ', _BASETIME)
-                            }})
-                seq = self.cc_session.group_sendmsg(cmd, 'Stats')
-                self.cc_session.group_recvmsg(True, seq)
-                answer = isc.config.ccsession.create_answer(0)
+                stats_data = self._get_stats_data()
+                valid = self.ccs.get_module_spec().validate_statistics(
+                    True, stats_data["data"])
+                if valid:
+                    cmd = isc.config.ccsession.create_command('set', stats_data)
+                    seq = self.cc_session.group_sendmsg(cmd, 'Stats')
+                    # Consume the answer, in case it becomes a orphan message.
+                    try:
+                        self.cc_session.group_recvmsg(False, seq)
+                    except isc.cc.session.SessionTimeout:
+                        pass
+                    answer = isc.config.ccsession.create_answer(0)
+                else:
+                    logger.fatal(BIND10_INVALID_STATISTICS_DATA);
+                    answer = isc.config.ccsession.create_answer(
+                        1, "specified statistics data is invalid")
             elif command == "ping":
                 answer = isc.config.ccsession.create_answer(0, "pong")
             elif command == "show_processes":
@@ -333,7 +320,7 @@ class BoB:
                                                             "Unknown command")
         return answer
 
-    def kill_started_processes(self):
+    def kill_started_components(self):
         """
             Called as part of the exception handling when a process fails to
             start, this runs through the list of started processes, killing
@@ -341,29 +328,25 @@ class BoB:
         """
         logger.info(BIND10_KILLING_ALL_PROCESSES)
 
-        for pid in self.processes:
-            logger.info(BIND10_KILL_PROCESS, self.processes[pid].name)
-            self.processes[pid].process.kill()
-        self.processes = {}
+        for pid in self.components:
+            logger.info(BIND10_KILL_PROCESS, self.components[pid].name())
+            self.components[pid].kill(True)
+        self.components = {}
 
-    def read_bind10_config(self):
+    def _read_bind10_config(self):
         """
             Reads the parameters associated with the BoB module itself.
 
-            At present these are the components to start although arguably this
-            information should be in the configuration for the appropriate
-            module itself. (However, this would cause difficulty in the case of
-            xfrin/xfrout and zone manager as we don't need to start those if we
-            are not running the authoritative server.)
+            This means the list of components we should start now.
+
+            This could easily be combined into start_all_processes, but
+            it stays because of historical reasons and because the tests
+            replace the method sometimes.
         """
         logger.info(BIND10_READING_BOSS_CONFIGURATION)
 
         config_data = self.ccs.get_full_config()
-        self.cfg_start_auth = config_data.get("start_auth")
-        self.cfg_start_resolver = config_data.get("start_resolver")
-
-        logger.info(BIND10_CONFIGURATION_START_AUTH, self.cfg_start_auth)
-        logger.info(BIND10_CONFIGURATION_START_RESOLVER, self.cfg_start_resolver)
+        self.__propagate_component_config(config_data['components'])
 
     def log_starting(self, process, port = None, address = None):
         """
@@ -399,22 +382,42 @@ class BoB:
         else:
             logger.debug(DBG_PROCESS, BIND10_STARTED_PROCESS_PID, self.curproc, pid)
 
+    def process_running(self, msg, who):
+        """
+            Some processes return a message to the Boss after they have
+            started to indicate that they are running.  The form of the
+            message is a dictionary with contents {"running:", "<process>"}.
+            This method checks the passed message and returns True if the
+            "who" process is contained in the message (so is presumably
+            running).  It returns False for all other conditions and will
+            log an error if appropriate.
+        """
+        if msg is not None:
+            try:
+                if msg["running"] == who:
+                    return True
+                else:
+                    logger.error(BIND10_STARTUP_UNEXPECTED_MESSAGE, msg)
+            except:
+                logger.error(BIND10_STARTUP_UNRECOGNISED_MESSAGE, msg)
+        
+        return False
+
     # The next few methods start the individual processes of BIND-10.  They
     # are called via start_all_processes().  If any fail, an exception is
     # raised which is caught by the caller of start_all_processes(); this kills
     # processes started up to that point before terminating the program.
 
-    def start_msgq(self, c_channel_env):
+    def start_msgq(self):
         """
             Start the message queue and connect to the command channel.
         """
         self.log_starting("b10-msgq")
-        c_channel = ProcessInfo("b10-msgq", ["b10-msgq"], c_channel_env,
+        msgq_proc = ProcessInfo("b10-msgq", ["b10-msgq"], self.c_channel_env,
                                 True, not self.verbose, uid=self.uid,
                                 username=self.username)
-        c_channel.spawn()
-        self.processes[c_channel.pid] = c_channel
-        self.log_started(c_channel.pid)
+        msgq_proc.spawn()
+        self.log_started(msgq_proc.pid)
 
         # Now connect to the c-channel
         cc_connect_start = time.time()
@@ -429,7 +432,13 @@ class BoB:
             except isc.cc.session.SessionError:
                 time.sleep(0.1)
 
-    def start_cfgmgr(self, c_channel_env):
+        # Subscribe to the message queue.  The only messages we expect to receive
+        # on this channel are once relating to process startup.
+        self.cc_session.group_subscribe("Boss")
+
+        return msgq_proc
+
+    def start_cfgmgr(self):
         """
             Starts the configuration manager process
         """
@@ -440,17 +449,25 @@ class BoB:
         if self.config_filename is not None:
             args.append("--config-filename=" + self.config_filename)
         bind_cfgd = ProcessInfo("b10-cfgmgr", args,
-                                c_channel_env, uid=self.uid,
+                                self.c_channel_env, uid=self.uid,
                                 username=self.username)
         bind_cfgd.spawn()
-        self.processes[bind_cfgd.pid] = bind_cfgd
         self.log_started(bind_cfgd.pid)
 
-        # sleep until b10-cfgmgr is fully up and running, this is a good place
-        # to have a (short) timeout on synchronized groupsend/receive
-        # TODO: replace the sleep by a listen for ConfigManager started
-        # message
-        time.sleep(1)
+        # Wait for the configuration manager to start up as subsequent initialization
+        # cannot proceed without it.  The time to wait can be set on the command line.
+        time_remaining = self.wait_time
+        msg, env = self.cc_session.group_recvmsg()
+        while time_remaining > 0 and not self.process_running(msg, "ConfigManager"):
+            logger.debug(DBG_PROCESS, BIND10_WAIT_CFGMGR)
+            time.sleep(1)
+            time_remaining = time_remaining - 1
+            msg, env = self.cc_session.group_recvmsg()
+        
+        if not self.process_running(msg, "ConfigManager"):
+            raise ProcessStartError("Configuration manager process has not started")
+
+        return bind_cfgd
 
     def start_ccsession(self, c_channel_env):
         """
@@ -458,13 +475,17 @@ class BoB:
 
             The argument c_channel_env is unused but is supplied to keep the
             argument list the same for all start_xxx methods.
+
+            With regards to logging, note that as the CC session is not a
+            process, the log_starting/log_started methods are not used.
         """
-        self.log_starting("ccsession")
+        logger.info(BIND10_STARTING_CC)
         self.ccs = isc.config.ModuleCCSession(SPECFILE_LOCATION, 
                                       self.config_handler,
-                                      self.command_handler)
+                                      self.command_handler,
+                                      socket_file = self.msgq_socket_file)
         self.ccs.start()
-        self.log_started()
+        logger.debug(DBG_PROCESS, BIND10_STARTED_CC)
 
     # A couple of utility methods for starting processes...
 
@@ -479,10 +500,20 @@ class BoB:
         self.log_starting(name, port, address)
         newproc = ProcessInfo(name, args, c_channel_env)
         newproc.spawn()
-        self.processes[newproc.pid] = newproc
         self.log_started(newproc.pid)
+        return newproc
+
+    def register_process(self, pid, component):
+        """
+        Put another process into boss to watch over it.  When the process
+        dies, the component.failed() is called with the exit code.
 
-    def start_simple(self, name, c_channel_env, port=None, address=None):
+        It is expected the info is a isc.bind10.component.BaseComponent
+        subclass (or anything having the same interface).
+        """
+        self.components[pid] = component
+
+    def start_simple(self, name):
         """
             Most of the BIND-10 processes are started with the command:
 
@@ -499,7 +530,7 @@ class BoB:
             args += ['-v']
 
         # ... and start the process
-        self.start_process(name, args, c_channel_env, port, address)
+        return self.start_process(name, args, self.c_channel_env)
 
     # The next few methods start up the rest of the BIND-10 processes.
     # Although many of these methods are little more than a call to
@@ -507,10 +538,12 @@ class BoB:
     # where modifications can be made if the process start-up sequence changes
     # for a given process.
 
-    def start_auth(self, c_channel_env):
+    def start_auth(self):
         """
             Start the Authoritative server
         """
+        if self.uid is not None and self.__started:
+            logger.warn(BIND10_START_AS_NON_ROOT_AUTH)
         authargs = ['b10-auth']
         if self.nocache:
             authargs += ['-n']
@@ -520,14 +553,16 @@ class BoB:
             authargs += ['-v']
 
         # ... and start
-        self.start_process("b10-auth", authargs, c_channel_env)
+        return self.start_process("b10-auth", authargs, self.c_channel_env)
 
-    def start_resolver(self, c_channel_env):
+    def start_resolver(self):
         """
             Start the Resolver.  At present, all these arguments and switches
             are pure speculation.  As with the auth daemon, they should be
             read from the configuration database.
         """
+        if self.uid is not None and self.__started:
+            logger.warn(BIND10_START_AS_NON_ROOT_RESOLVER)
         self.curproc = "b10-resolver"
         # XXX: this must be read from the configuration manager in the future
         resargs = ['b10-resolver']
@@ -537,80 +572,38 @@ class BoB:
             resargs += ['-v']
 
         # ... and start
-        self.start_process("b10-resolver", resargs, c_channel_env)
-
-    def start_xfrout(self, c_channel_env):
-        self.start_simple("b10-xfrout", c_channel_env)
-
-    def start_xfrin(self, c_channel_env):
-        self.start_simple("b10-xfrin", c_channel_env)
+        return self.start_process("b10-resolver", resargs, self.c_channel_env)
 
-    def start_zonemgr(self, c_channel_env):
-        self.start_simple("b10-zonemgr", c_channel_env)
-
-    def start_stats(self, c_channel_env):
-        self.start_simple("b10-stats", c_channel_env)
-
-    def start_stats_httpd(self, c_channel_env):
-        self.start_simple("b10-stats-httpd", c_channel_env)
-
-    def start_dhcp6(self, c_channel_env):
-        self.start_simple("b10-dhcp6", c_channel_env)
-
-    def start_cmdctl(self, c_channel_env):
+    def start_cmdctl(self):
         """
             Starts the command control process
         """
         args = ["b10-cmdctl"]
         if self.cmdctl_port is not None:
             args.append("--port=" + str(self.cmdctl_port))
-        self.start_process("b10-cmdctl", args, c_channel_env, self.cmdctl_port)
+        if self.verbose:
+            args.append("-v")
+        return self.start_process("b10-cmdctl", args, self.c_channel_env,
+                                  self.cmdctl_port)
 
-    def start_all_processes(self):
+    def start_all_components(self):
         """
-            Starts up all the processes.  Any exception generated during the
-            starting of the processes is handled by the caller.
+            Starts up all the components.  Any exception generated during the
+            starting of the components is handled by the caller.
         """
-        c_channel_env = self.c_channel_env
-        self.start_msgq(c_channel_env)
-        self.start_cfgmgr(c_channel_env)
-        self.start_ccsession(c_channel_env)
-
-        # Extract the parameters associated with Bob.  This can only be
-        # done after the CC Session is started.
-        self.read_bind10_config()
-
-        # Continue starting the processes.  The authoritative server (if
-        # selected):
-        if self.cfg_start_auth:
-            self.start_auth(c_channel_env)
-
-        # ... and resolver (if selected):
-        if self.cfg_start_resolver:
-            self.start_resolver(c_channel_env)
-            self.started_resolver_family = True
-
-        # Everything after the main components can run as non-root.
-        # TODO: this is only temporary - once the privileged socket creator is
-        # fully working, nothing else will run as root.
-        if self.uid is not None:
-            posix.setuid(self.uid)
+        # Start the real core (sockcreator, msgq, cfgmgr)
+        self._component_configurator.startup(self.__core_components)
 
-        # xfrin/xfrout and the zone manager are only meaningful if the
-        # authoritative server has been started.
-        if self.cfg_start_auth:
-            self.start_xfrout(c_channel_env)
-            self.start_xfrin(c_channel_env)
-            self.start_zonemgr(c_channel_env)
-            self.started_auth_family = True
+        # Connect to the msgq. This is not a process, so it's not handled
+        # inside the configurator.
+        self.start_ccsession(self.c_channel_env)
 
-        # ... and finally start the remaining processes
-        self.start_stats(c_channel_env)
-        self.start_stats_httpd(c_channel_env)
-        self.start_cmdctl(c_channel_env)
+        # Extract the parameters associated with Bob.  This can only be
+        # done after the CC Session is started.  Note that the logging
+        # configuration may override the "-v" switch set on the command line.
+        self._read_bind10_config()
 
-        if self.cfg_start_dhcp6:
-            self.start_dhcp6(c_channel_env)
+        # TODO: Return the dropping of privileges
 
     def startup(self):
         """
@@ -634,97 +627,81 @@ class BoB:
             # this is the case we want, where the msgq is not running
             pass
 
-        # Start all processes.  If any one fails to start, kill all started
-        # processes and exit with an error indication.
+        # Start all components.  If any one fails to start, kill all started
+        # components and exit with an error indication.
         try:
             self.c_channel_env = c_channel_env
-            self.start_all_processes()
+            self.start_all_components()
         except Exception as e:
-            self.kill_started_processes()
+            self.kill_started_components()
             return "Unable to start " + self.curproc + ": " + str(e)
 
         # Started successfully
         self.runnable = True
+        self.__started = True
         return None
 
-    def stop_all_processes(self):
-        """Stop all processes."""
-        cmd = { "command": ['shutdown']}
-
-        self.cc_session.group_sendmsg(cmd, 'Cmdctl', 'Cmdctl')
-        self.cc_session.group_sendmsg(cmd, "ConfigManager", "ConfigManager")
-        self.cc_session.group_sendmsg(cmd, "Auth", "Auth")
-        self.cc_session.group_sendmsg(cmd, "Resolver", "Resolver")
-        self.cc_session.group_sendmsg(cmd, "Xfrout", "Xfrout")
-        self.cc_session.group_sendmsg(cmd, "Xfrin", "Xfrin")
-        self.cc_session.group_sendmsg(cmd, "Zonemgr", "Zonemgr")
-        self.cc_session.group_sendmsg(cmd, "Stats", "Stats")
-        self.cc_session.group_sendmsg(cmd, "StatsHttpd", "StatsHttpd")
-
     def stop_process(self, process, recipient):
         """
         Stop the given process, friendly-like. The process is the name it has
         (in logs, etc), the recipient is the address on msgq.
         """
         logger.info(BIND10_STOP_PROCESS, process)
-        # TODO: Some timeout to solve processes that don't want to die would
-        # help. We can even store it in the dict, it is used only as a set
-        self.expected_shutdowns[process] = 1
-        # Ask the process to die willingly
         self.cc_session.group_sendmsg({'command': ['shutdown']}, recipient,
             recipient)
 
-    # Series of stop_process wrappers
-    def stop_resolver(self):
-        self.stop_process('b10-resolver', 'Resolver')
-
-    def stop_auth(self):
-        self.stop_process('b10-auth', 'Auth')
-
-    def stop_xfrout(self):
-        self.stop_process('b10-xfrout', 'Xfrout')
+    def component_shutdown(self, exitcode=0):
+        """
+        Stop the Boss instance from a components' request. The exitcode
+        indicates the desired exit code.
 
-    def stop_xfrin(self):
-        self.stop_process('b10-xfrin', 'Xfrin')
+        If we did not start yet, it raises an exception, which is meant
+        to propagate through the component and configurator to the startup
+        routine and abort the startup immediately. If it is started up already,
+        we just mark it so we terminate soon.
 
-    def stop_zonemgr(self):
-        self.stop_process('b10-zonemgr', 'Zonemgr')
+        It does set the exit code in both cases.
+        """
+        self.exitcode = exitcode
+        if not self.__started:
+            raise Exception("Component failed during startup");
+        else:
+            self.runnable = False
 
     def shutdown(self):
         """Stop the BoB instance."""
         logger.info(BIND10_SHUTDOWN)
         # first try using the BIND 10 request to stop
         try:
-            self.stop_all_processes()
+            self._component_configurator.shutdown()
         except:
             pass
         # XXX: some delay probably useful... how much is uncertain
         # I have changed the delay from 0.5 to 1, but sometime it's 
         # still not enough.
-        time.sleep(1)  
+        time.sleep(1)
         self.reap_children()
         # next try sending a SIGTERM
-        processes_to_stop = list(self.processes.values())
-        for proc_info in processes_to_stop:
-            logger.info(BIND10_SEND_SIGTERM, proc_info.name,
-                        proc_info.pid)
+        components_to_stop = list(self.components.values())
+        for component in components_to_stop:
+            logger.info(BIND10_SEND_SIGTERM, component.name(), component.pid())
             try:
-                proc_info.process.terminate()
+                component.kill()
             except OSError:
                 # ignore these (usually ESRCH because the child
                 # finally exited)
                 pass
         # finally, send SIGKILL (unmaskable termination) until everybody dies
-        while self.processes:
+        while self.components:
             # XXX: some delay probably useful... how much is uncertain
             time.sleep(0.1)  
             self.reap_children()
-            processes_to_stop = list(self.processes.values())
-            for proc_info in processes_to_stop:
-                logger.info(BIND10_SEND_SIGKILL, proc_info.name,
-                            proc_info.pid)
+            components_to_stop = list(self.components.values())
+            for component in components_to_stop:
+                logger.info(BIND10_SEND_SIGKILL, component.name(),
+                            component.pid())
                 try:
-                    proc_info.process.kill()
+                    component.kill(True)
                 except OSError:
                     # ignore these (usually ESRCH because the child
                     # finally exited)
@@ -746,33 +723,20 @@ class BoB:
                 # XXX: should be impossible to get any other error here
                 raise
             if pid == 0: break
-            if pid in self.processes:
-                # One of the processes we know about.  Get information on it.
-                proc_info = self.processes.pop(pid)
-                proc_info.restart_schedule.set_run_stop_time()
-                self.dead_processes[proc_info.pid] = proc_info
-
-                # Write out message, but only if in the running state:
-                # During startup and shutdown, these messages are handled
-                # elsewhere.
-                if self.runnable:
-                    if exit_status is None:
-                        logger.warn(BIND10_PROCESS_ENDED_NO_EXIT_STATUS,
-                                    proc_info.name, proc_info.pid)
-                    else:
-                        logger.warn(BIND10_PROCESS_ENDED_WITH_EXIT_STATUS,
-                                    proc_info.name, proc_info.pid,
-                                    exit_status)
-
-                    # Was it a special process?
-                    if proc_info.name == "b10-msgq":
-                        logger.fatal(BIND10_MSGQ_DAEMON_ENDED)
-                        self.runnable = False
-
-                # If we're in 'brittle' mode, we want to shutdown after
-                # any process dies.
-                if self.brittle:
-                    self.runnable = False
+            if pid in self.components:
+                # One of the components we know about.  Get information on it.
+                component = self.components.pop(pid)
+                logger.info(BIND10_PROCESS_ENDED, component.name(), pid,
+                            exit_status)
+                if component.running() and self.runnable:
+                    # Tell it it failed. But only if it matters (we are
+                    # not shutting down and the component considers itself
+                    # to be running.
+                    component_restarted = component.failed(exit_status);
+                    # if the process wants to be restarted, but not just yet,
+                    # it returns False
+                    if not component_restarted:
+                        self.components_to_restart.append(component)
             else:
                 logger.info(BIND10_UNKNOWN_CHILD_PROCESS_ENDED, pid)
 
@@ -786,36 +750,24 @@ class BoB:
 
             The values returned can be safely passed into select() as the 
             timeout value.
+
         """
-        next_restart = None
-        # if we're shutting down, then don't restart
         if not self.runnable:
             return 0
-        # otherwise look through each dead process and try to restart
-        still_dead = {}
+        still_dead = []
+        # keep track of the first time we need to check this queue again,
+        # if at all
+        next_restart_time = None
         now = time.time()
-        for proc_info in self.dead_processes.values():
-            if proc_info.name in self.expected_shutdowns:
-                # We don't restart, we wanted it to die
-                del self.expected_shutdowns[proc_info.name]
-                continue
-            restart_time = proc_info.restart_schedule.get_restart_time(now)
-            if restart_time > now:
-                if (next_restart is None) or (next_restart > restart_time):
-                    next_restart = restart_time
-                still_dead[proc_info.pid] = proc_info
-            else:
-                logger.info(BIND10_RESURRECTING_PROCESS, proc_info.name)
-                try:
-                    proc_info.respawn()
-                    self.processes[proc_info.pid] = proc_info
-                    logger.info(BIND10_RESURRECTED_PROCESS, proc_info.name, proc_info.pid)
-                except:
-                    still_dead[proc_info.pid] = proc_info
-        # remember any processes that refuse to be resurrected
-        self.dead_processes = still_dead
-        # return the time when the next process is ready to be restarted
-        return next_restart
+        for component in self.components_to_restart:
+            if not component.restart(now):
+                still_dead.append(component)
+                if next_restart_time is None or\
+                   next_restart_time > component.get_restart_time():
+                    next_restart_time = component.get_restart_time()
+        self.components_to_restart = still_dead
+
+        return next_restart_time
 
 # global variables, needed for signal handlers
 options = None
@@ -878,8 +830,8 @@ def parse_args(args=sys.argv[1:], Parser=OptionParser):
     parser.add_option("--pid-file", dest="pid_file", type="string",
                       default=None,
                       help="file to dump the PID of the BIND 10 process")
-    parser.add_option("--brittle", dest="brittle", action="store_true",
-                      help="debugging flag: exit if any component dies")
+    parser.add_option("-w", "--wait", dest="wait_time", type="int",
+                      default=10, help="Time (in seconds) to wait for config manager to start up")
 
     (options, args) = parser.parse_args(args)
 
@@ -982,7 +934,8 @@ def main():
     # Go bob!
     boss_of_bind = BoB(options.msgq_socket_file, options.data_path,
                        options.config_file, options.nocache, options.verbose,
-                       setuid, username, options.cmdctl_port, options.brittle)
+                       setuid, username, options.cmdctl_port,
+                       options.wait_time)
     startup_result = boss_of_bind.startup()
     if startup_result:
         logger.fatal(BIND10_STARTUP_ERROR, startup_result)

src/bin/bind10/bob.spec

@@ -4,16 +4,71 @@
     "module_description": "Master process",
     "config_data": [
       {
-        "item_name": "start_auth",
-        "item_type": "boolean",
+        "item_name": "components",
+        "item_type": "named_set",
         "item_optional": false,
-        "item_default": true
-      },
-      {
-        "item_name": "start_resolver",
-        "item_type": "boolean",
-        "item_optional": false,
-        "item_default": false
+        "item_default": {
+          "b10-auth": { "special": "auth", "kind": "needed", "priority": 10 },
+          "setuid": {
+            "special": "setuid",
+            "priority": 5,
+            "kind": "dispensable"
+          },
+          "b10-xfrin": { "address": "Xfrin", "kind": "dispensable" },
+          "b10-xfrout": { "address": "Xfrout", "kind": "dispensable" },
+          "b10-zonemgr": { "address": "Zonemgr", "kind": "dispensable" },
+          "b10-stats": { "address": "Stats", "kind": "dispensable" },
+          "b10-stats-httpd": {
+            "address": "StatsHttpd",
+            "kind": "dispensable"
+          },
+          "b10-cmdctl": { "special": "cmdctl", "kind": "needed" }
+        },
+        "named_set_item_spec": {
+          "item_name": "component",
+          "item_type": "map",
+          "item_optional": false,
+          "item_default": { },
+          "map_item_spec": [
+            {
+              "item_name": "special",
+              "item_optional": true,
+              "item_type": "string"
+            },
+            {
+              "item_name": "process",
+              "item_optional": true,
+              "item_type": "string"
+            },
+            {
+              "item_name": "kind",
+              "item_optional": false,
+              "item_type": "string",
+              "item_default": "dispensable"
+            },
+            {
+              "item_name": "address",
+              "item_optional": true,
+              "item_type": "string"
+            },
+            {
+              "item_name": "params",
+              "item_optional": true,
+              "item_type": "list",
+              "list_item_spec": {
+                "item_name": "param",
+                "item_optional": false,
+                "item_type": "string",
+                "item_default": ""
+              }
+            },
+            {
+              "item_name": "priority",
+              "item_optional": true,
+              "item_type": "integer"
+            }
+          ]
+        }
       }
     ],
     "commands": [
@@ -37,6 +92,17 @@
         "command_description": "List the running BIND 10 processes",
         "command_args": []
       }
+    ],
+    "statistics": [
+      {
+        "item_name": "boot_time",
+        "item_type": "string",
+        "item_optional": false,
+        "item_default": "1970-01-01T00:00:00Z",
+        "item_title": "Boot time",
+        "item_description": "A date time when bind10 process starts initially",
+        "item_format": "date-time"
+      }
     ]
   }
 }

src/bin/bind10/creatorapi.txt

@@ -0,0 +1,123 @@
+Socket creator API
+==================
+
+This API is between Boss and other modules to allow them requesting of sockets.
+For simplicity, we will use the socket creator for all (even non-privileged)
+ports for now, but we should have some function where we can abstract it later.
+
+Goals
+-----
+* Be able to request a socket of any combination IPv4/IPv6 UDP/TCP bound to given
+  port and address (sockets that are not bound to anything can be created
+  without privileges, therefore are not requested from the socket creator).
+* Allow to provide the same socket to multiple modules (eg. multiple running
+  auth servers).
+* Allow releasing the sockets (in case all modules using it give it up,
+  terminate or crash).
+* Allow restricting of the sharing (don't allow shared socket between auth
+  and recursive, as the packets would often get to the wrong application,
+  show error instead).
+* Get the socket to the application.
+
+Transport of sockets
+--------------------
+It seems we are stuck with current msgq for a while and there's a chance the
+new replacement will not be able to send sockets inbound. So, we need another
+channel.
+
+The boss will create a unix-domain socket and listen on it. When something
+requests a socket over the command channel and the socket is created, some kind
+of token is returned to the application (which will represent the future
+socket). The application then connects to the unix-domain socket, sends the
+token over the connection (so Boss will know which socket to send there, in case
+multiple applications ask for sockets simultaneously) and Boss sends the socket
+in return.
+
+In theory, we could send the requests directly over the unix-domain
+socket, but it has two disadvantages:
+* The msgq handles serializing/deserializing of structured
+  information (like the parameters to be used), we would have to do it
+  manually on the socket.
+* We could place some kind of security in front of msgq (in case file
+  permissions are not enough, for example if they are not honored on
+  socket files, as indicated in the first paragraph of:
+  http://lkml.indiana.edu/hypermail/linux/kernel/0505.2/0008.html).
+  The socket would have to be secured separately. With the tokens,
+  there's some level of security already - someone not having the
+  token can't request a priviledged socket.
+
+Caching of sockets
+------------------
+To allow sending the same socket to multiple application, the Boss process will
+hold a cache. Each socket that is created and sent is kept open in Boss and
+preserved there as well. A reference count is kept with each of them.
+
+When another application asks for the same socket, it is simply sent from the
+cache instead of creating it again by the creator.
+
+When application gives the socket willingly (by sending a message over the
+command channel), the reference count can be decreased without problems. But
+when the application terminates or crashes, we need to decrease it as well.
+There's a problem, since we don't know which command channel connection (eg.
+lname) belongs to which PID. Furthermore, the applications don't need to be
+started by boss.
+
+There are two possibilities:
+* Let the msgq send messages about disconnected clients (eg. group message to
+  some name). This one is better if we want to migrate to dbus, since dbus
+  already has this capability as well as sending the sockets inbound (at least it
+  seems so on unix) and we could get rid of the unix-domain socket completely.
+* Keep the unix-domain connections open forever. Boss can remember which socket
+  was sent to which connection and when the connection closes (because the
+  application crashed), it can drop all the references on the sockets. This
+  seems easier to implement.
+
+The commands
+------------
+* Command to release a socket. This one would have single parameter, the token
+  used to get the socket. After this, boss would decrease its reference count
+  and if it drops to zero, close its own copy of the socket. This should be used
+  when the module stops using the socket (and after closes it). The
+  library could remember the file-descriptor to token mapping (for
+  common applications that don't request the same socket multiple
+  times in parallel).
+* Command to request a socket. It would have parameters to specify which socket
+  (IP address, address family, port) and how to allow sharing. Sharing would be
+  one of:
+  - None
+  - Same kind of application (however, it is not entirely clear what
+    this means, in case it won't work out intuitively, we'll need to
+    define it somehow)
+  - Any kind of application
+  And a kind of application would be provided, to decide if the sharing is
+  possible (eg. if auth allows sharing with the same kind and something else
+  allows sharing with anything, the sharing is not possible, two auths can).
+
+  It would return either error (the socket can't be created or sharing is not
+  possible) or the token. Then there would be some time for the application to
+  pick up the requested socket.
+
+Examples
+--------
+We probably would have a library with blocking calls to request the
+sockets, so a code could look like:
+
+(socket_fd, token) = request_socket(address, port, 'UDP', SHARE_SAMENAME, 'test-application')
+sock = socket.fromfd(socket_fd)
+
+# Some sock.send and sock.recv stuff here
+
+sock.close()
+release_socket(socket_fd) # or release_socket(token)
+
+Known limitations
+-----------------
+Currently the socket creator doesn't support specifying any socket
+options. If it turns out there are any options that need to be set
+before bind(), we'll need to extend it (and extend the protocol as
+well). If we want to support them, we'll have to solve a possible
+conflict (what to do when two applications request the same socket and
+want to share it, but want different options).
+
+The current socket creator doesn't know raw sockets, but if they are
+needed, it should be easy to add.

src/bin/bind10/run_bind10.sh.in

@@ -20,17 +20,17 @@ export PYTHON_EXEC
 
 BIND10_PATH=@abs_top_builddir@/src/bin/bind10
 
-PATH=@abs_top_builddir@/src/bin/msgq:@abs_top_builddir@/src/bin/auth:@abs_top_builddir@/src/bin/resolver:@abs_top_builddir@/src/bin/cfgmgr:@abs_top_builddir@/src/bin/cmdctl:@abs_top_builddir@/src/bin/stats:@abs_top_builddir@/src/bin/xfrin:@abs_top_builddir@/src/bin/xfrout:@abs_top_builddir@/src/bin/zonemgr:@abs_top_builddir@/src/bin/dhcp6:$PATH
+PATH=@abs_top_builddir@/src/bin/msgq:@abs_top_builddir@/src/bin/auth:@abs_top_builddir@/src/bin/resolver:@abs_top_builddir@/src/bin/cfgmgr:@abs_top_builddir@/src/bin/cmdctl:@abs_top_builddir@/src/bin/stats:@abs_top_builddir@/src/bin/xfrin:@abs_top_builddir@/src/bin/xfrout:@abs_top_builddir@/src/bin/zonemgr:@abs_top_builddir@/src/bin/dhcp6:@abs_top_builddir@/src/bin/sockcreator:$PATH
 export PATH
 
-PYTHONPATH=@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/xfr/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/python/isc/config
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/xfr/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/python/isc/config:@abs_top_builddir@/src/lib/python/isc/acl/.libs:@abs_top_builddir@/src/lib/python/isc/datasrc/.libs
 export PYTHONPATH
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 SET_ENV_LIBRARY_PATH=@SET_ENV_LIBRARY_PATH@
 if test $SET_ENV_LIBRARY_PATH = yes; then
-	@ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:$@ENV_LIBRARY_PATH@
+	@ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/acl/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:@abs_top_builddir@/src/lib/datasrc/.libs:$@ENV_LIBRARY_PATH@
 	export @ENV_LIBRARY_PATH@
 fi
 
@@ -45,6 +45,5 @@ export B10_FROM_BUILD
 BIND10_MSGQ_SOCKET_FILE=@abs_top_builddir@/msgq_socket
 export BIND10_MSGQ_SOCKET_FILE
 
-cd ${BIND10_PATH}
-exec ${PYTHON_EXEC} -O bind10 "$@"
+exec ${PYTHON_EXEC} -O ${BIND10_PATH}/bind10 "$@"
 

src/bin/bind10/tests/Makefile.am

@@ -2,13 +2,13 @@ PYCOVERAGE_RUN = @PYCOVERAGE_RUN@
 #PYTESTS = args_test.py bind10_test.py
 # NOTE: this has a generated test found in the builddir
 PYTESTS = bind10_test.py
-EXTRA_DIST = $(PYTESTS)
+noinst_SCRIPTS = $(PYTESTS)
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
@@ -20,8 +20,9 @@ if ENABLE_PYTHON_COVERAGE
 endif
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
+	chmod +x $(abs_builddir)/$$pytest ; \
 	$(LIBRARY_PATH_PLACEHOLDER) \
-	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/bind10 \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_srcdir)/src/bin:$(abs_top_builddir)/src/bin/bind10:$(abs_top_builddir)/src/lib/util/io/.libs \
 	BIND10_MSGQ_SOCKET_FILE=$(abs_top_builddir)/msgq_socket \
 		$(PYCOVERAGE_RUN) $(abs_builddir)/$$pytest || exit ; \
 	done

src/bin/bind10/tests/bind10_test.py.in

@@ -13,7 +13,7 @@
 # NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
 # WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 
-from bind10 import ProcessInfo, BoB, parse_args, dump_pid, unlink_pid_file, _BASETIME
+from bind10_src import ProcessInfo, BoB, parse_args, dump_pid, unlink_pid_file, _BASETIME
 
 # XXX: environment tests are currently disabled, due to the preprocessor
 #      setup that we have now complicating the environment
@@ -21,6 +21,7 @@ from bind10 import ProcessInfo, BoB, parse_args, dump_pid, unlink_pid_file, _BAS
 import unittest
 import sys
 import os
+import copy
 import signal
 import socket
 from isc.net.addr import IPAddr
@@ -103,17 +104,11 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.msgq_socket_file, None)
         self.assertEqual(bob.cc_session, None)
         self.assertEqual(bob.ccs, None)
-        self.assertEqual(bob.processes, {})
-        self.assertEqual(bob.dead_processes, {})
+        self.assertEqual(bob.components, {})
         self.assertEqual(bob.runnable, False)
         self.assertEqual(bob.uid, None)
         self.assertEqual(bob.username, None)
         self.assertEqual(bob.nocache, False)
-        self.assertEqual(bob.cfg_start_auth, True)
-        self.assertEqual(bob.cfg_start_resolver, False)
-
-        self.assertEqual(bob.cfg_start_dhcp4, False)
-        self.assertEqual(bob.cfg_start_dhcp6, False)
 
     def test_init_alternate_socket(self):
         bob = BoB("alt_socket_file")
@@ -121,25 +116,38 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.msgq_socket_file, "alt_socket_file")
         self.assertEqual(bob.cc_session, None)
         self.assertEqual(bob.ccs, None)
-        self.assertEqual(bob.processes, {})
-        self.assertEqual(bob.dead_processes, {})
+        self.assertEqual(bob.components, {})
         self.assertEqual(bob.runnable, False)
         self.assertEqual(bob.uid, None)
         self.assertEqual(bob.username, None)
         self.assertEqual(bob.nocache, False)
-        self.assertEqual(bob.cfg_start_auth, True)
-        self.assertEqual(bob.cfg_start_resolver, False)
-        self.assertEqual(bob.cfg_start_dhcp4, False)
-        self.assertEqual(bob.cfg_start_dhcp6, False)
 
     def test_command_handler(self):
         class DummySession():
             def group_sendmsg(self, msg, group):
                 (self.msg, self.group) = (msg, group)
             def group_recvmsg(self, nonblock, seq): pass
+        class DummyModuleCCSession():
+            module_spec = isc.config.module_spec.ModuleSpec({
+                    "module_name": "Boss",
+                    "statistics": [
+                        {
+                            "item_name": "boot_time",
+                            "item_type": "string",
+                            "item_optional": False,
+                            "item_default": "1970-01-01T00:00:00Z",
+                            "item_title": "Boot time",
+                            "item_description": "A date time when bind10 process starts initially",
+                            "item_format": "date-time"
+                            }
+                        ]
+                    })
+            def get_module_spec(self):
+                return self.module_spec
         bob = BoB()
         bob.verbose = True
         bob.cc_session = DummySession()
+        bob.ccs = DummyModuleCCSession()
         # a bad command
         self.assertEqual(bob.command_handler(-1, None),
                          isc.config.ccsession.create_answer(1, "bad command"))
@@ -147,14 +155,22 @@ class TestBoB(unittest.TestCase):
         self.assertEqual(bob.command_handler("shutdown", None),
                          isc.config.ccsession.create_answer(0))
         self.assertFalse(bob.runnable)
+        # "getstats" command
+        self.assertEqual(bob.command_handler("getstats", None),
+                         isc.config.ccsession.create_answer(0,
+                            { "owner": "Boss",
+                              "data": {
+                                'boot_time': time.strftime('%Y-%m-%dT%H:%M:%SZ', _BASETIME)
+                            }}))
         # "sendstats" command
         self.assertEqual(bob.command_handler("sendstats", None),
                          isc.config.ccsession.create_answer(0))
         self.assertEqual(bob.cc_session.group, "Stats")
         self.assertEqual(bob.cc_session.msg,
                          isc.config.ccsession.create_command(
-                'set', { "stats_data": {
-                        'bind10.boot_time': time.strftime('%Y-%m-%dT%H:%M:%SZ', _BASETIME)
+                "set", { "owner": "Boss",
+                         "data": {
+                        "boot_time": time.strftime("%Y-%m-%dT%H:%M:%SZ", _BASETIME)
                         }}))
         # "ping" command
         self.assertEqual(bob.command_handler("ping", None),
@@ -191,142 +207,192 @@ class MockBob(BoB):
         self.stats = False
         self.stats_httpd = False
         self.cmdctl = False
+        self.dhcp6 = False
+        self.dhcp4 = False
         self.c_channel_env = {}
-        self.processes = { }
-
-    def read_bind10_config(self):
+        self.components = { }
+        self.creator = False
+
+        class MockSockCreator(isc.bind10.component.Component):
+            def __init__(self, process, boss, kind, address=None, params=None):
+                isc.bind10.component.Component.__init__(self, process, boss,
+                                                        kind, 'SockCreator')
+                self._start_func = boss.start_creator
+
+        specials = isc.bind10.special_component.get_specials()
+        specials['sockcreator'] = MockSockCreator
+        self._component_configurator = \
+            isc.bind10.component.Configurator(self, specials)
+
+    def start_creator(self):
+        self.creator = True
+        procinfo = ProcessInfo('b10-sockcreator', ['/bin/false'])
+        procinfo.pid = 1
+        return procinfo
+
+    def _read_bind10_config(self):
         # Configuration options are set directly
         pass
 
-    def start_msgq(self, c_channel_env):
+    def start_msgq(self):
         self.msgq = True
-        self.processes[2] = ProcessInfo('b10-msgq', ['/bin/false'])
-        self.processes[2].pid = 2
-
-    def start_cfgmgr(self, c_channel_env):
-        self.cfgmgr = True
-        self.processes[3] = ProcessInfo('b10-cfgmgr', ['/bin/false'])
-        self.processes[3].pid = 3
+        procinfo = ProcessInfo('b10-msgq', ['/bin/false'])
+        procinfo.pid = 2
+        return procinfo
 
     def start_ccsession(self, c_channel_env):
+        # this is not a process, don't have to do anything with procinfo
         self.ccsession = True
-        self.processes[4] = ProcessInfo('b10-ccsession', ['/bin/false'])
-        self.processes[4].pid = 4
 
-    def start_auth(self, c_channel_env):
+    def start_cfgmgr(self):
+        self.cfgmgr = True
+        procinfo = ProcessInfo('b10-cfgmgr', ['/bin/false'])
+        procinfo.pid = 3
+        return procinfo
+
+    def start_auth(self):
         self.auth = True
-        self.processes[5] = ProcessInfo('b10-auth', ['/bin/false'])
-        self.processes[5].pid = 5
+        procinfo = ProcessInfo('b10-auth', ['/bin/false'])
+        procinfo.pid = 5
+        return procinfo
 
-    def start_resolver(self, c_channel_env):
+    def start_resolver(self):
         self.resolver = True
-        self.processes[6] = ProcessInfo('b10-resolver', ['/bin/false'])
-        self.processes[6].pid = 6
-
-    def start_xfrout(self, c_channel_env):
+        procinfo = ProcessInfo('b10-resolver', ['/bin/false'])
+        procinfo.pid = 6
+        return procinfo
+
+    def start_simple(self, name):
+        procmap = { 'b10-zonemgr': self.start_zonemgr,
+                    'b10-stats': self.start_stats,
+                    'b10-stats-httpd': self.start_stats_httpd,
+                    'b10-cmdctl': self.start_cmdctl,
+                    'b10-dhcp6': self.start_dhcp6,
+                    'b10-dhcp4': self.start_dhcp4,
+                    'b10-xfrin': self.start_xfrin,
+                    'b10-xfrout': self.start_xfrout }
+        return procmap[name]()
+
+    def start_xfrout(self):
         self.xfrout = True
-        self.processes[7] = ProcessInfo('b10-xfrout', ['/bin/false'])
-        self.processes[7].pid = 7
+        procinfo = ProcessInfo('b10-xfrout', ['/bin/false'])
+        procinfo.pid = 7
+        return procinfo
 
-    def start_xfrin(self, c_channel_env):
+    def start_xfrin(self):
         self.xfrin = True
-        self.processes[8] = ProcessInfo('b10-xfrin', ['/bin/false'])
-        self.processes[8].pid = 8
+        procinfo = ProcessInfo('b10-xfrin', ['/bin/false'])
+        procinfo.pid = 8
+        return procinfo
 
-    def start_zonemgr(self, c_channel_env):
+    def start_zonemgr(self):
         self.zonemgr = True
-        self.processes[9] = ProcessInfo('b10-zonemgr', ['/bin/false'])
-        self.processes[9].pid = 9
+        procinfo = ProcessInfo('b10-zonemgr', ['/bin/false'])
+        procinfo.pid = 9
+        return procinfo
 
-    def start_stats(self, c_channel_env):
+    def start_stats(self):
         self.stats = True
-        self.processes[10] = ProcessInfo('b10-stats', ['/bin/false'])
-        self.processes[10].pid = 10
+        procinfo = ProcessInfo('b10-stats', ['/bin/false'])
+        procinfo.pid = 10
+        return procinfo
 
-    def start_stats_httpd(self, c_channel_env):
+    def start_stats_httpd(self):
         self.stats_httpd = True
-        self.processes[11] = ProcessInfo('b10-stats-httpd', ['/bin/false'])
-        self.processes[11].pid = 11
+        procinfo = ProcessInfo('b10-stats-httpd', ['/bin/false'])
+        procinfo.pid = 11
+        return procinfo
 
-    def start_cmdctl(self, c_channel_env):
+    def start_cmdctl(self):
         self.cmdctl = True
-        self.processes[12] = ProcessInfo('b10-cmdctl', ['/bin/false'])
-        self.processes[12].pid = 12
+        procinfo = ProcessInfo('b10-cmdctl', ['/bin/false'])
+        procinfo.pid = 12
+        return procinfo
 
-    def start_dhcp6(self, c_channel_env):
+    def start_dhcp6(self):
         self.dhcp6 = True
-        self.processes[13] = ProcessInfo('b10-dhcp6', ['/bin/false'])
-        self.processes[13]
+        procinfo = ProcessInfo('b10-dhcp6', ['/bin/false'])
+        procinfo.pid = 13
+        return procinfo
 
-    def start_dhcp4(self, c_channel_env):
+    def start_dhcp4(self):
         self.dhcp4 = True
-        self.processes[14] = ProcessInfo('b10-dhcp4', ['/bin/false'])
-        self.processes[14]
-
-    # We don't really use all of these stop_ methods. But it might turn out
-    # someone would add some stop_ method to BoB and we want that one overriden
-    # in case he forgets to update the tests.
+        procinfo = ProcessInfo('b10-dhcp4', ['/bin/false'])
+        procinfo.pid = 14
+        return procinfo
+
+    def stop_process(self, process, recipient):
+        procmap = { 'b10-auth': self.stop_auth,
+                    'b10-resolver': self.stop_resolver,
+                    'b10-xfrout': self.stop_xfrout,
+                    'b10-xfrin': self.stop_xfrin,
+                    'b10-zonemgr': self.stop_zonemgr,
+                    'b10-stats': self.stop_stats,
+                    'b10-stats-httpd': self.stop_stats_httpd,
+                    'b10-cmdctl': self.stop_cmdctl }
+        procmap[process]()
+
+    # Some functions to pretend we stop processes, use by stop_process
     def stop_msgq(self):
         if self.msgq:
-            del self.processes[2]
+            del self.components[2]
         self.msgq = False
 
     def stop_cfgmgr(self):
         if self.cfgmgr:
-            del self.processes[3]
+            del self.components[3]
         self.cfgmgr = False
 
-    def stop_ccsession(self):
-        if self.ccssession:
-            del self.processes[4]
-        self.ccsession = False
-
     def stop_auth(self):
         if self.auth:
-            del self.processes[5]
+            del self.components[5]
         self.auth = False
 
     def stop_resolver(self):
         if self.resolver:
-            del self.processes[6]
+            del self.components[6]
         self.resolver = False
 
     def stop_xfrout(self):
         if self.xfrout:
-            del self.processes[7]
+            del self.components[7]
         self.xfrout = False
 
     def stop_xfrin(self):
         if self.xfrin:
-            del self.processes[8]
+            del self.components[8]
         self.xfrin = False
 
     def stop_zonemgr(self):
         if self.zonemgr:
-            del self.processes[9]
+            del self.components[9]
         self.zonemgr = False
 
     def stop_stats(self):
         if self.stats:
-            del self.processes[10]
+            del self.components[10]
         self.stats = False
 
     def stop_stats_httpd(self):
         if self.stats_httpd:
-            del self.processes[11]
+            del self.components[11]
         self.stats_httpd = False
 
     def stop_cmdctl(self):
         if self.cmdctl:
-            del self.processes[12]
+            del self.components[12]
         self.cmdctl = False
 
 class TestStartStopProcessesBob(unittest.TestCase):
     """
-    Check that the start_all_processes method starts the right combination
-    of processes and that the right processes are started and stopped
+    Check that the start_all_components method starts the right combination
+    of components and that the right components are started and stopped
     according to changes in configuration.
     """
+    def check_environment_unchanged(self):
+        # Check whether the environment has not been changed
+        self.assertEqual(original_os_environ, os.environ)
+
     def check_started(self, bob, core, auth, resolver):
         """
         Check that the right sets of services are started. The ones that
@@ -337,6 +403,7 @@ class TestStartStopProcessesBob(unittest.TestCase):
         self.assertEqual(bob.msgq, core)
         self.assertEqual(bob.cfgmgr, core)
         self.assertEqual(bob.ccsession, core)
+        self.assertEqual(bob.creator, core)
         self.assertEqual(bob.auth, auth)
         self.assertEqual(bob.resolver, resolver)
         self.assertEqual(bob.xfrout, auth)
@@ -345,6 +412,7 @@ class TestStartStopProcessesBob(unittest.TestCase):
         self.assertEqual(bob.stats, core)
         self.assertEqual(bob.stats_httpd, core)
         self.assertEqual(bob.cmdctl, core)
+        self.check_environment_unchanged()
 
     def check_preconditions(self, bob):
         self.check_started(bob, False, False, False)
@@ -352,9 +420,10 @@ class TestStartStopProcessesBob(unittest.TestCase):
     def check_started_none(self, bob):
         """
         Check that the situation is according to configuration where no servers
-        should be started. Some processes still need to be running.
+        should be started. Some components still need to be running.
         """
         self.check_started(bob, True, False, False)
+        self.check_environment_unchanged()
 
     def check_started_both(self, bob):
         """
@@ -362,96 +431,86 @@ class TestStartStopProcessesBob(unittest.TestCase):
         (auth and resolver) are enabled.
         """
         self.check_started(bob, True, True, True)
+        self.check_environment_unchanged()
 
     def check_started_auth(self, bob):
         """
-        Check the set of processes needed to run auth only is started.
+        Check the set of components needed to run auth only is started.
         """
         self.check_started(bob, True, True, False)
+        self.check_environment_unchanged()
 
     def check_started_resolver(self, bob):
         """
-        Check the set of processes needed to run resolver only is started.
+        Check the set of components needed to run resolver only is started.
         """
         self.check_started(bob, True, False, True)
+        self.check_environment_unchanged()
 
     def check_started_dhcp(self, bob, v4, v6):
         """
         Check if proper combinations of DHCPv4 and DHCpv6 can be started
         """
-        v4found = 0
-        v6found = 0
-
-        for pid in bob.processes:
-            if (bob.processes[pid].name == "b10-dhcp4"):
-                v4found += 1
-            if (bob.processes[pid].name == "b10-dhcp6"):
-                v6found += 1
-
-        # there should be exactly one DHCPv4 daemon (if v4==True)
-        # there should be exactly one DHCPv6 daemon (if v6==True)
-        self.assertEqual(v4==True, v4found==1)
-        self.assertEqual(v6==True, v6found==1)
-
-    # Checks the processes started when starting neither auth nor resolver
-    # is specified.
-    def test_start_none(self):
-        # Create BoB and ensure correct initialization
-        bob = MockBob()
-        self.check_preconditions(bob)
-
-        # Start processes and check what was started
-        bob.cfg_start_auth = False
-        bob.cfg_start_resolver = False
-
-        bob.start_all_processes()
-        self.check_started_none(bob)
-
-    # Checks the processes started when starting only the auth process
-    def test_start_auth(self):
-        # Create BoB and ensure correct initialization
+        self.assertEqual(v4, bob.dhcp4)
+        self.assertEqual(v6, bob.dhcp6)
+        self.check_environment_unchanged()
+
+    def construct_config(self, start_auth, start_resolver):
+        # The things that are common, not turned on an off
+        config = {}
+        config['b10-stats'] = { 'kind': 'dispensable', 'address': 'Stats' }
+        config['b10-stats-httpd'] = { 'kind': 'dispensable',
+                                      'address': 'StatsHttpd' }
+        config['b10-cmdctl'] = { 'kind': 'needed', 'special': 'cmdctl' }
+        if start_auth:
+            config['b10-auth'] = { 'kind': 'needed', 'special': 'auth' }
+            config['b10-xfrout'] = { 'kind': 'dispensable',
+                                     'address': 'Xfrout' }
+            config['b10-xfrin'] = { 'kind': 'dispensable',
+                                    'address': 'Xfrin' }
+            config['b10-zonemgr'] = { 'kind': 'dispensable',
+                                      'address': 'Zonemgr' }
+        if start_resolver:
+            config['b10-resolver'] = { 'kind': 'needed',
+                                       'special': 'resolver' }
+        return {'components': config}
+
+    def config_start_init(self, start_auth, start_resolver):
+        """
+        Test the configuration is loaded at the startup.
+        """
         bob = MockBob()
-        self.check_preconditions(bob)
-
-        # Start processes and check what was started
-        bob.cfg_start_auth = True
-        bob.cfg_start_resolver = False
+        config = self.construct_config(start_auth, start_resolver)
+        class CC:
+            def get_full_config(self):
+                return config
+        # Provide the fake CC with data
+        bob.ccs = CC()
+        # And make sure it's not overwritten
+        def start_ccsession():
+            bob.ccsession = True
+        bob.start_ccsession = lambda _: start_ccsession()
+        # We need to return the original _read_bind10_config
+        bob._read_bind10_config = lambda: BoB._read_bind10_config(bob)
+        bob.start_all_components()
+        self.check_started(bob, True, start_auth, start_resolver)
+        self.check_environment_unchanged()
 
-        bob.start_all_processes()
-
-        self.check_started_auth(bob)
+    def test_start_none(self):
+        self.config_start_init(False, False)
 
-    # Checks the processes started when starting only the resolver process
     def test_start_resolver(self):
-        # Create BoB and ensure correct initialization
-        bob = MockBob()
-        self.check_preconditions(bob)
-
-        # Start processes and check what was started
-        bob.cfg_start_auth = False
-        bob.cfg_start_resolver = True
-
-        bob.start_all_processes()
+        self.config_start_init(False, True)
 
-        self.check_started_resolver(bob)
+    def test_start_auth(self):
+        self.config_start_init(True, False)
 
-    # Checks the processes started when starting both auth and resolver process
     def test_start_both(self):
-        # Create BoB and ensure correct initialization
-        bob = MockBob()
-        self.check_preconditions(bob)
-
-        # Start processes and check what was started
-        bob.cfg_start_auth = True
-        bob.cfg_start_resolver = True
-
-        bob.start_all_processes()
-
-        self.check_started_both(bob)
+        self.config_start_init(True, True)
 
     def test_config_start(self):
         """
-        Test that the configuration starts and stops processes according
+        Test that the configuration starts and stops components according
         to configuration changes.
         """
 
@@ -459,17 +518,13 @@ class TestStartStopProcessesBob(unittest.TestCase):
         bob = MockBob()
         self.check_preconditions(bob)
 
-        # Start processes (nothing much should be started, as in
-        # test_start_none)
-        bob.cfg_start_auth = False
-        bob.cfg_start_resolver = False
-
-        bob.start_all_processes()
+        bob.start_all_components()
         bob.runnable = True
+        bob.config_handler(self.construct_config(False, False))
         self.check_started_none(bob)
 
         # Enable both at once
-        bob.config_handler({'start_auth': True, 'start_resolver': True})
+        bob.config_handler(self.construct_config(True, True))
         self.check_started_both(bob)
 
         # Not touched by empty change
@@ -477,11 +532,11 @@ class TestStartStopProcessesBob(unittest.TestCase):
         self.check_started_both(bob)
 
         # Not touched by change to the same configuration
-        bob.config_handler({'start_auth': True, 'start_resolver': True})
+        bob.config_handler(self.construct_config(True, True))
         self.check_started_both(bob)
 
         # Turn them both off again
-        bob.config_handler({'start_auth': False, 'start_resolver': False})
+        bob.config_handler(self.construct_config(False, False))
         self.check_started_none(bob)
 
         # Not touched by empty change
@@ -489,47 +544,45 @@ class TestStartStopProcessesBob(unittest.TestCase):
         self.check_started_none(bob)
 
         # Not touched by change to the same configuration
-        bob.config_handler({'start_auth': False, 'start_resolver': False})
+        bob.config_handler(self.construct_config(False, False))
         self.check_started_none(bob)
 
         # Start and stop auth separately
-        bob.config_handler({'start_auth': True})
+        bob.config_handler(self.construct_config(True, False))
         self.check_started_auth(bob)
 
-        bob.config_handler({'start_auth': False})
+        bob.config_handler(self.construct_config(False, False))
         self.check_started_none(bob)
 
         # Start and stop resolver separately
-        bob.config_handler({'start_resolver': True})
+        bob.config_handler(self.construct_config(False, True))
         self.check_started_resolver(bob)
 
-        bob.config_handler({'start_resolver': False})
+        bob.config_handler(self.construct_config(False, False))
         self.check_started_none(bob)
 
         # Alternate
-        bob.config_handler({'start_auth': True})
+        bob.config_handler(self.construct_config(True, False))
         self.check_started_auth(bob)
 
-        bob.config_handler({'start_auth': False, 'start_resolver': True})
+        bob.config_handler(self.construct_config(False, True))
         self.check_started_resolver(bob)
 
-        bob.config_handler({'start_auth': True, 'start_resolver': False})
+        bob.config_handler(self.construct_config(True, False))
         self.check_started_auth(bob)
 
     def test_config_start_once(self):
         """
-        Tests that a process is started only once.
+        Tests that a component is started only once.
         """
         # Create BoB and ensure correct initialization
         bob = MockBob()
         self.check_preconditions(bob)
 
-        # Start processes (both)
-        bob.cfg_start_auth = True
-        bob.cfg_start_resolver = True
+        bob.start_all_components()
 
-        bob.start_all_processes()
         bob.runnable = True
+        bob.config_handler(self.construct_config(True, True))
         self.check_started_both(bob)
 
         bob.start_auth = lambda: self.fail("Started auth again")
@@ -539,12 +592,11 @@ class TestStartStopProcessesBob(unittest.TestCase):
         bob.start_resolver = lambda: self.fail("Started resolver again")
 
         # Send again we want to start them. Should not do it, as they are.
-        bob.config_handler({'start_auth': True})
-        bob.config_handler({'start_resolver': True})
+        bob.config_handler(self.construct_config(True, True))
 
     def test_config_not_started_early(self):
         """
-        Test that processes are not started by the config handler before
+        Test that components are not started by the config handler before
         startup.
         """
         bob = MockBob()
@@ -558,27 +610,29 @@ class TestStartStopProcessesBob(unittest.TestCase):
 
         bob.config_handler({'start_auth': True, 'start_resolver': True})
 
-    # Checks that DHCP (v4 and v6) processes are started when expected
+    # Checks that DHCP (v4 and v6) components are started when expected
     def test_start_dhcp(self):
 
         # Create BoB and ensure correct initialization
         bob = MockBob()
         self.check_preconditions(bob)
 
-        # don't care about DNS stuff
-        bob.cfg_start_auth = False
-        bob.cfg_start_resolver = False
-
-        # v4 and v6 disabled
-        bob.cfg_start_dhcp6 = False
-        bob.cfg_start_dhcp4 = False
-        bob.start_all_processes()
+        bob.start_all_components()
+        bob.config_handler(self.construct_config(False, False))
         self.check_started_dhcp(bob, False, False)
 
+    def test_start_dhcp_v6only(self):
+        # Create BoB and ensure correct initialization
+        bob = MockBob()
+        self.check_preconditions(bob)
         # v6 only enabled
-        bob.cfg_start_dhcp6 = True
-        bob.cfg_start_dhcp4 = False
-        bob.start_all_processes()
+        bob.start_all_components()
+        bob.runnable = True
+        bob._BoB_started = True
+        config = self.construct_config(False, False)
+        config['components']['b10-dhcp6'] = { 'kind': 'needed',
+                                              'address': 'Dhcp6' }
+        bob.config_handler(config)
         self.check_started_dhcp(bob, False, True)
 
         # uncomment when dhcpv4 becomes implemented
@@ -592,6 +646,12 @@ class TestStartStopProcessesBob(unittest.TestCase):
         #bob.cfg_start_dhcp4 = True
         #self.check_started_dhcp(bob, True, True)
 
+class MockComponent:
+    def __init__(self, name, pid):
+        self.name = lambda: name
+        self.pid = lambda: pid
+
+
 class TestBossCmd(unittest.TestCase):
     def test_ping(self):
         """
@@ -601,7 +661,7 @@ class TestBossCmd(unittest.TestCase):
         answer = bob.command_handler("ping", None)
         self.assertEqual(answer, {'result': [0, 'pong']})
 
-    def test_show_processes(self):
+    def test_show_processes_empty(self):
         """
         Confirm getting a list of processes works.
         """
@@ -609,23 +669,16 @@ class TestBossCmd(unittest.TestCase):
         answer = bob.command_handler("show_processes", None)
         self.assertEqual(answer, {'result': [0, []]})
 
-    def test_show_processes_started(self):
+    def test_show_processes(self):
         """
         Confirm getting a list of processes works.
         """
         bob = MockBob()
-        bob.start_all_processes()
+        bob.register_process(1, MockComponent('first', 1))
+        bob.register_process(2, MockComponent('second', 2))
         answer = bob.command_handler("show_processes", None)
-        processes = [[2, 'b10-msgq'],
-                     [3, 'b10-cfgmgr'], 
-                     [4, 'b10-ccsession'],
-                     [5, 'b10-auth'],
-                     [7, 'b10-xfrout'],
-                     [8, 'b10-xfrin'], 
-                     [9, 'b10-zonemgr'],
-                     [10, 'b10-stats'], 
-                     [11, 'b10-stats-httpd'], 
-                     [12, 'b10-cmdctl']]
+        processes = [[1, 'first'],
+                     [2, 'second']]
         self.assertEqual(answer, {'result': [0, processes]})
 
 class TestParseArgs(unittest.TestCase):
@@ -679,15 +732,6 @@ class TestParseArgs(unittest.TestCase):
         options = parse_args(['--cmdctl-port=1234'], TestOptParser)
         self.assertEqual(1234, options.cmdctl_port)
 
-    def test_brittle(self):
-        """
-        Test we can use the "brittle" flag.
-        """
-        options = parse_args([], TestOptParser)
-        self.assertFalse(options.brittle)
-        options = parse_args(['--brittle'], TestOptParser)
-        self.assertTrue(options.brittle)
-
 class TestPIDFile(unittest.TestCase):
     def setUp(self):
         self.pid_file = '@builddir@' + os.sep + 'bind10.pid'
@@ -735,35 +779,160 @@ class TestPIDFile(unittest.TestCase):
         self.assertRaises(IOError, dump_pid,
                           'nonexistent_dir' + os.sep + 'bind10.pid')
 
-class TestBrittle(unittest.TestCase):
-    def test_brittle_disabled(self):
-        bob = MockBob()
-        bob.start_all_processes()
-        bob.runnable = True
+class TestBossComponents(unittest.TestCase):
+    """
+    Test the boss propagates component configuration properly to the
+    component configurator and acts sane.
+    """
+    def setUp(self):
+        self.__param = None
+        self.__called = False
+        self.__compconfig = {
+            'comp': {
+                'kind': 'needed',
+                'process': 'cat'
+            }
+        }
 
-        bob.reap_children()
-        self.assertTrue(bob.runnable)
+    def __unary_hook(self, param):
+        """
+        A hook function that stores the parameter for later examination.
+        """
+        self.__param = param
 
-    def simulated_exit(self):
-        ret_val = self.exit_info
-        self.exit_info = (0, 0)
-        return ret_val
+    def __nullary_hook(self):
+        """
+        A hook function that notes down it was called.
+        """
+        self.__called = True
 
-    def test_brittle_enabled(self):
+    def __check_core(self, config):
+        """
+        A function checking that the config contains parts for the valid
+        core component configuration.
+        """
+        self.assertIsNotNone(config)
+        for component in ['sockcreator', 'msgq', 'cfgmgr']:
+            self.assertTrue(component in config)
+            self.assertEqual(component, config[component]['special'])
+            self.assertEqual('core', config[component]['kind'])
+
+    def __check_extended(self, config):
+        """
+        This checks that the config contains the core and one more component.
+        """
+        self.__check_core(config)
+        self.assertTrue('comp' in config)
+        self.assertEqual('cat', config['comp']['process'])
+        self.assertEqual('needed', config['comp']['kind'])
+        self.assertEqual(4, len(config))
+
+    def test_correct_run(self):
+        """
+        Test the situation when we run in usual scenario, nothing fails,
+        we just start, reconfigure and then stop peacefully.
+        """
         bob = MockBob()
-        bob.start_all_processes()
+        # Start it
+        orig = bob._component_configurator.startup
+        bob._component_configurator.startup = self.__unary_hook
+        bob.start_all_components()
+        bob._component_configurator.startup = orig
+        self.__check_core(self.__param)
+        self.assertEqual(3, len(self.__param))
+
+        # Reconfigure it
+        self.__param = None
+        orig = bob._component_configurator.reconfigure
+        bob._component_configurator.reconfigure = self.__unary_hook
+        # Otherwise it does not work
         bob.runnable = True
+        bob.config_handler({'components': self.__compconfig})
+        self.__check_extended(self.__param)
+        currconfig = self.__param
+        # If we reconfigure it, but it does not contain the components part,
+        # nothing is called
+        bob.config_handler({})
+        self.assertEqual(self.__param, currconfig)
+        self.__param = None
+        bob._component_configurator.reconfigure = orig
+        # Check a configuration that messes up the core components is rejected.
+        compconf = dict(self.__compconfig)
+        compconf['msgq'] = { 'process': 'echo' }
+        result = bob.config_handler({'components': compconf})
+        # Check it rejected it
+        self.assertEqual(1, result['result'][0])
 
-        bob.brittle = True
-        self.exit_info = (5, 0)
-        bob._get_process_exit_status = self.simulated_exit
+        # We can't call shutdown, that one relies on the stuff in main
+        # We check somewhere else that the shutdown is actually called
+        # from there (the test_kills).
 
-        old_stdout = sys.stdout
-        sys.stdout = open("/dev/null", "w")
-        bob.reap_children()
-        sys.stdout = old_stdout
+    def test_kills(self):
+        """
+        Test that the boss kills components which don't want to stop.
+        """
+        bob = MockBob()
+        killed = []
+        class ImmortalComponent:
+            """
+            An immortal component. It does not stop when it is told so
+            (anyway it is not told so). It does not die if it is killed
+            the first time. It dies only when killed forcefully.
+            """
+            def kill(self, forceful=False):
+                killed.append(forceful)
+                if forceful:
+                    bob.components = {}
+            def pid(self):
+                return 1
+            def name(self):
+                return "Immortal"
+        bob.components = {}
+        bob.register_process(1, ImmortalComponent())
+
+        # While at it, we check the configurator shutdown is actually called
+        orig = bob._component_configurator.shutdown
+        bob._component_configurator.shutdown = self.__nullary_hook
+        self.__called = False
+
+        bob.shutdown()
+
+        self.assertEqual([False, True], killed)
+        self.assertTrue(self.__called)
+
+        bob._component_configurator.shutdown = orig
+
+    def test_component_shutdown(self):
+        """
+        Test the component_shutdown sets all variables accordingly.
+        """
+        bob = MockBob()
+        self.assertRaises(Exception, bob.component_shutdown, 1)
+        self.assertEqual(1, bob.exitcode)
+        bob._BoB__started = True
+        bob.component_shutdown(2)
+        self.assertEqual(2, bob.exitcode)
         self.assertFalse(bob.runnable)
 
+    def test_init_config(self):
+        """
+        Test initial configuration is loaded.
+        """
+        bob = MockBob()
+        # Start it
+        bob._component_configurator.reconfigure = self.__unary_hook
+        # We need to return the original read_bind10_config
+        bob._read_bind10_config = lambda: BoB._read_bind10_config(bob)
+        # And provide a session to read the data from
+        class CC:
+            pass
+        bob.ccs = CC()
+        bob.ccs.get_full_config = lambda: {'components': self.__compconfig}
+        bob.start_all_components()
+        self.__check_extended(self.__param)
+
 if __name__ == '__main__':
+    # store os.environ for test_unchanged_environment
+    original_os_environ = copy.deepcopy(os.environ)
     isc.log.resetUnitTestRootLogger()
     unittest.main()

src/bin/bindctl/Makefile.am

@@ -5,6 +5,8 @@ man_MANS = bindctl.1
 
 EXTRA_DIST = $(man_MANS) bindctl.xml
 
+noinst_SCRIPTS = run_bindctl.sh
+
 python_PYTHON = __init__.py bindcmd.py cmdparse.py exception.py moduleinfo.py \
 		mycollections.py
 pythondir = $(pyexecdir)/bindctl

src/bin/bindctl/bindcmd.py

@@ -46,6 +46,16 @@ except ImportError:
 # if we have readline support, use that, otherwise use normal stdio
 try:
     import readline
+    # This is a fix for the problem described in
+    # http://bind10.isc.org/ticket/1345
+    # If '-' is seen as a word-boundary, the final completion-step
+    # (as handled by the cmd module, and hence outside our reach) can
+    # mistakenly add data twice, resulting in wrong completion results
+    # The solution is to remove it.
+    delims = readline.get_completer_delims()
+    delims = delims.replace('-', '')
+    readline.set_completer_delims(delims)
+
     my_readline = readline.get_line_buffer
 except ImportError:
     my_readline = sys.stdin.readline
@@ -61,21 +71,21 @@ Type \"<module_name> <command_name> help\" for help on the specific command.
 \nAvailable module names: """
 
 class ValidatedHTTPSConnection(http.client.HTTPSConnection):
-    '''Overrides HTTPSConnection to support certification 
+    '''Overrides HTTPSConnection to support certification
     validation. '''
     def __init__(self, host, ca_certs):
         http.client.HTTPSConnection.__init__(self, host)
         self.ca_certs = ca_certs
 
     def connect(self):
-        ''' Overrides the connect() so that we do 
+        ''' Overrides the connect() so that we do
         certificate validation. '''
         sock = socket.create_connection((self.host, self.port),
                                         self.timeout)
         if self._tunnel_host:
             self.sock = sock
             self._tunnel()
-       
+
         req_cert = ssl.CERT_NONE
         if self.ca_certs:
             req_cert = ssl.CERT_REQUIRED
@@ -85,7 +95,7 @@ class ValidatedHTTPSConnection(http.client.HTTPSConnection):
                                     ca_certs=self.ca_certs)
 
 class BindCmdInterpreter(Cmd):
-    """simple bindctl example."""    
+    """simple bindctl example."""
 
     def __init__(self, server_port='localhost:8080', pem_file=None,
                  csv_file_dir=None):
@@ -118,29 +128,33 @@ class BindCmdInterpreter(Cmd):
                                       socket.gethostname())).encode())
         digest = session_id.hexdigest()
         return digest
-    
+
     def run(self):
         '''Parse commands from user and send them to cmdctl. '''
         try:
             if not self.login_to_cmdctl():
-                return
+                return 1
 
             self.cmdloop()
             print('\nExit from bindctl')
+            return 0
         except FailToLogin as err:
             # error already printed when this was raised, ignoring
-            pass
+            return 1
         except KeyboardInterrupt:
             print('\nExit from bindctl')
+            return 0
         except socket.error as err:
             print('Failed to send request, the connection is closed')
+            return 1
         except http.client.CannotSendRequest:
             print('Can not send request, the connection is busy')
+            return 1
 
     def _get_saved_user_info(self, dir, file_name):
-        ''' Read all the available username and password pairs saved in 
+        ''' Read all the available username and password pairs saved in
         file(path is "dir + file_name"), Return value is one list of elements
-        ['name', 'password'], If get information failed, empty list will be 
+        ['name', 'password'], If get information failed, empty list will be
         returned.'''
         if (not dir) or (not os.path.exists(dir)):
             return []
@@ -166,7 +180,7 @@ class BindCmdInterpreter(Cmd):
             if not os.path.exists(dir):
                 os.mkdir(dir, 0o700)
 
-            csvfilepath = dir + file_name 
+            csvfilepath = dir + file_name
             csvfile = open(csvfilepath, 'w')
             os.chmod(csvfilepath, 0o600)
             writer = csv.writer(csvfile)
@@ -180,7 +194,7 @@ class BindCmdInterpreter(Cmd):
         return True
 
     def login_to_cmdctl(self):
-        '''Login to cmdctl with the username and password inputted 
+        '''Login to cmdctl with the username and password inputted
         from user. After the login is sucessful, the username and
         password will be saved in 'default_user.csv', when run the next
         time, username and password saved in 'default_user.csv' will be
@@ -246,14 +260,14 @@ class BindCmdInterpreter(Cmd):
             if self.login_to_cmdctl():
                 # successful, so try send again
                 status, reply_msg = self._send_message(url, body)
-            
+
         if reply_msg:
             return json.loads(reply_msg.decode())
         else:
             return {}
-       
 
-    def send_POST(self, url, post_param = None): 
+
+    def send_POST(self, url, post_param = None):
         '''Send POST request to cmdctl, session id is send with the name
         'cookie' in header.
         Format: /module_name/command_name
@@ -312,12 +326,12 @@ class BindCmdInterpreter(Cmd):
     def _validate_cmd(self, cmd):
         '''validate the parameters and merge some parameters together,
         merge algorithm is based on the command line syntax, later, if
-        a better command line syntax come out, this function should be 
-        updated first.        
+        a better command line syntax come out, this function should be
+        updated first.
         '''
         if not cmd.module in self.modules:
             raise CmdUnknownModuleSyntaxError(cmd.module)
-        
+
         module_info = self.modules[cmd.module]
         if not module_info.has_command_with_name(cmd.command):
             raise CmdUnknownCmdSyntaxError(cmd.module, cmd.command)
@@ -325,17 +339,17 @@ class BindCmdInterpreter(Cmd):
         command_info = module_info.get_command_with_name(cmd.command)
         manda_params = command_info.get_mandatory_param_names()
         all_params = command_info.get_param_names()
-        
+
         # If help is entered, don't do further parameter validation.
         for val in cmd.params.keys():
             if val == "help":
                 return
-        
-        params = cmd.params.copy()       
-        if not params and manda_params:            
-            raise CmdMissParamSyntaxError(cmd.module, cmd.command, manda_params[0])            
+
+        params = cmd.params.copy()
+        if not params and manda_params:
+            raise CmdMissParamSyntaxError(cmd.module, cmd.command, manda_params[0])
         elif params and not all_params:
-            raise CmdUnknownParamSyntaxError(cmd.module, cmd.command, 
+            raise CmdUnknownParamSyntaxError(cmd.module, cmd.command,
                                              list(params.keys())[0])
         elif params:
             param_name = None
@@ -366,7 +380,7 @@ class BindCmdInterpreter(Cmd):
                         param_name = command_info.get_param_name_by_position(name, param_count)
                         cmd.params[param_name] = cmd.params[name]
                         del cmd.params[name]
-                        
+
                 elif not name in all_params:
                     raise CmdUnknownParamSyntaxError(cmd.module, cmd.command, name)
 
@@ -375,7 +389,7 @@ class BindCmdInterpreter(Cmd):
                 if not name in params and not param_nr in params:
                     raise CmdMissParamSyntaxError(cmd.module, cmd.command, name)
                 param_nr += 1
-        
+
         # Convert parameter value according parameter spec file.
         # Ignore check for commands belongs to module 'config'
         if cmd.module != CONFIG_MODULE_NAME:
@@ -384,9 +398,9 @@ class BindCmdInterpreter(Cmd):
                 try:
                     cmd.params[param_name] = isc.config.config_data.convert_type(param_spec, cmd.params[param_name])
                 except isc.cc.data.DataTypeError as e:
-                    raise isc.cc.data.DataTypeError('Invalid parameter value for \"%s\", the type should be \"%s\" \n' 
+                    raise isc.cc.data.DataTypeError('Invalid parameter value for \"%s\", the type should be \"%s\" \n'
                                                      % (param_name, param_spec['item_type']) + str(e))
-    
+
     def _handle_cmd(self, cmd):
         '''Handle a command entered by the user'''
         if cmd.command == "help" or ("help" in cmd.params.keys()):
@@ -398,6 +412,8 @@ class BindCmdInterpreter(Cmd):
                 print("Error: " + str(dte))
             except isc.cc.data.DataNotFoundError as dnfe:
                 print("Error: " + str(dnfe))
+            except isc.cc.data.DataAlreadyPresentError as dape:
+                print("Error: " + str(dape))
             except KeyError as ke:
                 print("Error: missing " + str(ke))
         else:
@@ -406,7 +422,7 @@ class BindCmdInterpreter(Cmd):
     def add_module_info(self, module_info):
         '''Add the information about one module'''
         self.modules[module_info.name] = module_info
-        
+
     def get_module_names(self):
         '''Return the names of all known modules'''
         return list(self.modules.keys())
@@ -438,15 +454,15 @@ class BindCmdInterpreter(Cmd):
                     subsequent_indent="    " +
                     " " * CONST_BINDCTL_HELP_INDENT_WIDTH,
                     width=70))
-    
+
     def onecmd(self, line):
         if line == 'EOF' or line.lower() == "quit":
             self.conn.close()
             return True
-            
+
         if line == 'h':
             line = 'help'
-        
+
         Cmd.onecmd(self, line)
 
     def remove_prefix(self, list, prefix):
@@ -474,7 +490,7 @@ class BindCmdInterpreter(Cmd):
                 cmd = BindCmdParse(cur_line)
                 if not cmd.params and text:
                     hints = self._get_command_startswith(cmd.module, text)
-                else:                       
+                else:
                     hints = self._get_param_startswith(cmd.module, cmd.command,
                                                        text)
                     if cmd.module == CONFIG_MODULE_NAME:
@@ -490,8 +506,8 @@ class BindCmdInterpreter(Cmd):
 
             except CmdMissCommandNameFormatError as e:
                 if not text.strip(): # command name is empty
-                    hints = self.modules[e.module].get_command_names()                    
-                else: 
+                    hints = self.modules[e.module].get_command_names()
+                else:
                     hints = self._get_module_startswith(text)
 
             except CmdCommandNameFormatError as e:
@@ -505,44 +521,43 @@ class BindCmdInterpreter(Cmd):
                 hints = []
 
             self.hint = hints
-            #self._append_space_to_hint()
 
         if state < len(self.hint):
             return self.hint[state]
         else:
             return None
-            
 
-    def _get_module_startswith(self, text):       
+
+    def _get_module_startswith(self, text):
         return [module
-                for module in self.modules 
+                for module in self.modules
                 if module.startswith(text)]
 
 
     def _get_command_startswith(self, module, text):
-        if module in self.modules:            
+        if module in self.modules:
             return [command
-                    for command in self.modules[module].get_command_names() 
+                    for command in self.modules[module].get_command_names()
                     if command.startswith(text)]
-        
-        return []                    
-                        
 
-    def _get_param_startswith(self, module, command, text):        
+        return []
+
+
+    def _get_param_startswith(self, module, command, text):
         if module in self.modules:
-            module_info = self.modules[module]            
-            if command in module_info.get_command_names():                
+            module_info = self.modules[module]
+            if command in module_info.get_command_names():
                 cmd_info = module_info.get_command_with_name(command)
-                params = cmd_info.get_param_names() 
+                params = cmd_info.get_param_names()
                 hint = []
-                if text:    
+                if text:
                     hint = [val for val in params if val.startswith(text)]
                 else:
                     hint = list(params)
-                
+
                 if len(hint) == 1 and hint[0] != "help":
-                    hint[0] = hint[0] + " ="    
-                
+                    hint[0] = hint[0] + " ="
+
                 return hint
 
         return []
@@ -559,24 +574,24 @@ class BindCmdInterpreter(Cmd):
             self._print_correct_usage(err)
         except isc.cc.data.DataTypeError as err:
             print("Error! ", err)
-            
-    def _print_correct_usage(self, ept):        
+
+    def _print_correct_usage(self, ept):
         if isinstance(ept, CmdUnknownModuleSyntaxError):
             self.do_help(None)
-            
+
         elif isinstance(ept, CmdUnknownCmdSyntaxError):
             self.modules[ept.module].module_help()
-            
+
         elif isinstance(ept, CmdMissParamSyntaxError) or \
              isinstance(ept, CmdUnknownParamSyntaxError):
              self.modules[ept.module].command_help(ept.command)
-                 
-                
+
+
     def _append_space_to_hint(self):
         """Append one space at the end of complete hint."""
         self.hint = [(val + " ") for val in self.hint]
-            
-            
+
+
     def _handle_help(self, cmd):
         if cmd.command == "help":
             self.modules[cmd.module].module_help()
@@ -634,7 +649,15 @@ class BindCmdInterpreter(Cmd):
                     # we have more data to show
                     line += "/"
                 else:
-                    line += "\t" + json.dumps(value_map['value'])
+                    # if type is named_set, don't print value if None
+                    # (it is either {} meaning empty, or None, meaning
+                    # there actually is data, but not to be shown with
+                    # the current command
+                    if value_map['type'] == 'named_set' and\
+                       value_map['value'] is None:
+                        line += "/\t"
+                    else:
+                        line += "\t" + json.dumps(value_map['value'])
                 line += "\t" + value_map['type']
                 line += "\t"
                 if value_map['default']:
@@ -649,10 +672,9 @@ class BindCmdInterpreter(Cmd):
                 data, default = self.config_data.get_value(identifier)
                 print(json.dumps(data))
         elif cmd.command == "add":
-            if 'value' in cmd.params:
-                self.config_data.add_value(identifier, cmd.params['value'])
-            else:
-                self.config_data.add_value(identifier)
+            self.config_data.add_value(identifier,
+                                       cmd.params.get('value_or_name'),
+                                       cmd.params.get('value_for_set'))
         elif cmd.command == "remove":
             if 'value' in cmd.params:
                 self.config_data.remove_value(identifier, cmd.params['value'])
@@ -679,7 +701,7 @@ class BindCmdInterpreter(Cmd):
             except isc.config.ModuleCCSessionError as mcse:
                 print(str(mcse))
         elif cmd.command == "diff":
-            print(self.config_data.get_local_changes());
+            print(self.config_data.get_local_changes())
         elif cmd.command == "go":
             self.go(identifier)
 

src/bin/bindctl/bindctl_main.py.in

@@ -50,17 +50,28 @@ def prepare_config_commands(tool):
     cmd.add_param(param)
     module.add_command(cmd)
 
-    cmd = CommandInfo(name = "add", desc = "Add an entry to configuration list. If no value is given, a default value is added.")
+    cmd = CommandInfo(name = "add", desc =
+        "Add an entry to configuration list or a named set. "
+        "When adding to a list, the command has one optional argument, "
+        "a value to add to the list. The value must be in correct JSON "
+        "and complete. When adding to a named set, it has one "
+        "mandatory parameter (the name to add), and an optional "
+        "parameter value, similar to when adding to a list. "
+        "In either case, when no value is given, an entry will be "
+        "constructed with default values.")
     param = ParamInfo(name = "identifier", type = "string", optional=True, desc = DEFAULT_IDENTIFIER_DESC)
     cmd.add_param(param)
-    param = ParamInfo(name = "value", type = "string", optional=True, desc = "Specifies a value to add to the list. It must be in correct JSON format and complete.")
+    param = ParamInfo(name = "value_or_name", type = "string", optional=True, desc = "Specifies a value to add to the list, or the name when adding to a named set. It must be in correct JSON format and complete.")
+    cmd.add_param(param)
+    module.add_command(cmd)
+    param = ParamInfo(name = "value_for_set", type = "string", optional=True, desc = "Specifies an optional value to add to the named map. It must be in correct JSON format and complete.")
     cmd.add_param(param)
     module.add_command(cmd)
 
-    cmd = CommandInfo(name = "remove", desc = "Remove entry from configuration list.")
+    cmd = CommandInfo(name = "remove", desc = "Remove entry from configuration list or named set.")
     param = ParamInfo(name = "identifier", type = "string", optional=True, desc = DEFAULT_IDENTIFIER_DESC)
     cmd.add_param(param)
-    param = ParamInfo(name = "value", type = "string", optional=True, desc = "Specifies a value to remove from the list. It must be in correct JSON format and complete.")
+    param = ParamInfo(name = "value", type = "string", optional=True, desc = "When identifier is a list, specifies a value to remove from the list. It must be in correct JSON format and complete. When it is a named set, specifies the name to remove.")
     cmd.add_param(param)
     module.add_command(cmd)
 
@@ -135,4 +146,5 @@ if __name__ == '__main__':
     tool = BindCmdInterpreter(server_addr, pem_file=options.cert_chain,
                               csv_file_dir=options.csv_file_dir)
     prepare_config_commands(tool)
-    tool.run()
+    result = tool.run()
+    sys.exit(result)

src/bin/bindctl/run_bindctl.sh.in

@@ -20,14 +20,14 @@ export PYTHON_EXEC
 
 BINDCTL_PATH=@abs_top_builddir@/src/bin/bindctl
 
-PYTHONPATH=@abs_top_srcdir@/src/bin:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/bin:@abs_top_srcdir@/src/lib/python
+PYTHONPATH=@abs_top_srcdir@/src/bin:@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/bin:@abs_top_srcdir@/src/lib/python
 export PYTHONPATH
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 SET_ENV_LIBRARY_PATH=@SET_ENV_LIBRARY_PATH@
 if test $SET_ENV_LIBRARY_PATH = yes; then
-	@ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:$@ENV_LIBRARY_PATH@
+	@ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:@abs_top_builddir@/src/lib/datasrc/.libs:$@ENV_LIBRARY_PATH@
 	export @ENV_LIBRARY_PATH@
 fi
 

src/bin/bindctl/tests/Makefile.am

@@ -6,7 +6,7 @@ EXTRA_DIST = $(PYTESTS)
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
@@ -19,6 +19,6 @@ endif
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
 	$(LIBRARY_PATH_PLACEHOLDER) \
-	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/bindctl:$(abs_top_srcdir)/src/bin  \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/bindctl:$(abs_top_srcdir)/src/bin  \
 	$(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \
 	done

src/bin/bindctl/tests/bindctl_test.py

@@ -31,14 +31,14 @@ from bindctl_main import set_bindctl_options
 from bindctl import cmdparse
 from bindctl import bindcmd
 from bindctl.moduleinfo import *
-from bindctl.exception import *    
+from bindctl.exception import *
 try:
     from collections import OrderedDict
 except ImportError:
     from mycollections import OrderedDict
 
 class TestCmdLex(unittest.TestCase):
-    
+
     def my_assert_raise(self, exception_type, cmd_line):
         self.assertRaises(exception_type, cmdparse.BindCmdParse, cmd_line)
 
@@ -48,13 +48,13 @@ class TestCmdLex(unittest.TestCase):
         assert cmd.module == "zone"
         assert cmd.command == "add"
         self.assertEqual(len(cmd.params), 0)
-        
-    
+
+
     def testCommandWithParameters(self):
         lines = {"zone add zone_name = cnnic.cn, file = cnnic.cn.file master=1.1.1.1",
                  "zone add zone_name = \"cnnic.cn\", file ='cnnic.cn.file' master=1.1.1.1  ",
                  "zone add zone_name = 'cnnic.cn\", file ='cnnic.cn.file' master=1.1.1.1, " }
-        
+
         for cmd_line in lines:
             cmd = cmdparse.BindCmdParse(cmd_line)
             assert cmd.module == "zone"
@@ -75,7 +75,7 @@ class TestCmdLex(unittest.TestCase):
         cmd = cmdparse.BindCmdParse('zone cmd name = 1\"\'34**&2 ,value=  44\"\'\"')
         self.assertEqual(cmd.params['name'], '1\"\'34**&2')
         self.assertEqual(cmd.params['value'], '44\"\'\"')
-            
+
         cmd = cmdparse.BindCmdParse('zone cmd name =  1\'34**&2value=44\"\'\" value = \"==============\'')
         self.assertEqual(cmd.params['name'], '1\'34**&2value=44\"\'\"')
         self.assertEqual(cmd.params['value'], '==============')
@@ -83,34 +83,34 @@ class TestCmdLex(unittest.TestCase):
         cmd = cmdparse.BindCmdParse('zone cmd name =    \"1234, 567890 \" value ==&*/')
         self.assertEqual(cmd.params['name'], '1234, 567890 ')
         self.assertEqual(cmd.params['value'], '=&*/')
-            
+
     def testCommandWithListParam(self):
         cmd = cmdparse.BindCmdParse("zone set zone_name='cnnic.cn', master='1.1.1.1, 2.2.2.2'")
-        assert cmd.params["master"] == '1.1.1.1, 2.2.2.2'            
-        
+        assert cmd.params["master"] == '1.1.1.1, 2.2.2.2'
+
     def testCommandWithHelpParam(self):
         cmd = cmdparse.BindCmdParse("zone add help")
         assert cmd.params["help"] == "help"
-        
+
         cmd = cmdparse.BindCmdParse("zone add help *&)&)*&&$#$^%")
         assert cmd.params["help"] == "help"
         self.assertEqual(len(cmd.params), 1)
-        
+
 
     def testCmdModuleNameFormatError(self):
         self.my_assert_raise(CmdModuleNameFormatError, "zone=good")
-        self.my_assert_raise(CmdModuleNameFormatError, "zo/ne")        
-        self.my_assert_raise(CmdModuleNameFormatError, "")        
+        self.my_assert_raise(CmdModuleNameFormatError, "zo/ne")
+        self.my_assert_raise(CmdModuleNameFormatError, "")
         self.my_assert_raise(CmdModuleNameFormatError, "=zone")
-        self.my_assert_raise(CmdModuleNameFormatError, "zone,")        
-        
-        
+        self.my_assert_raise(CmdModuleNameFormatError, "zone,")
+
+
     def testCmdMissCommandNameFormatError(self):
         self.my_assert_raise(CmdMissCommandNameFormatError, "zone")
         self.my_assert_raise(CmdMissCommandNameFormatError, "zone ")
         self.my_assert_raise(CmdMissCommandNameFormatError, "help ")
-        
-             
+
+
     def testCmdCommandNameFormatError(self):
         self.my_assert_raise(CmdCommandNameFormatError, "zone =d")
         self.my_assert_raise(CmdCommandNameFormatError, "zone z=d")
@@ -119,11 +119,11 @@ class TestCmdLex(unittest.TestCase):
         self.my_assert_raise(CmdCommandNameFormatError, "zone zdd/ \"")
 
 class TestCmdSyntax(unittest.TestCase):
-    
+
     def _create_bindcmd(self):
         """Create one bindcmd"""
-        
-        tool = bindcmd.BindCmdInterpreter()        
+
+        tool = bindcmd.BindCmdInterpreter()
         string_spec = { 'item_type' : 'string',
                        'item_optional' : False,
                        'item_default' : ''}
@@ -135,40 +135,40 @@ class TestCmdSyntax(unittest.TestCase):
         load_cmd = CommandInfo(name = "load")
         load_cmd.add_param(zone_file_param)
         load_cmd.add_param(zone_name)
-        
-        param_master = ParamInfo(name = "master", optional = True, param_spec = string_spec)                                 
-        param_master = ParamInfo(name = "port", optional = True, param_spec = int_spec)                                 
-        param_allow_update = ParamInfo(name = "allow_update", optional = True, param_spec = string_spec)                                           
+
+        param_master = ParamInfo(name = "master", optional = True, param_spec = string_spec)
+        param_master = ParamInfo(name = "port", optional = True, param_spec = int_spec)
+        param_allow_update = ParamInfo(name = "allow_update", optional = True, param_spec = string_spec)
         set_cmd = CommandInfo(name = "set")
         set_cmd.add_param(param_master)
         set_cmd.add_param(param_allow_update)
         set_cmd.add_param(zone_name)
-        
-        reload_all_cmd = CommandInfo(name = "reload_all")        
-        
-        zone_module = ModuleInfo(name = "zone")                             
+
+        reload_all_cmd = CommandInfo(name = "reload_all")
+
+        zone_module = ModuleInfo(name = "zone")
         zone_module.add_command(load_cmd)
         zone_module.add_command(set_cmd)
         zone_module.add_command(reload_all_cmd)
-        
+
         tool.add_module_info(zone_module)
         return tool
-        
-        
+
+
     def setUp(self):
         self.bindcmd = self._create_bindcmd()
-        
-        
+
+
     def no_assert_raise(self, cmd_line):
         cmd = cmdparse.BindCmdParse(cmd_line)
-        self.bindcmd._validate_cmd(cmd) 
-        
-        
+        self.bindcmd._validate_cmd(cmd)
+
+
     def my_assert_raise(self, exception_type, cmd_line):
         cmd = cmdparse.BindCmdParse(cmd_line)
-        self.assertRaises(exception_type, self.bindcmd._validate_cmd, cmd)  
-        
-        
+        self.assertRaises(exception_type, self.bindcmd._validate_cmd, cmd)
+
+
     def testValidateSuccess(self):
         self.no_assert_raise("zone load zone_file='cn' zone_name='cn'")
         self.no_assert_raise("zone load zone_file='cn', zone_name='cn', ")
@@ -178,27 +178,27 @@ class TestCmdSyntax(unittest.TestCase):
         self.no_assert_raise("zone set allow_update='1.1.1.1' zone_name='cn'")
         self.no_assert_raise("zone set zone_name='cn'")
         self.my_assert_raise(isc.cc.data.DataTypeError, "zone set zone_name ='cn', port='cn'")
-        self.no_assert_raise("zone reload_all")        
-        
-    
+        self.no_assert_raise("zone reload_all")
+
+
     def testCmdUnknownModuleSyntaxError(self):
         self.my_assert_raise(CmdUnknownModuleSyntaxError, "zoned d")
         self.my_assert_raise(CmdUnknownModuleSyntaxError, "dd dd  ")
-        
-              
+
+
     def testCmdUnknownCmdSyntaxError(self):
         self.my_assert_raise(CmdUnknownCmdSyntaxError, "zone dd")
-        
+
     def testCmdMissParamSyntaxError(self):
         self.my_assert_raise(CmdMissParamSyntaxError, "zone load zone_file='cn'")
         self.my_assert_raise(CmdMissParamSyntaxError, "zone load zone_name='cn'")
         self.my_assert_raise(CmdMissParamSyntaxError, "zone set allow_update='1.1.1.1'")
         self.my_assert_raise(CmdMissParamSyntaxError, "zone set ")
-        
+
     def testCmdUnknownParamSyntaxError(self):
         self.my_assert_raise(CmdUnknownParamSyntaxError, "zone load zone_d='cn'")
-        self.my_assert_raise(CmdUnknownParamSyntaxError, "zone reload_all zone_name = 'cn'")  
-       
+        self.my_assert_raise(CmdUnknownParamSyntaxError, "zone reload_all zone_name = 'cn'")
+
 class TestModuleInfo(unittest.TestCase):
 
     def test_get_param_name_by_position(self):
@@ -212,36 +212,36 @@ class TestModuleInfo(unittest.TestCase):
         self.assertEqual('sex', cmd.get_param_name_by_position(2, 3))
         self.assertEqual('data', cmd.get_param_name_by_position(2, 4))
         self.assertEqual('data', cmd.get_param_name_by_position(2, 4))
-        
+
         self.assertRaises(KeyError, cmd.get_param_name_by_position, 4, 4)
 
 
-    
+
 class TestNameSequence(unittest.TestCase):
     """
     Test if the module/command/parameters is saved in the order creation
     """
-    
+
     def _create_bindcmd(self):
-        """Create one bindcmd"""     
-        
+        """Create one bindcmd"""
+
         self._cmd = CommandInfo(name = "load")
         self.module = ModuleInfo(name = "zone")
-        self.tool = bindcmd.BindCmdInterpreter()        
+        self.tool = bindcmd.BindCmdInterpreter()
         for random_str in self.random_names:
             self._cmd.add_param(ParamInfo(name = random_str))
             self.module.add_command(CommandInfo(name = random_str))
-            self.tool.add_module_info(ModuleInfo(name = random_str))  
-        
+            self.tool.add_module_info(ModuleInfo(name = random_str))
+
     def setUp(self):
         self.random_names = ['1erdfeDDWsd', '3fe', '2009erd', 'Fe231', 'tere142', 'rei8WD']
         self._create_bindcmd()
-        
-    def testSequence(self):        
+
+    def testSequence(self):
         param_names = self._cmd.get_param_names()
         cmd_names = self.module.get_command_names()
         module_names = self.tool.get_module_names()
-        
+
         i = 0
         while i < len(self.random_names):
             assert self.random_names[i] == param_names[i+1]
@@ -342,7 +342,7 @@ class TestConfigCommands(unittest.TestCase):
         # validate log message for socket.err
         socket_err_output = io.StringIO()
         sys.stdout = socket_err_output
-        self.assertRaises(None, self.tool.run())
+        self.assertEqual(1, self.tool.run())
         self.assertEqual("Failed to send request, the connection is closed\n",
                          socket_err_output.getvalue())
         socket_err_output.close()
@@ -350,7 +350,7 @@ class TestConfigCommands(unittest.TestCase):
         # validate log message for http.client.CannotSendRequest
         cannot_send_output = io.StringIO()
         sys.stdout = cannot_send_output
-        self.assertRaises(None, self.tool.run())
+        self.assertEqual(1, self.tool.run())
         self.assertEqual("Can not send request, the connection is busy\n",
                          cannot_send_output.getvalue())
         cannot_send_output.close()
@@ -472,4 +472,4 @@ class TestCommandLineOptions(unittest.TestCase):
 
 if __name__== "__main__":
     unittest.main()
-    
+

src/bin/cfgmgr/b10-cfgmgr.py.in

@@ -28,7 +28,7 @@ import os.path
 import isc.log
 isc.log.init("b10-cfgmgr")
 from isc.config.cfgmgr import ConfigManager, ConfigManagerDataReadError, logger
-from cfgmgr_messages import *
+from isc.log_messages.cfgmgr_messages import *
 
 isc.util.process.rename()
 

src/bin/cfgmgr/plugins/Makefile.am

@@ -1,11 +1,14 @@
 SUBDIRS = tests
-EXTRA_DIST = README tsig_keys.py tsig_keys.spec
-EXTRA_DIST += logging.spec b10logging.py
+
+EXTRA_DIST = README logging.spec tsig_keys.spec
 
 config_plugindir = @prefix@/share/@PACKAGE@/config_plugins
-config_plugin_DATA = tsig_keys.py tsig_keys.spec
-config_plugin_DATA += b10logging.py logging.spec
+config_plugin_DATA = logging.spec tsig_keys.spec
+
+python_PYTHON = b10logging.py tsig_keys.py
+pythondir = $(config_plugindir)
 
+CLEANFILES = b10logging.pyc tsig_keys.pyc
 CLEANDIRS = __pycache__
 
 clean-local:

src/bin/cfgmgr/plugins/tests/Makefile.am

@@ -7,7 +7,7 @@ EXTRA_DIST = $(PYTESTS)
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
@@ -19,8 +19,8 @@ if ENABLE_PYTHON_COVERAGE
 endif
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
-	env B10_TEST_PLUGIN_DIR=$(abs_srcdir)/..:$(abs_builddir)/.. \
-	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/cfgmgr:$(abs_top_builddir)/src/lib/dns/python/.libs \
+	B10_TEST_PLUGIN_DIR=$(abs_srcdir)/..:$(abs_builddir)/.. \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/cfgmgr:$(abs_top_builddir)/src/lib/dns/python/.libs \
 	$(LIBRARY_PATH_PLACEHOLDER) \
 	$(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \
 	done

src/bin/cfgmgr/tests/Makefile.am

@@ -1,13 +1,14 @@
 PYCOVERAGE_RUN = @PYCOVERAGE_RUN@
 PYTESTS = b10-cfgmgr_test.py
 
-EXTRA_DIST = $(PYTESTS) testdata/plugins/testplugin.py
+noinst_SCRIPTS = $(PYTESTS)
+EXTRA_DIST = testdata/plugins/testplugin.py
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
@@ -19,9 +20,10 @@ if ENABLE_PYTHON_COVERAGE
 endif
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
-	env TESTDATA_PATH=$(abs_srcdir)/testdata \
+	chmod +x $(abs_builddir)/$$pytest ; \
+	TESTDATA_PATH=$(abs_srcdir)/testdata \
 	$(LIBRARY_PATH_PLACEHOLDER) \
-	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/cfgmgr:$(abs_top_builddir)/src/lib/python/isc/config \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/cfgmgr:$(abs_top_builddir)/src/lib/python/isc/config \
 	$(PYCOVERAGE_RUN) $(abs_builddir)/$$pytest || exit ; \
 	done
 

src/bin/cmdctl/Makefile.am

@@ -3,7 +3,9 @@ SUBDIRS = . tests
 pkglibexecdir = $(libexecdir)/@PACKAGE@
 
 pkglibexec_SCRIPTS = b10-cmdctl
-pyexec_DATA = cmdctl_messages.py
+
+nodist_pylogmessage_PYTHON = $(PYTHON_LOGMSGPKG_DIR)/work/cmdctl_messages.py
+pylogmessagedir = $(pyexecdir)/isc/log_messages/
 
 b10_cmdctldir = $(pkgdatadir)
 
@@ -19,7 +21,9 @@ b10_cmdctl_DATA += cmdctl.spec
 
 EXTRA_DIST = $(CMDCTL_CONFIGURATIONS)
 
-CLEANFILES=	b10-cmdctl cmdctl.pyc cmdctl.spec cmdctl_messages.py cmdctl_messages.pyc
+CLEANFILES= b10-cmdctl cmdctl.pyc cmdctl.spec
+CLEANFILES += $(PYTHON_LOGMSGPKG_DIR)/work/cmdctl_messages.py
+CLEANFILES += $(PYTHON_LOGMSGPKG_DIR)/work/cmdctl_messages.pyc
 
 man_MANS = b10-cmdctl.8
 EXTRA_DIST += $(man_MANS) b10-cmdctl.xml cmdctl_messages.mes
@@ -34,11 +38,12 @@ endif
 cmdctl.spec: cmdctl.spec.pre
 	$(SED) -e "s|@@SYSCONFDIR@@|$(sysconfdir)|" cmdctl.spec.pre >$@
 
-cmdctl_messages.py: cmdctl_messages.mes
-	$(top_builddir)/src/lib/log/compiler/message -p $(top_srcdir)/src/bin/cmdctl/cmdctl_messages.mes
+$(PYTHON_LOGMSGPKG_DIR)/work/cmdctl_messages.py : cmdctl_messages.mes
+	$(top_builddir)/src/lib/log/compiler/message \
+	-d $(PYTHON_LOGMSGPKG_DIR)/work -p $(srcdir)/cmdctl_messages.mes
 
 # this is done here since configure.ac AC_OUTPUT doesn't expand exec_prefix
-b10-cmdctl: cmdctl.py cmdctl_messages.py
+b10-cmdctl: cmdctl.py $(PYTHON_LOGMSGPKG_DIR)/work/cmdctl_messages.py
 	$(SED) "s|@@PYTHONPATH@@|@pyexecdir@|" cmdctl.py >$@
 	chmod a+x $@
 

src/bin/cmdctl/cmdctl.py.in

@@ -17,12 +17,12 @@
 
 ''' cmdctl module is the configuration entry point for all commands from bindctl
 or some other web tools client of bind10. cmdctl is pure https server which provi-
-des RESTful API. When command client connecting with cmdctl, it should first login 
-with legal username and password. 
-    When cmdctl starting up, it will collect command specification and 
+des RESTful API. When command client connecting with cmdctl, it should first login
+with legal username and password.
+    When cmdctl starting up, it will collect command specification and
 configuration specification/data of other available modules from configmanager, then
 wait for receiving request from client, parse the request and resend the request to
-the proper module. When getting the request result from the module, send back the 
+the proper module. When getting the request result from the module, send back the
 resut to client.
 '''
 
@@ -47,19 +47,14 @@ import isc.net.parse
 from optparse import OptionParser, OptionValueError
 from hashlib import sha1
 from isc.util import socketserver_mixin
-from cmdctl_messages import *
-
-# TODO: these debug-levels are hard-coded here; we are planning on
-# creating a general set of debug levels, see ticket #1074. When done,
-# we should remove these values and use the general ones in the
-# logger.debug calls
-
-# Debug level for communication with BIND10
-DBG_CMDCTL_MESSAGING = 30
+from isc.log_messages.cmdctl_messages import *
 
 isc.log.init("b10-cmdctl")
 logger = isc.log.Logger("cmdctl")
 
+# Debug level for communication with BIND10
+DBG_CMDCTL_MESSAGING = logger.DBGLVL_COMMAND
+
 try:
     import threading
 except ImportError:
@@ -86,16 +81,16 @@ SPECFILE_LOCATION = SPECFILE_PATH + os.sep + "cmdctl.spec"
 
 class CmdctlException(Exception):
     pass
-       
+
 class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
     '''https connection request handler.
     Currently only GET and POST are supported.  '''
     def do_GET(self):
-        '''The client should send its session id in header with 
+        '''The client should send its session id in header with
         the name 'cookie'
         '''
         self.session_id = self.headers.get('cookie')
-        rcode, reply = http.client.OK, []        
+        rcode, reply = http.client.OK, []
         if self._is_session_valid():
             if self._is_user_logged_in():
                 rcode, reply = self._handle_get_request()
@@ -111,16 +106,16 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
     def _handle_get_request(self):
         '''Currently only support the following three url GET request '''
         id, module = self._parse_request_path()
-        return self.server.get_reply_data_for_GET(id, module) 
+        return self.server.get_reply_data_for_GET(id, module)
 
     def _is_session_valid(self):
-        return self.session_id 
+        return self.session_id
 
     def _is_user_logged_in(self):
         login_time = self.server.user_sessions.get(self.session_id)
         if not login_time:
             return False
-        
+
         idle_time = time.time() - login_time
         if idle_time > self.server.idle_timeout:
             return False
@@ -130,7 +125,7 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
 
     def _parse_request_path(self):
         '''Parse the url, the legal url should like /ldh or /ldh/ldh '''
-        groups = URL_PATTERN.match(self.path) 
+        groups = URL_PATTERN.match(self.path)
         if not groups:
             return (None, None)
         else:
@@ -138,8 +133,8 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
 
     def do_POST(self):
         '''Process POST request. '''
-        '''Process user login and send command to proper module  
-        The client should send its session id in header with 
+        '''Process user login and send command to proper module
+        The client should send its session id in header with
         the name 'cookie'
         '''
         self.session_id = self.headers.get('cookie')
@@ -153,7 +148,7 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
                 rcode, reply = http.client.UNAUTHORIZED, ["please login"]
         else:
             rcode, reply = http.client.BAD_REQUEST, ["session isn't valid"]
-      
+
         self.send_response(rcode)
         self.end_headers()
         self.wfile.write(json.dumps(reply).encode())
@@ -174,12 +169,12 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
         length = self.headers.get('Content-Length')
 
         if not length:
-            return False, ["invalid username or password"]     
+            return False, ["invalid username or password"]
 
         try:
             user_info = json.loads((self.rfile.read(int(length))).decode())
         except:
-            return False, ["invalid username or password"]                
+            return False, ["invalid username or password"]
 
         user_name = user_info.get('username')
         if not user_name:
@@ -198,7 +193,7 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
             return False, ["username or password error"]
 
         return True, None
-   
+
 
     def _handle_post_request(self):
         '''Handle all the post request from client. '''
@@ -220,7 +215,7 @@ class SecureHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
         if rcode != 0:
             ret = http.client.BAD_REQUEST
         return ret, reply
-    
+
     def log_request(self, code='-', size='-'):
         '''Rewrite the log request function, log nothing.'''
         pass
@@ -244,11 +239,11 @@ class CommandControl():
 
     def _setup_session(self):
         '''Setup the session for receving the commands
-        sent from other modules. There are two sessions 
-        for cmdctl, one(self.module_cc) is used for receiving 
-        commands sent from other modules, another one (self._cc) 
-        is used to send the command from Bindctl or other tools 
-        to proper modules.''' 
+        sent from other modules. There are two sessions
+        for cmdctl, one(self.module_cc) is used for receiving
+        commands sent from other modules, another one (self._cc)
+        is used to send the command from Bindctl or other tools
+        to proper modules.'''
         self._cc = isc.cc.Session()
         self._module_cc = isc.config.ModuleCCSession(SPECFILE_LOCATION,
                                               self.config_handler,
@@ -256,7 +251,7 @@ class CommandControl():
         self._module_name = self._module_cc.get_module_spec().get_module_name()
         self._cmdctl_config_data = self._module_cc.get_full_config()
         self._module_cc.start()
-    
+
     def _accounts_file_check(self, filepath):
         ''' Check whether the accounts file is valid, each row
         should be a list with 3 items.'''
@@ -293,7 +288,7 @@ class CommandControl():
                 errstr = self._accounts_file_check(new_config[key])
             else:
                 errstr = 'unknown config item: ' + key
-            
+
             if errstr != None:
                 logger.error(CMDCTL_BAD_CONFIG_DATA, errstr);
                 return ccsession.create_answer(1, errstr)
@@ -319,7 +314,7 @@ class CommandControl():
                 self.modules_spec[args[0]] = args[1]
 
         elif command == ccsession.COMMAND_SHUTDOWN:
-            #When cmdctl get 'shutdown' command from boss, 
+            #When cmdctl get 'shutdown' command from boss,
             #shutdown the outer httpserver.
             self._httpserver.shutdown()
             self._serving = False
@@ -389,12 +384,12 @@ class CommandControl():
         specs = self.get_modules_spec()
         if module_name not in specs.keys():
             return 1, {'error' : 'unknown module'}
-       
+
         spec_obj = isc.config.module_spec.ModuleSpec(specs[module_name], False)
         errors = []
         if not spec_obj.validate_command(command_name, params, errors):
             return 1, {'error': errors[0]}
-        
+
         return self.send_command(module_name, command_name, params)
 
     def send_command(self, module_name, command_name, params = None):
@@ -405,7 +400,7 @@ class CommandControl():
                      command_name, module_name)
 
         if module_name == self._module_name:
-            # Process the command sent to cmdctl directly. 
+            # Process the command sent to cmdctl directly.
             answer = self.command_handler(command_name, params)
         else:
             msg = ccsession.create_command(command_name, params)
@@ -434,7 +429,7 @@ class CommandControl():
 
         logger.error(CMDCTL_COMMAND_ERROR, command_name, module_name, errstr)
         return 1, {'error': errstr}
-    
+
     def get_cmdctl_config_data(self):
         ''' If running in source code tree, use keyfile, certificate
         and user accounts file in source code. '''
@@ -458,13 +453,15 @@ class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
     '''Make the server address can be reused.'''
     allow_reuse_address = True
 
-    def __init__(self, server_address, RequestHandlerClass, 
+    def __init__(self, server_address, RequestHandlerClass,
                  CommandControlClass,
                  idle_timeout = 1200, verbose = False):
         '''idle_timeout: the max idle time for login'''
         socketserver_mixin.NoPollMixIn.__init__(self)
         try:
             http.server.HTTPServer.__init__(self, server_address, RequestHandlerClass)
+            logger.debug(DBG_CMDCTL_MESSAGING, CMDCTL_STARTED,
+                         server_address[0], server_address[1])
         except socket.error as err:
             raise CmdctlException("Error creating server, because: %s \n" % str(err))
 
@@ -477,9 +474,9 @@ class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
         self._accounts_file = None
 
     def _create_user_info(self, accounts_file):
-        '''Read all user's name and its' salt, hashed password 
+        '''Read all user's name and its' salt, hashed password
         from accounts file.'''
-        if (self._accounts_file == accounts_file) and (len(self._user_infos) > 0): 
+        if (self._accounts_file == accounts_file) and (len(self._user_infos) > 0):
             return
 
         with self._lock:
@@ -500,10 +497,10 @@ class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
         self._accounts_file = accounts_file
         if len(self._user_infos) == 0:
             logger.error(CMDCTL_NO_USER_ENTRIES_READ)
-         
+
     def get_user_info(self, username):
         '''Get user's salt and hashed string. If the user
-        doesn't exist, return None, or else, the list 
+        doesn't exist, return None, or else, the list
         [salt, hashed password] will be returned.'''
         with self._lock:
             info = self._user_infos.get(username)
@@ -512,9 +509,9 @@ class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
     def save_user_session_id(self, session_id):
         ''' Record user's id and login time. '''
         self.user_sessions[session_id] = time.time()
-        
+
     def _check_key_and_cert(self, key, cert):
-        # TODO, check the content of key/certificate file 
+        # TODO, check the content of key/certificate file
         if not os.path.exists(key):
             raise CmdctlException("key file '%s' doesn't exist " % key)
 
@@ -529,7 +526,7 @@ class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
                                       certfile = cert,
                                       keyfile = key,
                                       ssl_version = ssl.PROTOCOL_SSLv23)
-            return ssl_sock 
+            return ssl_sock
         except (ssl.SSLError, CmdctlException) as err :
             logger.info(CMDCTL_SSL_SETUP_FAILURE_USER_DENIED, err)
             self.close_request(sock)
@@ -546,18 +543,18 @@ class SecureHTTPServer(socketserver_mixin.NoPollMixIn,
 
     def get_reply_data_for_GET(self, id, module):
         '''Currently only support the following three url GET request '''
-        rcode, reply = http.client.NO_CONTENT, []        
+        rcode, reply = http.client.NO_CONTENT, []
         if not module:
             if id == CONFIG_DATA_URL:
                 rcode, reply = http.client.OK, self.cmdctl.get_config_data()
             elif id == MODULE_SPEC_URL:
                 rcode, reply = http.client.OK, self.cmdctl.get_modules_spec()
-        
-        return rcode, reply 
+
+        return rcode, reply
 
     def send_command_to_module(self, module_name, command_name, params):
         return self.cmdctl.send_command_with_check(module_name, command_name, params)
-   
+
 httpd = None
 
 def signal_handler(signal, frame):
@@ -571,10 +568,9 @@ def set_signal_handler():
 
 def run(addr = 'localhost', port = 8080, idle_timeout = 1200, verbose = False):
     ''' Start cmdctl as one https server. '''
-    if verbose:
-        sys.stdout.write("[b10-cmdctl] starting on %s port:%d\n" %(addr, port))
-    httpd = SecureHTTPServer((addr, port), SecureHTTPRequestHandler, 
+    httpd = SecureHTTPServer((addr, port), SecureHTTPRequestHandler,
                              CommandControl, idle_timeout, verbose)
+
     httpd.serve_forever()
 
 def check_port(option, opt_str, value, parser):
@@ -612,6 +608,8 @@ if __name__ == '__main__':
     (options, args) = parser.parse_args()
     result = 1                  # in case of failure
     try:
+        if options.verbose:
+            logger.set_severity("DEBUG", 99)
         run(options.addr, options.port, options.idle_timeout, options.verbose)
         result = 0
     except isc.cc.SessionError as err:

src/bin/cmdctl/cmdctl_messages.mes

@@ -64,12 +64,15 @@ be set up. The specific error is given in the log message. Possible
 causes may be that the ssl request itself was bad, or the local key or
 certificate file could not be read.
 
+% CMDCTL_STARTED cmdctl is listening for connections on %1:%2
+The cmdctl daemon has started and is now listening for connections.
+
 % CMDCTL_STOPPED_BY_KEYBOARD keyboard interrupt, shutting down
 There was a keyboard interrupt signal to stop the cmdctl daemon. The
 daemon will now shut down.
 
 % CMDCTL_UNCAUGHT_EXCEPTION uncaught exception: %1
-The b10-cdmctl daemon encountered an uncaught exception and
+The b10-cmdctl daemon encountered an uncaught exception and
 will now shut down. This is indicative of a programming error and
 should not happen under normal circumstances. The exception message
 is printed.

src/bin/cmdctl/run_b10-cmdctl.sh.in

@@ -19,9 +19,17 @@ PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
 export PYTHON_EXEC
 
 CMD_CTRLD_PATH=@abs_top_builddir@/src/bin/cmdctl
-PYTHONPATH=@abs_top_srcdir@/src/lib/python
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/xfr/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/python/isc/config:@abs_top_builddir@/src/lib/python/isc/acl/.libs:@abs_top_builddir@/src/lib/python/isc/datasrc/.libs
 export PYTHONPATH
 
+# If necessary (rare cases), explicitly specify paths to dynamic libraries
+# required by loadable python modules.
+SET_ENV_LIBRARY_PATH=@SET_ENV_LIBRARY_PATH@
+if test $SET_ENV_LIBRARY_PATH = yes; then
+        @ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:@abs_top_builddir@/src/lib/datasrc/.libs:$@ENV_LIBRARY_PATH@
+        export @ENV_LIBRARY_PATH@
+fi
+
 BIND10_MSGQ_SOCKET_FILE=@abs_top_builddir@/msgq_socket
 export BIND10_MSGQ_SOCKET_FILE
 

src/bin/cmdctl/tests/Makefile.am

@@ -6,7 +6,7 @@ EXTRA_DIST = $(PYTESTS)
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
@@ -19,7 +19,7 @@ endif
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
 	$(LIBRARY_PATH_PLACEHOLDER) \
-	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/cmdctl \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/cmdctl \
 	CMDCTL_SPEC_PATH=$(abs_top_builddir)/src/bin/cmdctl \
 	CMDCTL_SRC_PATH=$(abs_top_srcdir)/src/bin/cmdctl \
 	$(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \

src/bin/dhcp6/Makefile.am

@@ -4,9 +4,7 @@ AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
 AM_CPPFLAGS += -I$(top_srcdir)/src/bin -I$(top_builddir)/src/bin
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/dns -I$(top_builddir)/src/lib/dns
 AM_CPPFLAGS += -I$(top_srcdir)/src/lib/cc -I$(top_builddir)/src/lib/cc
-AM_CPPFLAGS += -I$(top_srcdir)/src/lib/asiolink
-AM_CPPFLAGS += -I$(top_builddir)/src/lib/asiolink
-AM_CPPFLAGS += $(BOOST_INCLUDES)
+ AM_CPPFLAGS += $(BOOST_INCLUDES)
 
 AM_CXXFLAGS = $(B10_CXXFLAGS)
 
@@ -19,34 +17,30 @@ pkglibexecdir = $(libexecdir)/@PACKAGE@
 CLEANFILES = *.gcno *.gcda spec_config.h
 
 man_MANS = b10-dhcp6.8
-EXTRA_DIST = $(man_MANS) dhcp6.spec
+EXTRA_DIST = $(man_MANS) dhcp6.spec interfaces.txt
 
-#if ENABLE_MAN
-#b10-dhcp6.8: b10-dhcp6.xml
-#	xsltproc --novalid --xinclude --nonet -o $@ http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl $(srcdir)/b10-dhcp6.xml
-#endif
+if ENABLE_MAN
+
+b10-dhcp6.8: b10-dhcp6.xml
+	xsltproc --novalid --xinclude --nonet -o $@ http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl $(srcdir)/b10-dhcp6.xml
+
+endif
 
 spec_config.h: spec_config.h.pre
 	$(SED) -e "s|@@LOCALSTATEDIR@@|$(localstatedir)|" spec_config.h.pre >$@
 
 BUILT_SOURCES = spec_config.h
 pkglibexec_PROGRAMS = b10-dhcp6
-b10_dhcp6_SOURCES = main.cc
-b10_dhcp6_SOURCES += dhcp6.h
-b10_dhcp6_LDADD =  $(top_builddir)/src/lib/datasrc/libdatasrc.la
-b10_dhcp6_LDADD += $(top_builddir)/src/lib/dns/libdns++.la
-b10_dhcp6_LDADD += $(top_builddir)/src/lib/config/libcfgclient.la
-b10_dhcp6_LDADD += $(top_builddir)/src/lib/cc/libcc.la
+
+b10_dhcp6_SOURCES = main.cc iface_mgr.cc dhcp6_srv.cc
+b10_dhcp6_SOURCES += iface_mgr.h dhcp6_srv.h
+
+b10_dhcp6_LDADD = $(top_builddir)/src/lib/dhcp/libdhcp.la
 b10_dhcp6_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la
-b10_dhcp6_LDADD += $(top_builddir)/src/lib/asiodns/libasiodns.la
 b10_dhcp6_LDADD += $(top_builddir)/src/lib/asiolink/libasiolink.la
 b10_dhcp6_LDADD += $(top_builddir)/src/lib/log/liblog.la
-b10_dhcp6_LDADD += $(top_builddir)/src/lib/xfr/libxfr.la
-b10_dhcp6_LDADD += $(top_builddir)/src/lib/server_common/libserver_common.la
-b10_dhcp6_LDADD += $(SQLITE_LIBS)
 
 # TODO: config.h.in is wrong because doesn't honor pkgdatadir
 # and can't use @datadir@ because doesn't expand default ${prefix}
 b10_dhcp6dir = $(pkgdatadir)
-b10_dhcp6_DATA = dhcp6.spec
-
+b10_dhcp6_DATA = dhcp6.spec interfaces.txt

src/bin/dhcp6/b10-dhcp6.8

@@ -1,13 +1,13 @@
 '\" t
-.\"     Title: b10-dhpc6
+.\"     Title: b10-dhcp6
 .\"    Author: [FIXME: author] [see http://docbook.sf.net/el/author]
 .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
-.\"      Date: March 8, 2011
+.\"      Date: October 27, 2011
 .\"    Manual: BIND10
 .\"    Source: BIND10
 .\"  Language: English
 .\"
-.TH "B10\-DHCP6" "8" "March 8, 2011" "BIND10" "BIND10"
+.TH "B10\-DHCP6" "8" "October 27, 2011" "BIND10" "BIND10"
 .\" -----------------------------------------------------------------
 .\" * set default formatting
 .\" -----------------------------------------------------------------
@@ -19,31 +19,32 @@
 .\" * MAIN CONTENT STARTS HERE *
 .\" -----------------------------------------------------------------
 .SH "NAME"
-b10-dhcp6 \- DHCPv6 daemon in BIND10 architecture
+b10-dhcp6 \- DHCPv6 server in BIND 10 architecture
 .SH "SYNOPSIS"
 .HP \w'\fBb10\-dhcp6\fR\ 'u
-\fBb10\-dhcp6\fR [\fB\-u\ \fR\fB\fIusername\fR\fR] [\fB\-v\fR]
+\fBb10\-dhcp6\fR [\fB\-v\fR]
 .SH "DESCRIPTION"
 .PP
 The
 \fBb10\-dhcp6\fR
-daemon will provide DHCPv6 server implementation when it becomes functional.
+daemon will provide the DHCPv6 server implementation when it becomes functional\&.
+.SH "ARGUMENTS"
 .PP
+The arguments are as follows:
+.PP
+\fB\-v\fR
+.RS 4
+Enable verbose mode\&.
+.RE
 .SH "SEE ALSO"
 .PP
 
-\fBb10-cfgmgr\fR(8),
-\fBb10-loadzone\fR(8),
-\fBb10-msgq\fR(8),
-\fBb10-stats\fR(8),
-\fBb10-zonemgr\fR(8),
-\fBbind10\fR(8),
-BIND 10 Guide\&.
+\fBbind10\fR(8)\&.
 .SH "HISTORY"
 .PP
 The
 \fBb10\-dhcp6\fR
-daemon was first coded in June 2011\&.
+daemon was first coded in June 2011 by Tomek Mrugalski\&.
 .SH "COPYRIGHT"
 .br
 Copyright \(co 2011 Internet Systems Consortium, Inc. ("ISC")

src/bin/dhcp6/b10-dhcp6.xml

@@ -0,0 +1,98 @@
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+               "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+	       [<!ENTITY mdash "&#8212;">]>
+<!--
+ - Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+ -
+ - Permission to use, copy, modify, and/or distribute this software for any
+ - purpose with or without fee is hereby granted, provided that the above
+ - copyright notice and this permission notice appear in all copies.
+ -
+ - THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+ - REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+ - AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+ - INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+ - LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+ - OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+ - PERFORMANCE OF THIS SOFTWARE.
+-->
+
+<refentry>
+
+  <refentryinfo>
+    <date>October 27, 2011</date>
+  </refentryinfo>
+
+  <refmeta>
+    <refentrytitle>b10-dhcp6</refentrytitle>
+    <manvolnum>8</manvolnum>
+    <refmiscinfo>BIND10</refmiscinfo>
+  </refmeta>
+
+  <refnamediv>
+    <refname>b10-dhcp6</refname>
+    <refpurpose>DHCPv6 server in BIND 10 architecture</refpurpose>
+  </refnamediv>
+
+  <docinfo>
+    <copyright>
+      <year>2011</year>
+      <holder>Internet Systems Consortium, Inc. ("ISC")</holder>
+    </copyright>
+  </docinfo>
+
+  <refsynopsisdiv>
+    <cmdsynopsis>
+      <command>b10-dhcp6</command>
+      <arg><option>-v</option></arg>
+    </cmdsynopsis>
+  </refsynopsisdiv>
+
+  <refsect1>
+    <title>DESCRIPTION</title>
+    <para>
+      The <command>b10-dhcp6</command> daemon will provide the
+       DHCPv6 server implementation when it becomes functional.
+    </para>
+
+  </refsect1>
+
+  <refsect1>
+    <title>ARGUMENTS</title>
+
+    <para>The arguments are as follows:</para>
+
+    <variablelist>
+
+      <varlistentry>
+        <term><option>-v</option></term>
+        <listitem><para>
+          Enable verbose mode.
+<!-- TODO: what does this do? -->
+        </para></listitem>
+      </varlistentry>
+
+    </variablelist>
+  </refsect1>
+
+  <refsect1>
+    <title>SEE ALSO</title>
+    <para>
+      <citerefentry>
+        <refentrytitle>bind10</refentrytitle><manvolnum>8</manvolnum>
+      </citerefentry>.
+    </para>
+  </refsect1>
+
+  <refsect1>
+    <title>HISTORY</title>
+    <para>
+      The <command>b10-dhcp6</command> daemon was first coded in
+      June 2011 by Tomek Mrugalski.
+    </para>
+  </refsect1>
+</refentry><!--
+ - Local variables:
+ - mode: sgml
+ - End:
+-->

src/bin/dhcp6/dhcp6.h

@@ -1,213 +0,0 @@
-/* dhcp6.h
-
-   DHCPv6 Protocol structures... */
-
-/*
- * Copyright (c) 2006-2011 by Internet Systems Consortium, Inc. ("ISC")
- *
- * Permission to use, copy, modify, and distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
- * OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
- *
- *   Internet Systems Consortium, Inc.
- *   950 Charter Street
- *   Redwood City, CA 94063
- *   <info@isc.org>
- *   https://www.isc.org/
- */
-
-
-/* DHCPv6 Option codes: */
-
-#define D6O_CLIENTID				1 /* RFC3315 */
-#define D6O_SERVERID				2
-#define D6O_IA_NA				3
-#define D6O_IA_TA				4
-#define D6O_IAADDR				5
-#define D6O_ORO					6
-#define D6O_PREFERENCE				7
-#define D6O_ELAPSED_TIME			8
-#define D6O_RELAY_MSG				9
-/* Option code 10 unassigned. */
-#define D6O_AUTH				11
-#define D6O_UNICAST				12
-#define D6O_STATUS_CODE				13
-#define D6O_RAPID_COMMIT			14
-#define D6O_USER_CLASS				15
-#define D6O_VENDOR_CLASS			16
-#define D6O_VENDOR_OPTS				17
-#define D6O_INTERFACE_ID			18
-#define D6O_RECONF_MSG				19
-#define D6O_RECONF_ACCEPT			20
-#define D6O_SIP_SERVERS_DNS			21 /* RFC3319 */
-#define D6O_SIP_SERVERS_ADDR			22 /* RFC3319 */
-#define D6O_NAME_SERVERS			23 /* RFC3646 */
-#define D6O_DOMAIN_SEARCH			24 /* RFC3646 */
-#define D6O_IA_PD				25 /* RFC3633 */
-#define D6O_IAPREFIX				26 /* RFC3633 */
-#define D6O_NIS_SERVERS				27 /* RFC3898 */
-#define D6O_NISP_SERVERS			28 /* RFC3898 */
-#define D6O_NIS_DOMAIN_NAME			29 /* RFC3898 */
-#define D6O_NISP_DOMAIN_NAME			30 /* RFC3898 */
-#define D6O_SNTP_SERVERS			31 /* RFC4075 */
-#define D6O_INFORMATION_REFRESH_TIME		32 /* RFC4242 */
-#define D6O_BCMCS_SERVER_D			33 /* RFC4280 */
-#define D6O_BCMCS_SERVER_A			34 /* RFC4280 */
-/* 35 is unassigned */
-#define D6O_GEOCONF_CIVIC			36 /* RFC4776 */
-#define D6O_REMOTE_ID				37 /* RFC4649 */
-#define D6O_SUBSCRIBER_ID			38 /* RFC4580 */
-#define D6O_CLIENT_FQDN				39 /* RFC4704 */
-#define D6O_PANA_AGENT				40 /* paa-option */
-#define D6O_NEW_POSIX_TIMEZONE			41 /* RFC4833 */
-#define D6O_NEW_TZDB_TIMEZONE			42 /* RFC4833 */
-#define D6O_ERO					43 /* RFC4994 */
-#define D6O_LQ_QUERY				44 /* RFC5007 */
-#define D6O_CLIENT_DATA				45 /* RFC5007 */
-#define D6O_CLT_TIME				46 /* RFC5007 */
-#define D6O_LQ_RELAY_DATA			47 /* RFC5007 */
-#define D6O_LQ_CLIENT_LINK			48 /* RFC5007 */
-
-/* 
- * Status Codes, from RFC 3315 section 24.4, and RFC 3633, 5007.
- */
-#define STATUS_Success		 0
-#define STATUS_UnspecFail	 1
-#define STATUS_NoAddrsAvail	 2
-#define STATUS_NoBinding	 3
-#define STATUS_NotOnLink	 4 
-#define STATUS_UseMulticast	 5 
-#define STATUS_NoPrefixAvail	 6
-#define STATUS_UnknownQueryType	 7
-#define STATUS_MalformedQuery	 8
-#define STATUS_NotConfigured	 9
-#define STATUS_NotAllowed	10
-
-/* 
- * DHCPv6 message types, defined in section 5.3 of RFC 3315 
- */
-#define DHCPV6_SOLICIT		    1
-#define DHCPV6_ADVERTISE	    2
-#define DHCPV6_REQUEST		    3
-#define DHCPV6_CONFIRM		    4
-#define DHCPV6_RENEW		    5
-#define DHCPV6_REBIND		    6
-#define DHCPV6_REPLY		    7
-#define DHCPV6_RELEASE		    8
-#define DHCPV6_DECLINE		    9
-#define DHCPV6_RECONFIGURE	   10
-#define DHCPV6_INFORMATION_REQUEST 11
-#define DHCPV6_RELAY_FORW	   12
-#define DHCPV6_RELAY_REPL	   13
-#define DHCPV6_LEASEQUERY	   14
-#define DHCPV6_LEASEQUERY_REPLY    15
-
-extern const char *dhcpv6_type_names[];
-extern const int dhcpv6_type_name_max;
-
-/* DUID type definitions (RFC3315 section 9).
- */
-#define DUID_LLT	1
-#define DUID_EN		2
-#define DUID_LL		3
-
-/* Offsets into IA_*'s where Option spaces commence.  */
-#define IA_NA_OFFSET 12 /* IAID, T1, T2, all 4 octets each */
-#define IA_TA_OFFSET  4 /* IAID only, 4 octets */
-#define IA_PD_OFFSET 12 /* IAID, T1, T2, all 4 octets each */
-
-/* Offset into IAADDR's where Option spaces commence. */
-#define IAADDR_OFFSET 24
-
-/* Offset into IAPREFIX's where Option spaces commence. */
-#define IAPREFIX_OFFSET 25
-
-/* Offset into LQ_QUERY's where Option spaces commence. */
-#define LQ_QUERY_OFFSET 17
-
-/* 
- * DHCPv6 well-known multicast addressess, from section 5.1 of RFC 3315 
- */
-#define All_DHCP_Relay_Agents_and_Servers "FF02::1:2"
-#define All_DHCP_Servers "FF05::1:3"
-
-/*
- * DHCPv6 Retransmission Constants (RFC3315 section 5.5, RFC 5007)
- */
-
-#define SOL_MAX_DELAY     1
-#define SOL_TIMEOUT       1
-#define SOL_MAX_RT      120
-#define REQ_TIMEOUT       1
-#define REQ_MAX_RT       30
-#define REQ_MAX_RC       10
-#define CNF_MAX_DELAY     1
-#define CNF_TIMEOUT       1
-#define CNF_MAX_RT        4
-#define CNF_MAX_RD       10
-#define REN_TIMEOUT      10
-#define REN_MAX_RT      600
-#define REB_TIMEOUT      10
-#define REB_MAX_RT      600
-#define INF_MAX_DELAY     1
-#define INF_TIMEOUT       1
-#define INF_MAX_RT      120
-#define REL_TIMEOUT       1
-#define REL_MAX_RC        5
-#define DEC_TIMEOUT       1
-#define DEC_MAX_RC        5
-#define REC_TIMEOUT       2
-#define REC_MAX_RC        8
-#define HOP_COUNT_LIMIT  32
-#define LQ6_TIMEOUT       1
-#define LQ6_MAX_RT       10
-#define LQ6_MAX_RC        5
-
-/* 
- * Normal packet format, defined in section 6 of RFC 3315 
- */
-struct dhcpv6_packet {
-	unsigned char msg_type;
-	unsigned char transaction_id[3];
-	unsigned char options[FLEXIBLE_ARRAY_MEMBER];
-};
-
-/* Offset into DHCPV6 Reply packets where Options spaces commence. */
-#define REPLY_OPTIONS_INDEX 4
-
-/* 
- * Relay packet format, defined in section 7 of RFC 3315 
- */
-struct dhcpv6_relay_packet {
-	unsigned char msg_type;
-	unsigned char hop_count;
-	unsigned char link_address[16];
-	unsigned char peer_address[16];
-	unsigned char options[FLEXIBLE_ARRAY_MEMBER];
-};
-
-/* Leasequery query-types (RFC 5007) */
-
-#define LQ6QT_BY_ADDRESS	1
-#define LQ6QT_BY_CLIENTID	2
-
-/*
- * DUID time starts 2000-01-01.
- * This constant is the number of seconds since 1970-01-01,
- * when the Unix epoch began.
- */
-#define DUID_TIME_EPOCH 946684800
-
-/* Information-Request Time option (RFC 4242) */
-
-#define IRT_DEFAULT	86400
-#define IRT_MINIMUM	600
-

src/bin/dhcp6/dhcp6_srv.cc

@@ -0,0 +1,231 @@
+// Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#include "dhcp/dhcp6.h"
+#include "dhcp/pkt6.h"
+#include "dhcp6/iface_mgr.h"
+#include "dhcp6/dhcp6_srv.h"
+#include "dhcp/option6_ia.h"
+#include "dhcp/option6_iaaddr.h"
+#include "asiolink/io_address.h"
+
+using namespace std;
+using namespace isc;
+using namespace isc::dhcp;
+using namespace isc::asiolink;
+
+Dhcpv6Srv::Dhcpv6Srv() {
+    cout << "Initialization" << endl;
+
+    // first call to instance() will create IfaceMgr (it's a singleton)
+    // it may throw something if things go wrong
+    IfaceMgr::instance();
+
+    /// @todo: instantiate LeaseMgr here once it is imlpemented.
+
+    setServerID();
+
+    shutdown = false;
+}
+
+Dhcpv6Srv::~Dhcpv6Srv() {
+    cout << "DHCPv6 Srv shutdown." << endl;
+}
+
+bool
+Dhcpv6Srv::run() {
+    while (!shutdown) {
+        boost::shared_ptr<Pkt6> query; // client's message
+        boost::shared_ptr<Pkt6> rsp;   // server's response
+
+        query = IfaceMgr::instance().receive();
+
+        if (query) {
+            if (!query->unpack()) {
+                cout << "Failed to parse incoming packet" << endl;
+                continue;
+            }
+            switch (query->getType()) {
+            case DHCPV6_SOLICIT:
+                rsp = processSolicit(query);
+                break;
+            case DHCPV6_REQUEST:
+                rsp = processRequest(query);
+                break;
+            case DHCPV6_RENEW:
+                rsp = processRenew(query);
+                break;
+            case DHCPV6_REBIND:
+                rsp = processRebind(query);
+                break;
+            case DHCPV6_CONFIRM:
+                rsp = processConfirm(query);
+                break;
+            case DHCPV6_RELEASE:
+                rsp = processRelease(query);
+                break;
+            case DHCPV6_DECLINE:
+                rsp = processDecline(query);
+                break;
+            case DHCPV6_INFORMATION_REQUEST:
+                rsp = processInfRequest(query);
+                break;
+            default:
+                cout << "Unknown pkt type received:"
+                     << query->getType() << endl;
+            }
+
+            cout << "Received " << query->data_len_ << " bytes packet type="
+                 << query->getType() << endl;
+            cout << query->toText();
+            if (rsp) {
+                rsp->remote_addr_ = query->remote_addr_;
+                rsp->local_addr_ = query->local_addr_;
+                rsp->remote_port_ = DHCP6_CLIENT_PORT;
+                rsp->local_port_ = DHCP6_SERVER_PORT;
+                rsp->ifindex_ = query->ifindex_;
+                rsp->iface_ = query->iface_;
+                cout << "Replying with:" << rsp->getType() << endl;
+                cout << rsp->toText();
+                cout << "----" << endl;
+                if (rsp->pack()) {
+                    cout << "#### pack successful." << endl;
+                }
+                IfaceMgr::instance().send(rsp);
+            }
+        }
+
+        // TODO add support for config session (see src/bin/auth/main.cc)
+        //      so this daemon can be controlled from bob
+    }
+
+    return (true);
+}
+
+void
+Dhcpv6Srv::setServerID() {
+    /// TODO implement this for real once interface detection is done.
+    /// Use hardcoded server-id for now
+
+    boost::shared_array<uint8_t> srvid(new uint8_t[14]);
+    srvid[0] = 0;
+    srvid[1] = 1; // DUID type 1 = DUID-LLT (see section 9.2 of RFC3315)
+    srvid[2] = 0;
+    srvid[3] = 6; // HW type = ethernet (I think. I'm typing this from my head
+                  // in hotel, without Internet connection)
+    for (int i=4; i<14; i++) {
+        srvid[i]=i-4;
+    }
+    serverid_ = boost::shared_ptr<Option>(new Option(Option::V6,
+                                                     D6O_SERVERID,
+                                                     srvid,
+                                                     0, 14));
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processSolicit(boost::shared_ptr<Pkt6> solicit) {
+
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_ADVERTISE,
+                                           solicit->getTransid(),
+                                           Pkt6::UDP));
+
+    /// TODO Rewrite this once LeaseManager is implemented.
+
+    // answer client's IA (this is mostly a dummy,
+    // so let's answer only first IA and hope there is only one)
+    boost::shared_ptr<Option> ia_opt = solicit->getOption(D6O_IA_NA);
+    if (ia_opt) {
+        // found IA
+        Option* tmp = ia_opt.get();
+        Option6IA* ia_req = dynamic_cast<Option6IA*>(tmp);
+        if (ia_req) {
+            boost::shared_ptr<Option6IA>
+                ia_rsp(new Option6IA(D6O_IA_NA, ia_req->getIAID()));
+            ia_rsp->setT1(1500);
+            ia_rsp->setT2(2600);
+            boost::shared_ptr<Option6IAAddr>
+                addr(new Option6IAAddr(D6O_IAADDR,
+                                       IOAddress("2001:db8:1234:5678::abcd"),
+                                       5000, 7000));
+            ia_rsp->addOption(addr);
+            reply->addOption(ia_rsp);
+        }
+    }
+
+    // add client-id
+    boost::shared_ptr<Option> clientid = solicit->getOption(D6O_CLIENTID);
+    if (clientid) {
+        reply->addOption(clientid);
+    }
+
+    // add server-id
+    reply->addOption(getServerID());
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processRequest(boost::shared_ptr<Pkt6> request) {
+    /// TODO: Implement processRequest() for real
+    boost::shared_ptr<Pkt6> reply = processSolicit(request);
+    reply->setType(DHCPV6_REPLY);
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processRenew(boost::shared_ptr<Pkt6> renew) {
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_REPLY,
+                                           renew->getTransid(),
+                                           Pkt6::UDP));
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processRebind(boost::shared_ptr<Pkt6> rebind) {
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_REPLY,
+                                           rebind->getTransid(),
+                                           Pkt6::UDP));
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processConfirm(boost::shared_ptr<Pkt6> confirm) {
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_REPLY,
+                                           confirm->getTransid(),
+                                           Pkt6::UDP));
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processRelease(boost::shared_ptr<Pkt6> release) {
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_REPLY,
+                                           release->getTransid(),
+                                           Pkt6::UDP));
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processDecline(boost::shared_ptr<Pkt6> decline) {
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_REPLY,
+                                           decline->getTransid(),
+                                           Pkt6::UDP));
+    return reply;
+}
+
+boost::shared_ptr<Pkt6>
+Dhcpv6Srv::processInfRequest(boost::shared_ptr<Pkt6> infRequest) {
+    boost::shared_ptr<Pkt6> reply(new Pkt6(DHCPV6_REPLY,
+                                           infRequest->getTransid(),
+                                           Pkt6::UDP));
+    return reply;
+}

src/bin/dhcp6/dhcp6_srv.h

@@ -0,0 +1,156 @@
+// Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#ifndef DHCPV6_SRV_H
+#define DHCPV6_SRV_H
+
+#include <boost/shared_ptr.hpp>
+#include <boost/noncopyable.hpp>
+#include "dhcp/pkt6.h"
+#include "dhcp/option.h"
+#include <iostream>
+
+namespace isc {
+
+namespace dhcp {
+/// @brief DHCPv6 server service.
+///
+/// This singleton class represents DHCPv6 server. It contains all
+/// top-level methods and routines necessary for server operation.
+/// In particular, it instantiates IfaceMgr, loads or generates DUID
+/// that is going to be used as server-identifier, receives incoming
+/// packets, processes them, manages leases assignment and generates
+/// appropriate responses.
+class Dhcpv6Srv : public boost::noncopyable {
+
+public:
+    /// @brief Default constructor.
+    ///
+    /// Instantiates necessary services, required to run DHCPv6 server.
+    /// In particular, creates IfaceMgr that will be responsible for
+    /// network interaction. Will instantiate lease manager, and load
+    /// old or create new DUID.
+    Dhcpv6Srv();
+
+    /// @brief Destructor. Used during DHCPv6 service shutdown.
+    ~Dhcpv6Srv();
+
+    /// @brief Returns server-intentifier option
+    ///
+    /// @return server-id option
+    boost::shared_ptr<isc::dhcp::Option>
+    getServerID() { return serverid_; }
+
+    /// @brief Main server processing loop.
+    ///
+    /// Main server processing loop. Receives incoming packets, verifies
+    /// their correctness, generates appropriate answer (if needed) and
+    /// transmits respones.
+    ///
+    /// @return true, if being shut down gracefully, fail if experienced
+    ///         critical error.
+    bool run();
+
+protected:
+    /// @brief Processes incoming SOLICIT and returns response.
+    ///
+    /// Processes received SOLICIT message and verifies that its sender
+    /// should be served. In particular IA, TA and PD options are populated
+    /// with to-be assinged addresses, temporary addresses and delegated
+    /// prefixes, respectively. In the usual 4 message exchange, server is
+    /// expected to respond with ADVERTISE message. However, if client
+    /// requests rapid-commit and server supports it, REPLY will be sent
+    /// instead of ADVERTISE and requested leases will be assigned
+    /// immediately.
+    ///
+    /// @param solicit SOLICIT message received from client
+    ///
+    /// @return ADVERTISE, REPLY message or NULL
+    boost::shared_ptr<Pkt6>
+    processSolicit(boost::shared_ptr<Pkt6> solicit);
+
+    /// @brief Processes incoming REQUEST and returns REPLY response.
+    ///
+    /// Processes incoming REQUEST message and verifies that its sender
+    /// should be served. In particular IA, TA and PD options are populated
+    /// with assinged addresses, temporary addresses and delegated
+    /// prefixes, respectively. Uses LeaseMgr to allocate or update existing
+    /// leases.
+    ///
+    /// @param request a message received from client
+    ///
+    /// @return REPLY message or NULL
+    boost::shared_ptr<Pkt6>
+    processRequest(boost::shared_ptr<Pkt6> request);
+
+    /// @brief Stub function that will handle incoming RENEW messages.
+    ///
+    /// @param renew message received from client
+    boost::shared_ptr<Pkt6>
+    processRenew(boost::shared_ptr<Pkt6> renew);
+
+    /// @brief Stub function that will handle incoming REBIND messages.
+    ///
+    /// @param rebind message received from client
+    boost::shared_ptr<Pkt6>
+    processRebind(boost::shared_ptr<Pkt6> rebind);
+
+    /// @brief Stub function that will handle incoming CONFIRM messages.
+    ///
+    /// @param confirm message received from client
+    boost::shared_ptr<Pkt6>
+    processConfirm(boost::shared_ptr<Pkt6> confirm);
+
+    /// @brief Stub function that will handle incoming RELEASE messages.
+    ///
+    /// @param release message received from client
+    boost::shared_ptr<Pkt6>
+    processRelease(boost::shared_ptr<Pkt6> release);
+
+    /// @brief Stub function that will handle incoming DECLINE messages.
+    ///
+    /// @param decline message received from client
+    boost::shared_ptr<Pkt6>
+    processDecline(boost::shared_ptr<Pkt6> decline);
+
+    /// @brief Stub function that will handle incoming INF-REQUEST messages.
+    ///
+    /// @param infRequest message received from client
+    boost::shared_ptr<Pkt6>
+    processInfRequest(boost::shared_ptr<Pkt6> infRequest);
+
+    /// @brief Sets server-identifier.
+    ///
+    /// This method attempts to set server-identifier DUID. It loads it
+    /// from a file. If file load fails, it generates new DUID using
+    /// interface link-layer addresses (EUI-64) + timestamp (DUID type
+    /// duid-llt, see RFC3315, section 9.2). If there are no suitable
+    /// interfaces present, exception it thrown
+    ///
+    /// @throws isc::Unexpected Failed to read DUID file and no suitable
+    ///         interfaces for new DUID generation are detected.
+    void setServerID();
+
+    /// server DUID (to be sent in server-identifier option)
+    boost::shared_ptr<isc::dhcp::Option> serverid_;
+
+    /// indicates if shutdown is in progress. Setting it to true will
+    /// initiate server shutdown procedure.
+    volatile bool shutdown;
+};
+
+}; // namespace isc::dhcp
+}; // namespace isc
+
+#endif // DHCP6_SRV_H

src/bin/dhcp6/iface_mgr.cc

@@ -0,0 +1,542 @@
+// Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#include <sstream>
+#include <fstream>
+#include <string.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "dhcp/dhcp6.h"
+#include "dhcp6/iface_mgr.h"
+#include "exceptions/exceptions.h"
+
+using namespace std;
+using namespace isc;
+using namespace isc::asiolink;
+using namespace isc::dhcp;
+
+namespace isc {
+
+/// IfaceMgr is a singleton implementation
+IfaceMgr* IfaceMgr::instance_ = 0;
+
+void
+IfaceMgr::instanceCreate() {
+    if (instance_) {
+        // no need to do anything. Instance is already created.
+        // Who called it again anyway? Uh oh. Had to be us, as
+        // this is private method.
+        return;
+    }
+    instance_ = new IfaceMgr();
+}
+
+IfaceMgr&
+IfaceMgr::instance() {
+    if (instance_ == 0) {
+        instanceCreate();
+    }
+    return (*instance_);
+}
+
+IfaceMgr::Iface::Iface(const std::string& name, int ifindex)
+    :name_(name), ifindex_(ifindex), mac_len_(0) {
+
+    memset(mac_, 0, sizeof(mac_));
+}
+
+std::string
+IfaceMgr::Iface::getFullName() const {
+    ostringstream tmp;
+    tmp << name_ << "/" << ifindex_;
+    return (tmp.str());
+}
+
+std::string
+IfaceMgr::Iface::getPlainMac() const {
+    ostringstream tmp;
+    tmp.fill('0');
+    tmp << hex;
+    for (int i = 0; i < mac_len_; i++) {
+        tmp.width(2);
+        tmp << mac_[i];
+        if (i < mac_len_-1) {
+            tmp << ":";
+        }
+    }
+    return (tmp.str());
+}
+
+IfaceMgr::IfaceMgr()
+    :control_buf_len_(CMSG_SPACE(sizeof(struct in6_pktinfo))),
+     control_buf_(new char[control_buf_len_])
+{
+
+    cout << "IfaceMgr initialization." << endl;
+
+    try {
+        // required for sending/receiving packets
+        // let's keep it in front, just in case someone
+        // wants to send anything during initialization
+
+        // control_buf_ = boost::scoped_array<char>();
+
+        detectIfaces();
+
+        if (!openSockets()) {
+            isc_throw(Unexpected, "Failed to open/bind sockets.");
+        }
+    } catch (const std::exception& ex) {
+        cout << "IfaceMgr creation failed:" << ex.what() << endl;
+
+        // TODO Uncomment this (or call LOG_FATAL) once
+        // interface detection is implemented. Otherwise
+        // it is not possible to run tests in a portable
+        // way (see detectIfaces() method).
+        // throw ex;
+    }
+}
+
+IfaceMgr::~IfaceMgr() {
+    // control_buf_ is deleted automatically (scoped_ptr)
+    control_buf_len_ = 0;
+}
+
+void
+IfaceMgr::detectIfaces() {
+    string ifaceName, linkLocal;
+
+    // TODO do the actual detection. Currently interface detection is faked
+    //      by reading a text file.
+
+    cout << "Interface detection is not implemented yet. "
+         << "Reading interfaces.txt file instead." << endl;
+    cout << "Please use format: interface-name link-local-address" << endl;
+
+    try {
+        ifstream interfaces("interfaces.txt");
+
+        if (!interfaces.good()) {
+            cout << "Failed to read interfaces.txt file." << endl;
+            isc_throw(Unexpected, "Failed to read interfaces.txt");
+        }
+        interfaces >> ifaceName;
+        interfaces >> linkLocal;
+
+        cout << "Detected interface " << ifaceName << "/" << linkLocal << endl;
+
+        Iface iface(ifaceName, if_nametoindex( ifaceName.c_str() ) );
+        IOAddress addr(linkLocal);
+        iface.addrs_.push_back(addr);
+        ifaces_.push_back(iface);
+        interfaces.close();
+    } catch (const std::exception& ex) {
+        // TODO: deallocate whatever memory we used
+        // not that important, since this function is going to be
+        // thrown away as soon as we get proper interface detection
+        // implemented
+
+        // TODO Do LOG_FATAL here
+        std::cerr << "Interface detection failed." << std::endl;
+        throw ex;
+    }
+}
+
+bool
+IfaceMgr::openSockets() {
+    int sock;
+
+    for (IfaceLst::iterator iface=ifaces_.begin();
+         iface!=ifaces_.end();
+         ++iface) {
+
+        for (Addr6Lst::iterator addr=iface->addrs_.begin();
+             addr!=iface->addrs_.end();
+             ++addr) {
+
+            sock = openSocket(iface->name_, *addr,
+                              DHCP6_SERVER_PORT);
+            if (sock<0) {
+                cout << "Failed to open unicast socket." << endl;
+                return (false);
+            }
+            sendsock_ = sock;
+
+            sock = openSocket(iface->name_,
+                              IOAddress(ALL_DHCP_RELAY_AGENTS_AND_SERVERS),
+                              DHCP6_SERVER_PORT);
+            if (sock<0) {
+                cout << "Failed to open multicast socket." << endl;
+                close(sendsock_);
+                return (false);
+            }
+            recvsock_ = sock;
+        }
+    }
+
+    return (true);
+}
+
+void
+IfaceMgr::printIfaces(std::ostream& out /*= std::cout*/) {
+    for (IfaceLst::const_iterator iface=ifaces_.begin();
+         iface!=ifaces_.end();
+         ++iface) {
+        out << "Detected interface " << iface->getFullName() << endl;
+        out << "  " << iface->addrs_.size() << " addr(s):" << endl;
+        for (Addr6Lst::const_iterator addr=iface->addrs_.begin();
+             addr != iface->addrs_.end();
+             ++addr) {
+            out << "  " << addr->toText() << endl;
+        }
+        out << "  mac: " << iface->getPlainMac() << endl;
+    }
+}
+
+IfaceMgr::Iface*
+IfaceMgr::getIface(int ifindex) {
+    for (IfaceLst::iterator iface=ifaces_.begin();
+         iface!=ifaces_.end();
+         ++iface) {
+        if (iface->ifindex_ == ifindex)
+            return (&(*iface));
+    }
+
+    return (NULL); // not found
+}
+
+IfaceMgr::Iface*
+IfaceMgr::getIface(const std::string& ifname) {
+    for (IfaceLst::iterator iface=ifaces_.begin();
+         iface!=ifaces_.end();
+         ++iface) {
+        if (iface->name_ == ifname)
+            return (&(*iface));
+    }
+
+    return (NULL); // not found
+}
+
+int
+IfaceMgr::openSocket(const std::string& ifname,
+                     const IOAddress& addr,
+                     int port) {
+    struct sockaddr_in6 addr6;
+
+    cout << "Creating socket on " << ifname << "/" << addr.toText()
+         << "/port=" << port << endl;
+
+    memset(&addr6, 0, sizeof(addr6));
+    addr6.sin6_family = AF_INET6;
+    addr6.sin6_port = htons(port);
+    addr6.sin6_scope_id = if_nametoindex(ifname.c_str());
+
+    memcpy(&addr6.sin6_addr,
+           addr.getAddress().to_v6().to_bytes().data(),
+           sizeof(addr6.sin6_addr));
+#ifdef HAVE_SA_LEN
+    addr6->sin6_len = sizeof(addr6);
+#endif
+
+    // TODO: use sockcreator once it becomes available
+
+    // make a socket
+    int sock = socket(AF_INET6, SOCK_DGRAM, 0);
+    if (sock < 0) {
+        cout << "Failed to create UDP6 socket." << endl;
+        return (-1);
+    }
+
+    /* Set the REUSEADDR option so that we don't fail to start if
+       we're being restarted. */
+    int flag = 1;
+    if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR,
+                   (char *)&flag, sizeof(flag)) < 0) {
+        cout << "Can't set SO_REUSEADDR option on dhcpv6 socket." << endl;
+        close(sock);
+        return (-1);
+    }
+
+    if (bind(sock, (struct sockaddr *)&addr6, sizeof(addr6)) < 0) {
+        cout << "Failed to bind socket " << sock << " to " << addr.toText()
+             << "/port=" << port << endl;
+        close(sock);
+        return (-1);
+    }
+#ifdef IPV6_RECVPKTINFO
+    /* RFC3542 - a new way */
+    if (setsockopt(sock, IPPROTO_IPV6, IPV6_RECVPKTINFO,
+                   &flag, sizeof(flag)) != 0) {
+        cout << "setsockopt: IPV6_RECVPKTINFO failed." << endl;
+        close(sock);
+        return (-1);
+    }
+#else
+    /* RFC2292 - an old way */
+    if (setsockopt(sock, IPPROTO_IPV6, IPV6_PKTINFO,
+                   &flag, sizeof(flag)) != 0) {
+        cout << "setsockopt: IPV6_PKTINFO: failed." << endl;
+        close(sock);
+        return (-1);
+    }
+#endif
+
+    // multicast stuff
+
+    if (addr.getAddress().to_v6().is_multicast()) {
+        // both mcast (ALL_DHCP_RELAY_AGENTS_AND_SERVERS and ALL_DHCP_SERVERS)
+        // are link and site-scoped, so there is no sense to join those groups
+        // with global addresses.
+
+        if ( !joinMcast( sock, ifname,
+                         string(ALL_DHCP_RELAY_AGENTS_AND_SERVERS) ) ) {
+            close(sock);
+            return (-1);
+        }
+    }
+
+    cout << "Created socket " << sock << " on " << ifname << "/" <<
+        addr.toText() << "/port=" << port << endl;
+
+    return (sock);
+}
+
+bool
+IfaceMgr::joinMcast(int sock, const std::string& ifname,
+const std::string & mcast) {
+
+    struct ipv6_mreq mreq;
+
+    if (inet_pton(AF_INET6, mcast.c_str(),
+                  &mreq.ipv6mr_multiaddr) <= 0) {
+        cout << "Failed to convert " << ifname
+             << " to IPv6 multicast address." << endl;
+        return (false);
+    }
+
+    mreq.ipv6mr_interface = if_nametoindex(ifname.c_str());
+    if (setsockopt(sock, IPPROTO_IPV6, IPV6_JOIN_GROUP,
+                   &mreq, sizeof(mreq)) < 0) {
+        cout << "Failed to join " << mcast << " multicast group." << endl;
+        return (false);
+    }
+
+    cout << "Joined multicast " << mcast << " group." << endl;
+
+    return (true);
+}
+
+bool
+IfaceMgr::send(boost::shared_ptr<Pkt6>& pkt) {
+    struct msghdr m;
+    struct iovec v;
+    int result;
+    struct in6_pktinfo *pktinfo;
+    struct cmsghdr *cmsg;
+    memset(&control_buf_[0], 0, control_buf_len_);
+
+    /*
+     * Initialize our message header structure.
+     */
+    memset(&m, 0, sizeof(m));
+
+    /*
+     * Set the target address we're sending to.
+     */
+    sockaddr_in6 to;
+    memset(&to, 0, sizeof(to));
+    to.sin6_family = AF_INET6;
+    to.sin6_port = htons(pkt->remote_port_);
+    memcpy(&to.sin6_addr,
+           pkt->remote_addr_.getAddress().to_v6().to_bytes().data(),
+           16);
+    to.sin6_scope_id = pkt->ifindex_;
+
+    m.msg_name = &to;
+    m.msg_namelen = sizeof(to);
+
+    /*
+     * Set the data buffer we're sending. (Using this wacky
+     * "scatter-gather" stuff... we only have a single chunk
+     * of data to send, so we declare a single vector entry.)
+     */
+    v.iov_base = (char *) &pkt->data_[0];
+    v.iov_len = pkt->data_len_;
+    m.msg_iov = &v;
+    m.msg_iovlen = 1;
+
+    /*
+     * Setting the interface is a bit more involved.
+     *
+     * We have to create a "control message", and set that to
+     * define the IPv6 packet information. We could set the
+     * source address if we wanted, but we can safely let the
+     * kernel decide what that should be.
+     */
+    m.msg_control = &control_buf_[0];
+    m.msg_controllen = control_buf_len_;
+    cmsg = CMSG_FIRSTHDR(&m);
+    cmsg->cmsg_level = IPPROTO_IPV6;
+    cmsg->cmsg_type = IPV6_PKTINFO;
+    cmsg->cmsg_len = CMSG_LEN(sizeof(*pktinfo));
+    pktinfo = (struct in6_pktinfo *)CMSG_DATA(cmsg);
+    memset(pktinfo, 0, sizeof(*pktinfo));
+    pktinfo->ipi6_ifindex = pkt->ifindex_;
+    m.msg_controllen = cmsg->cmsg_len;
+
+    result = sendmsg(sendsock_, &m, 0);
+    if (result < 0) {
+        cout << "Send packet failed." << endl;
+    }
+    cout << "Sent " << result << " bytes." << endl;
+
+    cout << "Sent " << pkt->data_len_ << " bytes over "
+         << pkt->iface_ << "/" << pkt->ifindex_ << " interface: "
+         << " dst=" << pkt->remote_addr_.toText()
+         << ", src=" << pkt->local_addr_.toText()
+         << endl;
+
+    return (result);
+}
+
+boost::shared_ptr<Pkt6>
+IfaceMgr::receive() {
+    struct msghdr m;
+    struct iovec v;
+    int result;
+    struct cmsghdr* cmsg;
+    struct in6_pktinfo* pktinfo;
+    struct sockaddr_in6 from;
+    struct in6_addr to_addr;
+    boost::shared_ptr<Pkt6> pkt;
+    char addr_str[INET6_ADDRSTRLEN];
+
+    try {
+        // RFC3315 states that server responses may be
+        // fragmented if they are over MTU. There is no
+        // text whether client's packets may be larger
+        // than 1500. Nevertheless to be on the safe side
+        // we use larger buffer. This buffer limit is checked
+        // during reception (see iov_len below), so we are
+        // safe
+        pkt = boost::shared_ptr<Pkt6>(new Pkt6(65536));
+    } catch (const std::exception& ex) {
+        cout << "Failed to create new packet." << endl;
+        return (boost::shared_ptr<Pkt6>()); // NULL
+    }
+
+    memset(&control_buf_[0], 0, control_buf_len_);
+
+    memset(&from, 0, sizeof(from));
+    memset(&to_addr, 0, sizeof(to_addr));
+
+    /*
+     * Initialize our message header structure.
+     */
+    memset(&m, 0, sizeof(m));
+
+    /*
+     * Point so we can get the from address.
+     */
+    m.msg_name = &from;
+    m.msg_namelen = sizeof(from);
+
+    /*
+     * Set the data buffer we're receiving. (Using this wacky
+     * "scatter-gather" stuff... but we that doesn't really make
+     * sense for us, so we use a single vector entry.)
+     */
+    v.iov_base = (void*)&pkt->data_[0];
+    v.iov_len = pkt->data_len_;
+    m.msg_iov = &v;
+    m.msg_iovlen = 1;
+
+    /*
+     * Getting the interface is a bit more involved.
+     *
+     * We set up some space for a "control message". We have
+     * previously asked the kernel to give us packet
+     * information (when we initialized the interface), so we
+     * should get the destination address from that.
+     */
+    m.msg_control = &control_buf_[0];
+    m.msg_controllen = control_buf_len_;
+
+    result = recvmsg(recvsock_, &m, 0);
+
+    if (result >= 0) {
+        /*
+         * If we did read successfully, then we need to loop
+         * through the control messages we received and
+         * find the one with our destination address.
+         *
+         * We also keep a flag to see if we found it. If we
+         * didn't, then we consider this to be an error.
+         */
+        int found_pktinfo = 0;
+        cmsg = CMSG_FIRSTHDR(&m);
+        while (cmsg != NULL) {
+            if ((cmsg->cmsg_level == IPPROTO_IPV6) &&
+                (cmsg->cmsg_type == IPV6_PKTINFO)) {
+                pktinfo = (struct in6_pktinfo*)CMSG_DATA(cmsg);
+                to_addr = pktinfo->ipi6_addr;
+                pkt->ifindex_ = pktinfo->ipi6_ifindex;
+                found_pktinfo = 1;
+            }
+            cmsg = CMSG_NXTHDR(&m, cmsg);
+        }
+        if (!found_pktinfo) {
+            cout << "Unable to find pktinfo" << endl;
+            return (boost::shared_ptr<Pkt6>()); // NULL
+        }
+    } else {
+        cout << "Failed to receive data." << endl;
+        return (boost::shared_ptr<Pkt6>()); // NULL
+    }
+
+    // That's ugly.
+    // TODO add IOAddress constructor that will take struct in6_addr*
+    // TODO: there's from_bytes() method added in IOAddress. Use it!
+    inet_ntop(AF_INET6, &to_addr, addr_str,INET6_ADDRSTRLEN);
+    pkt->local_addr_ = IOAddress(string(addr_str));
+
+    // TODO: there's from_bytes() method added in IOAddress. Use it!
+    inet_ntop(AF_INET6, &from.sin6_addr, addr_str, INET6_ADDRSTRLEN);
+    pkt->remote_addr_ = IOAddress(string(addr_str));
+
+    pkt->remote_port_ = ntohs(from.sin6_port);
+
+    Iface* received = getIface(pkt->ifindex_);
+    if (received) {
+        pkt->iface_ = received->name_;
+    } else {
+        cout << "Received packet over unknown interface (ifindex="
+             << pkt->ifindex_ << ")." << endl;
+        return (boost::shared_ptr<Pkt6>()); // NULL
+    }
+
+    pkt->data_len_ = result;
+
+    // TODO Move this to LOG_DEBUG
+    cout << "Received " << pkt->data_len_ << " bytes over "
+         << pkt->iface_ << "/" << pkt->ifindex_ << " interface: "
+         << " src=" << pkt->remote_addr_.toText()
+         << ", dst=" << pkt->local_addr_.toText()
+         << endl;
+
+    return (pkt);
+}
+
+}

src/bin/dhcp6/iface_mgr.h

@@ -0,0 +1,229 @@
+// Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#ifndef IFACE_MGR_H
+#define IFACE_MGR_H
+
+#include <list>
+#include <boost/shared_ptr.hpp>
+#include <boost/scoped_array.hpp>
+#include <boost/noncopyable.hpp>
+#include "asiolink/io_address.h"
+#include "dhcp/pkt6.h"
+
+namespace isc {
+
+namespace dhcp {
+/// @brief handles network interfaces, transmission and reception
+///
+/// IfaceMgr is an interface manager class that detects available network
+/// interfaces, configured addresses, link-local addresses, and provides
+/// API for using sockets.
+///
+class IfaceMgr : public boost::noncopyable {
+public:
+    /// type that defines list of addresses
+    typedef std::list<isc::asiolink::IOAddress> Addr6Lst;
+
+    /// maximum MAC address length (Infiniband uses 20 bytes)
+    static const unsigned int MAX_MAC_LEN = 20;
+
+    /// @brief represents a single network interface
+    ///
+    /// Iface structure represents network interface with all useful
+    /// information, like name, interface index, MAC address and
+    /// list of assigned addresses
+    struct Iface {
+        /// constructor
+        Iface(const std::string& name, int ifindex);
+
+        /// returns full interface name in format ifname/ifindex
+        std::string getFullName() const;
+
+        /// returns link-layer address a plain text
+        std::string getPlainMac() const;
+
+        /// network interface name
+        std::string name_;
+
+        /// interface index (a value that uniquely indentifies an interface)
+        int ifindex_;
+
+        /// list of assigned addresses
+        Addr6Lst addrs_;
+
+        /// link-layer address
+        uint8_t mac_[MAX_MAC_LEN];
+
+        /// length of link-layer address (usually 6)
+        int mac_len_;
+
+        /// socket used to sending data
+        int sendsock_;
+
+        /// socket used for receiving data
+        int recvsock_;
+    };
+
+    // TODO performance improvement: we may change this into
+    //      2 maps (ifindex-indexed and name-indexed) and
+    //      also hide it (make it public make tests easier for now)
+
+    /// type that holds a list of interfaces
+    typedef std::list<Iface> IfaceLst;
+
+    /// IfaceMgr is a singleton class. This method returns reference
+    /// to its sole instance.
+    ///
+    /// @return the only existing instance of interface manager
+    static IfaceMgr& instance();
+
+    /// @brief Returns interface with specified interface index
+    ///
+    /// @param ifindex index of searched interface
+    ///
+    /// @return interface with requested index (or NULL if no such
+    ///         interface is present)
+    ///
+    Iface*
+    getIface(int ifindex);
+
+    /// @brief Returns interface with specified interface name
+    ///
+    /// @param ifname name of searched interface
+    ///
+    /// @return interface with requested name (or NULL if no such
+    ///         interface is present)
+    ///
+    Iface*
+    getIface(const std::string& ifname);
+
+    /// debugging method that prints out all available interfaces
+    ///
+    /// @param out specifies stream to print list of interfaces to
+    void
+    printIfaces(std::ostream& out = std::cout);
+
+    /// @brief Sends a packet.
+    ///
+    /// Sends a packet. All parameters for actual transmission are specified in
+    /// Pkt6 structure itself. That includes destination address, src/dst port
+    /// and interface over which data will be sent.
+    ///
+    /// @param pkt packet to be sent
+    ///
+    /// @return true if sending was successful
+    bool
+    send(boost::shared_ptr<Pkt6>& pkt);
+
+    /// @brief Tries to receive packet over open sockets.
+    ///
+    /// Attempts to receive a single packet of any of the open sockets.
+    /// If reception is successful and all information about its sender
+    /// are obtained, Pkt6 object is created and returned.
+    ///
+    /// TODO Start using select() and add timeout to be able
+    /// to not wait infinitely, but rather do something useful
+    /// (e.g. remove expired leases)
+    ///
+    /// @return Pkt6 object representing received packet (or NULL)
+    boost::shared_ptr<Pkt6> receive();
+
+    // don't use private, we need derived classes in tests
+protected:
+
+    /// @brief Protected constructor.
+    ///
+    /// Protected constructor. This is a singleton class. We don't want
+    /// anyone to create instances of IfaceMgr. Use instance() method
+    IfaceMgr();
+
+    ~IfaceMgr();
+
+    /// @brief Detects network interfaces.
+    ///
+    /// This method will eventually detect available interfaces. For now
+    /// it offers stub implementation. First interface name and link-local
+    /// IPv6 address is read from intefaces.txt file.
+    void
+    detectIfaces();
+
+    ///
+    /// Opens UDP/IPv6 socket and binds it to address, interface and port.
+    ///
+    /// @param ifname name of the interface
+    /// @param addr address to be bound.
+    /// @param port UDP port.
+    ///
+    /// @return socket descriptor, if socket creation, binding and multicast
+    /// group join were all successful. -1 otherwise.
+    int openSocket(const std::string& ifname,
+                   const isc::asiolink::IOAddress& addr,
+                   int port);
+
+    // TODO: having 2 maps (ifindex->iface and ifname->iface would)
+    //      probably be better for performance reasons
+
+    /// List of available interfaces
+    IfaceLst ifaces_;
+
+    /// a pointer to a sole instance of this class (a singleton)
+    static IfaceMgr * instance_;
+
+    // TODO: Also keep this interface on Iface once interface detection
+    // is implemented. We may need it e.g. to close all sockets on
+    // specific interface
+    int recvsock_; // TODO: should be fd_set eventually, but we have only
+    int sendsock_; // 2 sockets for now. Will do for until next release
+    // we can't use the same socket, as receiving socket
+    // is bound to multicast address. And we all know what happens
+    // to people who try to use multicast as source address.
+
+    /// length of the control_buf_ array
+    int control_buf_len_;
+
+    /// control-buffer, used in transmission and reception
+    boost::scoped_array<char> control_buf_;
+
+private:
+    /// Opens sockets on detected interfaces.
+    bool
+    openSockets();
+
+    /// creates a single instance of this class (a singleton implementation)
+    static void
+    instanceCreate();
+
+    /// @brief Joins IPv6 multicast group on a socket.
+    ///
+    /// Socket must be created and bound to an address. Note that this
+    /// address is different than the multicast address. For example DHCPv6
+    /// server should bind its socket to link-local address (fe80::1234...)
+    /// and later join ff02::1:2 multicast group.
+    ///
+    /// @param sock socket fd (socket must be bound)
+    /// @param ifname interface name (for link-scoped multicast groups)
+    /// @param mcast multicast address to join (e.g. "ff02::1:2")
+    ///
+    /// @return true if multicast join was successful
+    ///
+    bool
+    joinMcast(int sock, const std::string& ifname,
+              const std::string& mcast);
+};
+
+}; // namespace isc::dhcp
+}; // namespace isc
+
+#endif

src/bin/dhcp6/interfaces.txt

@@ -0,0 +1,10 @@
+eth0 fe80::21e:8cff:fe9b:7349
+
+#
+# only first line is read.
+# please use following format:
+# interface-name link-local-ipv6-address
+#
+# This file will become obsolete once proper interface detection 
+# is implemented.
+#

src/bin/dhcp6/main.cc

@@ -26,21 +26,23 @@
 #include <iostream>
 
 #include <exceptions/exceptions.h>
+#if 0
+// TODO cc is not used yet. It should be eventually
 #include <cc/session.h>
 #include <config/ccsession.h>
+#endif
 
 #include <util/buffer.h>
 #include <log/dummylog.h>
 
 #include <dhcp6/spec_config.h>
-
+#include "dhcp6/dhcp6_srv.h"
 
 using namespace std;
 using namespace isc::util;
-using namespace isc::data;
-using namespace isc::cc;
-using namespace isc::config;
-using namespace isc::util;
+
+using namespace isc;
+using namespace isc::dhcp;
 
 namespace {
 
@@ -48,9 +50,8 @@ bool verbose_mode = false;
 
 void
 usage() {
-    cerr << "Usage:  b10-dhcp6 [-u user] [-v]"
+    cerr << "Usage:  b10-dhcp6 [-v]"
          << endl;
-    cerr << "\t-u: change process UID to the specified user" << endl;
     cerr << "\t-v: verbose output" << endl;
     exit(1);
 }
@@ -59,40 +60,32 @@ usage() {
 int
 main(int argc, char* argv[]) {
     int ch;
-    const char* uid = NULL;
-    bool cache = true;
 
-    while ((ch = getopt(argc, argv, ":nu:v")) != -1) {
+    while ((ch = getopt(argc, argv, ":v")) != -1) {
         switch (ch) {
-        case 'n':
-            cache = false;
-            break;
-        case 'u':
-            uid = optarg;
-            break;
         case 'v':
             verbose_mode = true;
             isc::log::denabled = true;
             break;
-        case '?':
+        case ':':
         default:
             usage();
         }
     }
 
+    cout << "My pid=" << getpid() << endl;
+
     if (argc - optind > 0) {
         usage();
     }
 
     int ret = 0;
 
-    // XXX: we should eventually pass io_service here.
+    // TODO remainder of auth to dhcp6 code copy. We need to enable this in
+    //      dhcp6 eventually
 #if 0
     Session* cc_session = NULL;
-    Session* xfrin_session = NULL;
     Session* statistics_session = NULL;
-    bool xfrin_session_established = false; // XXX (see Trac #287)
-    bool statistics_session_established = false; // XXX (see Trac #287)
     ModuleCCSession* config_session = NULL;
 #endif
     try {
@@ -104,19 +97,16 @@ main(int argc, char* argv[]) {
             specfile = string(DHCP6_SPECFILE_LOCATION);
         }
 
-        // auth_server = new AuthSrv(cache, xfrout_client);
-        // auth_server->setVerbose(verbose_mode);
         cout << "[b10-dhcp6] Initiating DHCPv6 operation." << endl;
 
+        Dhcpv6Srv* srv = new Dhcpv6Srv();
+
+        srv->run();
+
     } catch (const std::exception& ex) {
         cerr << "[b10-dhcp6] Server failed: " << ex.what() << endl;
         ret = 1;
     }
 
-    while (true) {
-            sleep(10);
-            cout << "[b10-dhcp6] I'm alive." << endl;
-    }
-
     return (ret);
 }

src/bin/dhcp6/tests/Makefile.am

@@ -8,15 +8,57 @@ EXTRA_DIST = $(PYTESTS)
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
 check-local:
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
-	env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/bind10 \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_srcdir)/src/bin:$(abs_top_builddir)/src/bin/bind10:$(abs_top_builddir)/src/lib/util/io/.libs \
 	$(LIBRARY_PATH_PLACEHOLDER) \
 	BIND10_MSGQ_SOCKET_FILE=$(abs_top_builddir)/msgq_socket \
 		$(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \
 	done
+
+
+AM_CPPFLAGS = -I$(top_srcdir)/src/lib -I$(top_builddir)/src/lib
+AM_CPPFLAGS += -I$(top_builddir)/src/bin # for generated spec_config.h header
+AM_CPPFLAGS += -I$(top_srcdir)/src/bin
+AM_CPPFLAGS += -I$(top_builddir)/src/lib/cc
+AM_CPPFLAGS += -I$(top_srcdir)/src/lib/asiolink
+AM_CPPFLAGS += $(BOOST_INCLUDES)
+AM_CPPFLAGS += -DTEST_DATA_DIR=\"$(abs_top_srcdir)/src/lib/testutils/testdata\"
+AM_CPPFLAGS += -DTEST_DATA_BUILDDIR=\"$(abs_top_builddir)/src/bin/dhcp6/tests\"
+AM_CPPFLAGS += -DINSTALL_PROG=\"$(abs_top_srcdir)/install-sh\"
+
+CLEANFILES = $(builddir)/interfaces.txt
+
+AM_CXXFLAGS = $(B10_CXXFLAGS)
+
+if USE_STATIC_LINK
+AM_LDFLAGS = -static
+endif
+
+TESTS =
+if HAVE_GTEST
+
+TESTS += dhcp6_unittests
+
+dhcp6_unittests_SOURCES = ../iface_mgr.h ../iface_mgr.cc
+dhcp6_unittests_SOURCES += ../dhcp6_srv.h ../dhcp6_srv.cc
+dhcp6_unittests_SOURCES += dhcp6_unittests.cc
+dhcp6_unittests_SOURCES += iface_mgr_unittest.cc
+dhcp6_unittests_SOURCES += dhcp6_srv_unittest.cc
+
+dhcp6_unittests_CPPFLAGS = $(AM_CPPFLAGS) $(GTEST_INCLUDES)
+dhcp6_unittests_LDFLAGS = $(AM_LDFLAGS) $(GTEST_LDFLAGS)
+dhcp6_unittests_LDADD = $(GTEST_LDADD)
+dhcp6_unittests_LDADD += $(SQLITE_LIBS)
+dhcp6_unittests_LDADD += $(top_builddir)/src/lib/asiolink/libasiolink.la
+dhcp6_unittests_LDADD += $(top_builddir)/src/lib/dhcp/libdhcp.la
+dhcp6_unittests_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la
+dhcp6_unittests_LDADD += $(top_builddir)/src/lib/log/liblog.la
+endif
+
+noinst_PROGRAMS = $(TESTS)

src/bin/dhcp6/tests/dhcp6_srv_unittest.cc

@@ -0,0 +1,148 @@
+// Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#include <config.h>
+#include <iostream>
+#include <sstream>
+
+#include <arpa/inet.h>
+#include <gtest/gtest.h>
+
+#include "dhcp/dhcp6.h"
+#include "dhcp6/dhcp6_srv.h"
+#include "dhcp/option6_ia.h"
+
+using namespace std;
+using namespace isc;
+using namespace isc::dhcp;
+
+// namespace has to be named, because friends are defined in Dhcpv6Srv class
+// Maybe it should be isc::test?
+namespace test {
+
+class NakedDhcpv6Srv: public Dhcpv6Srv {
+    // "naked" Interface Manager, exposes internal fields
+public:
+    NakedDhcpv6Srv() { }
+
+    boost::shared_ptr<Pkt6>
+    processSolicit(boost::shared_ptr<Pkt6>& request) {
+        return Dhcpv6Srv::processSolicit(request);
+    }
+    boost::shared_ptr<Pkt6>
+    processRequest(boost::shared_ptr<Pkt6>& request) {
+        return Dhcpv6Srv::processRequest(request);
+    }
+};
+
+class Dhcpv6SrvTest : public ::testing::Test {
+public:
+    Dhcpv6SrvTest() {
+    }
+};
+
+TEST_F(Dhcpv6SrvTest, basic) {
+    // there's almost no code now. What's there provides echo capability
+    // that is just a proof of concept and will be removed soon
+    // No need to thoroughly test it
+
+    // srv has stubbed interface detection. It will read
+    // interfaces.txt instead. It will pretend to have detected
+    // fe80::1234 link-local address on eth0 interface. Obviously
+    // an attempt to bind this socket will fail.
+    EXPECT_NO_THROW( {
+        Dhcpv6Srv * srv = new Dhcpv6Srv();
+
+        delete srv;
+        });
+
+}
+
+TEST_F(Dhcpv6SrvTest, Solicit_basic) {
+    NakedDhcpv6Srv * srv = 0;
+    EXPECT_NO_THROW( srv = new NakedDhcpv6Srv(); );
+
+    // a dummy content for client-id
+    boost::shared_array<uint8_t> clntDuid(new uint8_t[32]);
+    for (int i=0; i<32; i++)
+        clntDuid[i] = 100+i;
+
+    boost::shared_ptr<Pkt6> sol =
+        boost::shared_ptr<Pkt6>(new Pkt6(DHCPV6_SOLICIT,
+                                         1234, Pkt6::UDP));
+
+    boost::shared_ptr<Option6IA> ia =
+        boost::shared_ptr<Option6IA>(new Option6IA(D6O_IA_NA, 234));
+    ia->setT1(1501);
+    ia->setT2(2601);
+    sol->addOption(ia);
+
+    // Let's not send address in solicit yet
+    // boost::shared_ptr<Option6IAAddr> addr(new Option6IAAddr(D6O_IAADDR,
+    //    IOAddress("2001:db8:1234:ffff::ffff"), 5001, 7001));
+    // ia->addOption(addr);
+    // sol->addOption(ia);
+
+    // constructed very simple SOLICIT message with:
+    // - client-id option (mandatory)
+    // - IA option (a request for address, without any addresses)
+
+    // expected returned ADVERTISE message:
+    // - copy of client-id
+    // - server-id
+    // - IA that includes IAADDR
+
+    boost::shared_ptr<Option> clientid =
+        boost::shared_ptr<Option>(new Option(Option::V6, D6O_CLIENTID,
+                                             clntDuid, 0, 16));
+    sol->addOption(clientid);
+
+    boost::shared_ptr<Pkt6> reply = srv->processSolicit(sol);
+
+    // check if we get response at all
+    ASSERT_TRUE( reply != boost::shared_ptr<Pkt6>() );
+
+    EXPECT_EQ( DHCPV6_ADVERTISE, reply->getType() );
+    EXPECT_EQ( 1234, reply->getTransid() );
+
+    boost::shared_ptr<Option> tmp = reply->getOption(D6O_IA_NA);
+    ASSERT_TRUE( tmp );
+
+    Option6IA * reply_ia = dynamic_cast<Option6IA*> ( tmp.get() );
+    EXPECT_EQ( 234, reply_ia->getIAID() );
+
+    // check that there's an address included
+    EXPECT_TRUE( reply_ia->getOption(D6O_IAADDR));
+
+    // check that server included our own client-id
+    tmp = reply->getOption(D6O_CLIENTID);
+    ASSERT_TRUE( tmp );
+    EXPECT_EQ(clientid->getType(), tmp->getType() );
+    ASSERT_EQ(clientid->len(), tmp->len() );
+
+    EXPECT_TRUE( clientid->getData() == tmp->getData() );
+
+    // check that server included its server-id
+    tmp = reply->getOption(D6O_SERVERID);
+    EXPECT_EQ(tmp->getType(), srv->getServerID()->getType() );
+    ASSERT_EQ(tmp->len(),  srv->getServerID()->len() );
+
+    EXPECT_TRUE(tmp->getData() == srv->getServerID()->getData());
+
+    // more checks to be implemented
+    delete srv;
+
+}
+
+}

src/bin/dhcp6/tests/dhcp6_test.py

@@ -13,7 +13,7 @@
 # NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
 # WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 
-from bind10 import ProcessInfo, parse_args, dump_pid, unlink_pid_file, _BASETIME
+from bind10_src import ProcessInfo, parse_args, dump_pid, unlink_pid_file, _BASETIME
 
 import unittest
 import sys

src/bin/dhcp6/tests/dhcp6_unittests.cc

@@ -0,0 +1,28 @@
+// Copyright (C) 2009  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#include <stdio.h>
+#include <gtest/gtest.h>
+#include <log/logger_support.h>
+
+int
+main(int argc, char* argv[]) {
+
+    ::testing::InitGoogleTest(&argc, argv);
+    isc::log::initLogger();
+
+    int result = RUN_ALL_TESTS();
+
+    return result;
+}

src/bin/dhcp6/tests/iface_mgr_unittest.cc

@@ -0,0 +1,367 @@
+// Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+#include <config.h>
+#include <iostream>
+#include <fstream>
+#include <sstream>
+
+#include <arpa/inet.h>
+#include <gtest/gtest.h>
+
+#include "io_address.h"
+#include "dhcp/pkt6.h"
+#include "dhcp6/iface_mgr.h"
+
+using namespace std;
+using namespace isc;
+using namespace isc::asiolink;
+using namespace isc::dhcp;
+
+// name of loopback interface detection
+char LOOPBACK[32] = "lo";
+
+namespace {
+const char* const INTERFACE_FILE = TEST_DATA_BUILDDIR "/interfaces.txt";
+
+class NakedIfaceMgr: public IfaceMgr {
+    // "naked" Interface Manager, exposes internal fields
+public:
+    NakedIfaceMgr() { }
+    IfaceLst & getIfacesLst() { return ifaces_; }
+    void setSendSock(int sock) { sendsock_ = sock; }
+    void setRecvSock(int sock) { recvsock_ = sock; }
+
+    int openSocket(const std::string& ifname,
+                   const isc::asiolink::IOAddress& addr,
+                   int port) {
+        return IfaceMgr::openSocket(ifname, addr, port);
+    }
+
+};
+
+// dummy class for now, but this will be expanded when needed
+class IfaceMgrTest : public ::testing::Test {
+public:
+    IfaceMgrTest() {
+    }
+};
+
+// We need some known interface to work reliably. Loopback interface
+// is named lo on Linux and lo0 on BSD boxes. We need to find out
+// which is available. This is not a real test, but rather a workaround
+// that will go away when interface detection is implemented.
+
+// NOTE: At this stage of development, write access to current directory
+// during running tests is required.
+TEST_F(IfaceMgrTest, loDetect) {
+
+    // poor man's interface detection
+    // it will go away as soon as proper interface detection
+    // is implemented
+    if (if_nametoindex("lo")>0) {
+        cout << "This is Linux, using lo as loopback." << endl;
+        sprintf(LOOPBACK, "lo");
+    } else if (if_nametoindex("lo0")>0) {
+        cout << "This is BSD, using lo0 as loopback." << endl;
+        sprintf(LOOPBACK, "lo0");
+    } else {
+        cout << "Failed to detect loopback interface. Neither "
+             << "lo or lo0 worked. I give up." << endl;
+        ASSERT_TRUE(false);
+    }
+}
+
+// uncomment this test to create packet writer. It will
+// write incoming DHCPv6 packets as C arrays. That is useful
+// for generating test sequences based on actual traffic
+//
+// TODO: this potentially should be moved to a separate tool
+//
+
+#if 0
+TEST_F(IfaceMgrTest, dhcp6Sniffer) {
+    // testing socket operation in a portable way is tricky
+    // without interface detection implemented
+
+    unlink("interfaces.txt");
+
+    ofstream interfaces("interfaces.txt", ios::ate);
+    interfaces << "eth0 fe80::21e:8cff:fe9b:7349";
+    interfaces.close();
+
+    NakedIfaceMgr * ifacemgr = new NakedIfaceMgr();
+
+    Pkt6 * pkt = 0;
+    int cnt = 0;
+    cout << "---8X-----------------------------------------" << endl;
+    while (true) {
+        pkt = ifacemgr->receive();
+
+        cout << "// Received " << pkt->data_len_ << " bytes packet:" << endl;
+        cout << "Pkt6 *capture" << cnt++ << "() {" << endl;
+        cout << "    Pkt6* pkt;" << endl;
+        cout << "    pkt = new Pkt6(" << pkt->data_len_ << ");" << endl;
+        cout << "    pkt->remote_port_ = " << pkt-> remote_port_ << ";" << endl;
+        cout << "    pkt->remote_addr_ = IOAddress(\""
+             << pkt->remote_addr_.toText() << "\");" << endl;
+        cout << "    pkt->local_port_ = " << pkt-> local_port_ << ";" << endl;
+        cout << "    pkt->local_addr_ = IOAddress(\""
+             << pkt->local_addr_.toText() << "\");" << endl;
+        cout << "    pkt->ifindex_ = " << pkt->ifindex_ << ";" << endl;
+        cout << "    pkt->iface_ = \"" << pkt->iface_ << "\";" << endl;
+
+        // TODO it is better to declare an array and then memcpy it to
+        // packet.
+        for (int i=0; i< pkt->data_len_; i++) {
+            cout << "    pkt->data_[" << i << "]="
+                 << (int)(unsigned char)pkt->data_[i] << "; ";
+            if (!(i%4))
+                cout << endl;
+        }
+        cout << endl;
+        cout << "    return (pkt);" << endl;
+        cout << "}" << endl << endl;
+
+        delete pkt;
+    }
+    cout << "---8X-----------------------------------------" << endl;
+
+    // never happens. Infinite loop is infinite
+    delete pkt;
+    delete ifacemgr;
+}
+#endif
+
+TEST_F(IfaceMgrTest, basic) {
+    // checks that IfaceManager can be instantiated
+
+    IfaceMgr & ifacemgr = IfaceMgr::instance();
+    ASSERT_TRUE(&ifacemgr != 0);
+}
+
+TEST_F(IfaceMgrTest, ifaceClass) {
+    // basic tests for Iface inner class
+
+    IfaceMgr::Iface * iface = new IfaceMgr::Iface("eth5", 7);
+
+    EXPECT_STREQ("eth5/7", iface->getFullName().c_str());
+
+    delete iface;
+
+}
+
+// TODO: Implement getPlainMac() test as soon as interface detection
+// is implemented.
+TEST_F(IfaceMgrTest, getIface) {
+
+    cout << "Interface checks. Please ignore socket binding errors." << endl;
+    NakedIfaceMgr * ifacemgr = new NakedIfaceMgr();
+
+    // interface name, ifindex
+    IfaceMgr::Iface iface1("lo1", 1);
+    IfaceMgr::Iface iface2("eth5", 2);
+    IfaceMgr::Iface iface3("en3", 5);
+    IfaceMgr::Iface iface4("e1000g0", 3);
+
+    // note: real interfaces may be detected as well
+    ifacemgr->getIfacesLst().push_back(iface1);
+    ifacemgr->getIfacesLst().push_back(iface2);
+    ifacemgr->getIfacesLst().push_back(iface3);
+    ifacemgr->getIfacesLst().push_back(iface4);
+
+    cout << "There are " << ifacemgr->getIfacesLst().size()
+         << " interfaces." << endl;
+    for (IfaceMgr::IfaceLst::iterator iface=ifacemgr->getIfacesLst().begin();
+         iface != ifacemgr->getIfacesLst().end();
+         ++iface) {
+        cout << "  " << iface->name_ << "/" << iface->ifindex_ << endl;
+    }
+
+
+    // check that interface can be retrieved by ifindex
+    IfaceMgr::Iface * tmp = ifacemgr->getIface(5);
+    // ASSERT_NE(NULL, tmp); is not supported. hmmmm.
+    ASSERT_TRUE( tmp != NULL );
+
+    EXPECT_STREQ( "en3", tmp->name_.c_str() );
+    EXPECT_EQ(5, tmp->ifindex_);
+
+    // check that interface can be retrieved by name
+    tmp = ifacemgr->getIface("lo1");
+    ASSERT_TRUE( tmp != NULL );
+
+    EXPECT_STREQ( "lo1", tmp->name_.c_str() );
+    EXPECT_EQ(1, tmp->ifindex_);
+
+    // check that non-existing interfaces are not returned
+    EXPECT_EQ(static_cast<void*>(NULL), ifacemgr->getIface("wifi0") );
+
+    delete ifacemgr;
+}
+
+TEST_F(IfaceMgrTest, detectIfaces) {
+
+    // test detects that interfaces can be detected
+    // there is no code for that now, but interfaces are
+    // read from file
+    fstream fakeifaces(INTERFACE_FILE, ios::out|ios::trunc);
+    fakeifaces << "eth0 fe80::1234";
+    fakeifaces.close();
+
+    // this is not usable on systems that don't have eth0
+    // interfaces. Nevertheless, this fake interface should
+    // be on list, but if_nametoindex() will fail.
+
+    NakedIfaceMgr * ifacemgr = new NakedIfaceMgr();
+
+    ASSERT_TRUE( ifacemgr->getIface("eth0") != NULL );
+
+    IfaceMgr::Iface * eth0 = ifacemgr->getIface("eth0");
+
+    // there should be one address
+    EXPECT_EQ(1, eth0->addrs_.size());
+
+    IOAddress * addr = &(*eth0->addrs_.begin());
+    ASSERT_TRUE( addr != NULL );
+
+    EXPECT_STREQ( "fe80::1234", addr->toText().c_str() );
+
+    delete ifacemgr;
+}
+
+// TODO: disabled due to other naming on various systems
+// (lo in Linux, lo0 in BSD systems)
+// Fix for this is available on 1186 branch, will reenable
+// this test once 1186 is merged
+TEST_F(IfaceMgrTest, DISABLED_sockets) {
+    // testing socket operation in a portable way is tricky
+    // without interface detection implemented
+
+    NakedIfaceMgr * ifacemgr = new NakedIfaceMgr();
+
+    IOAddress loAddr("::1");
+
+    // bind multicast socket to port 10547
+    int socket1 = ifacemgr->openSocket(LOOPBACK, loAddr, 10547);
+    EXPECT_GT(socket1, 0); // socket > 0
+
+    // bind unicast socket to port 10548
+    int socket2 = ifacemgr->openSocket(LOOPBACK, loAddr, 10548);
+    EXPECT_GT(socket2, 0);
+
+    // expect success. This address/port is already bound, but
+    // we are using SO_REUSEADDR, so we can bind it twice
+    int socket3 = ifacemgr->openSocket(LOOPBACK, loAddr, 10547);
+
+    // rebinding succeeds on Linux, fails on BSD
+    // TODO: add OS-specific defines here (or modify code to
+    // behave the same way on all OSes, but that may not be
+    // possible
+    // EXPECT_GT(socket3, 0); // socket > 0
+
+    // we now have 3 sockets open at the same time. Looks good.
+
+    close(socket1);
+    close(socket2);
+    close(socket3);
+
+    delete ifacemgr;
+}
+
+// TODO: disabled due to other naming on various systems
+// (lo in Linux, lo0 in BSD systems)
+TEST_F(IfaceMgrTest, DISABLED_socketsMcast) {
+    // testing socket operation in a portable way is tricky
+    // without interface detection implemented
+
+    NakedIfaceMgr * ifacemgr = new NakedIfaceMgr();
+
+    IOAddress loAddr("::1");
+    IOAddress mcastAddr("ff02::1:2");
+
+    // bind multicast socket to port 10547
+    int socket1 = ifacemgr->openSocket(LOOPBACK, mcastAddr, 10547);
+    EXPECT_GT(socket1, 0); // socket > 0
+
+    // expect success. This address/port is already bound, but
+    // we are using SO_REUSEADDR, so we can bind it twice
+    int socket2 = ifacemgr->openSocket(LOOPBACK, mcastAddr, 10547);
+    EXPECT_GT(socket2, 0);
+
+    // there's no good way to test negative case here.
+    // we would need non-multicast interface. We will be able
+    // to iterate thru available interfaces and check if there
+    // are interfaces without multicast-capable flag.
+
+    close(socket1);
+    close(socket2);
+
+    delete ifacemgr;
+}
+
+// TODO: disabled due to other naming on various systems
+// (lo in Linux, lo0 in BSD systems)
+// Fix for this is available on 1186 branch, will reenable
+// this test once 1186 is merged
+TEST_F(IfaceMgrTest, DISABLED_sendReceive) {
+    // testing socket operation in a portable way is tricky
+    // without interface detection implemented
+
+    fstream fakeifaces(INTERFACE_FILE, ios::out|ios::trunc);
+    fakeifaces << LOOPBACK << " ::1";
+    fakeifaces.close();
+
+    NakedIfaceMgr * ifacemgr = new NakedIfaceMgr();
+
+    // let's assume that every supported OS have lo interface
+    IOAddress loAddr("::1");
+    int socket1 = ifacemgr->openSocket(LOOPBACK, loAddr, 10547);
+    int socket2 = ifacemgr->openSocket(LOOPBACK, loAddr, 10546);
+
+    ifacemgr->setSendSock(socket2);
+    ifacemgr->setRecvSock(socket1);
+
+    boost::shared_ptr<Pkt6> sendPkt(new Pkt6(128) );
+
+    // prepare dummy payload
+    for (int i=0;i<128; i++) {
+        sendPkt->data_[i] = i;
+    }
+
+    sendPkt->remote_port_ = 10547;
+    sendPkt->remote_addr_ = IOAddress("::1");
+    sendPkt->ifindex_ = 1;
+    sendPkt->iface_ = LOOPBACK;
+
+    boost::shared_ptr<Pkt6> rcvPkt;
+
+    EXPECT_EQ(true, ifacemgr->send(sendPkt));
+
+    rcvPkt = ifacemgr->receive();
+
+    ASSERT_TRUE( rcvPkt ); // received our own packet
+
+    // let's check that we received what was sent
+    EXPECT_EQ(sendPkt->data_len_, rcvPkt->data_len_);
+    EXPECT_EQ(0, memcmp(&sendPkt->data_[0], &rcvPkt->data_[0],
+                        rcvPkt->data_len_) );
+
+    EXPECT_EQ(sendPkt->remote_addr_.toText(), rcvPkt->remote_addr_.toText());
+    EXPECT_EQ(rcvPkt->remote_port_, 10546);
+
+    delete ifacemgr;
+}
+
+}

src/bin/host/Makefile.am

@@ -13,6 +13,7 @@ CLEANFILES = *.gcno *.gcda
 bin_PROGRAMS = b10-host
 b10_host_SOURCES = host.cc
 b10_host_LDADD = $(top_builddir)/src/lib/dns/libdns++.la
+b10_host_LDADD += $(top_builddir)/src/lib/util/libutil.la
 b10_host_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la
 
 man_MANS = b10-host.1

src/bin/host/b10-host.1

@@ -103,10 +103,6 @@ It doesn\'t use
 at this time\&. The default name server used is 127\&.0\&.0\&.1\&.
 .PP
 
-\fBb10\-host\fR
-does not do reverse lookups by default yet (by detecting if name is a IPv4 or IPv6 address)\&.
-.PP
-
 \fB\-p\fR
 is not a standard feature\&.
 .SH "HISTORY"

src/bin/host/b10-host.xml

@@ -176,11 +176,6 @@
     </para>
 
     <para>
-      <command>b10-host</command> does not do reverse lookups by
-      default yet (by detecting if name is a IPv4 or IPv6 address).
-    </para>
-
-    <para>
       <option>-p</option> is not a standard feature.
     </para>
   </refsect1>

src/bin/loadzone/Makefile.am

@@ -1,5 +1,6 @@
 SUBDIRS = . tests/correct tests/error
 bin_SCRIPTS = b10-loadzone
+noinst_SCRIPTS = run_loadzone.sh
 
 CLEANFILES = b10-loadzone
 

src/bin/loadzone/run_loadzone.sh.in

@@ -18,14 +18,14 @@
 PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
 export PYTHON_EXEC
 
-PYTHONPATH=@abs_top_builddir@/src/lib/python
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_builddir@/src/lib/python
 export PYTHONPATH
 
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 SET_ENV_LIBRARY_PATH=@SET_ENV_LIBRARY_PATH@
 if test $SET_ENV_LIBRARY_PATH = yes; then
-	@ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:$@ENV_LIBRARY_PATH@
+	@ENV_LIBRARY_PATH@=@abs_top_builddir@/src/lib/dns/.libs:@abs_top_builddir@/src/lib/dns/python/.libs:@abs_top_builddir@/src/lib/cryptolink/.libs:@abs_top_builddir@/src/lib/cc/.libs:@abs_top_builddir@/src/lib/config/.libs:@abs_top_builddir@/src/lib/log/.libs:@abs_top_builddir@/src/lib/util/.libs:@abs_top_builddir@/src/lib/util/io/.libs:@abs_top_builddir@/src/lib/exceptions/.libs:@abs_top_builddir@/src/lib/datasrc/.libs:$@ENV_LIBRARY_PATH@
 	export @ENV_LIBRARY_PATH@
 fi
 

src/bin/loadzone/tests/correct/Makefile.am

@@ -13,11 +13,13 @@ EXTRA_DIST += ttl2.db
 EXTRA_DIST += ttlext.db
 EXTRA_DIST += example.db
 
+noinst_SCRIPTS = correct_test.sh
+
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # TODO: maybe use TESTS?

src/bin/loadzone/tests/correct/correct_test.sh.in

@@ -18,7 +18,7 @@
 PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
 export PYTHON_EXEC
 
-PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python
 export PYTHONPATH
 
 LOADZONE_PATH=@abs_top_builddir@/src/bin/loadzone

src/bin/loadzone/tests/error/Makefile.am

@@ -12,11 +12,13 @@ EXTRA_DIST += keyerror3.db
 EXTRA_DIST += originerr1.db
 EXTRA_DIST += originerr2.db
 
+noinst_SCRIPTS = error_test.sh
+
 # If necessary (rare cases), explicitly specify paths to dynamic libraries
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # TODO: use TESTS ?

src/bin/loadzone/tests/error/error_test.sh.in

@@ -18,7 +18,7 @@
 PYTHON_EXEC=${PYTHON_EXEC:-@PYTHON@}
 export PYTHON_EXEC
 
-PYTHONPATH=@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python
+PYTHONPATH=@abs_top_builddir@/src/lib/python/isc/log_messages:@abs_top_srcdir@/src/lib/python:@abs_top_builddir@/src/lib/python
 export PYTHONPATH
 
 LOADZONE_PATH=@abs_top_builddir@/src/bin/loadzone

src/bin/msgq/Makefile.am

@@ -1,7 +1,7 @@
 SUBDIRS = . tests
 
 pkglibexecdir = $(libexecdir)/@PACKAGE@
- 
+
 pkglibexec_SCRIPTS = b10-msgq
 
 CLEANFILES = b10-msgq msgq.pyc

src/bin/msgq/msgq.py.in

@@ -28,7 +28,6 @@ import struct
 import errno
 import time
 import select
-import pprint
 import random
 from optparse import OptionParser, OptionValueError
 import isc.util.process
@@ -96,10 +95,10 @@ class MsgQ:
                                "@PACKAGE_NAME@",
                                "msgq_socket").replace("${prefix}",
                                                       "@prefix@")
-    
+
     def __init__(self, socket_file=None, verbose=False):
         """Initialize the MsgQ master.
-        
+
         The socket_file specifies the path to the UNIX domain socket
         that the msgq process listens on. If it is None, the
         environment variable BIND10_MSGQ_SOCKET_FILE is used. If that
@@ -135,7 +134,7 @@ class MsgQ:
             self.poller = select.poll()
         except AttributeError:
             self.kqueue = select.kqueue()
-    
+
     def add_kqueue_socket(self, socket, write_filter=False):
         """Add a kquque filter for a socket.  By default the read
         filter is used; if write_filter is set to True, the write
@@ -167,7 +166,7 @@ class MsgQ:
                              self.socket_file)
 
         self.listen_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
-        
+
         if os.path.exists(self.socket_file):
             os.remove(self.socket_file)
         try:
@@ -196,7 +195,7 @@ class MsgQ:
 
         if self.verbose:
             sys.stdout.write("[b10-msgq] Listening\n")
-        
+
         self.runnable = True
 
     def process_accept(self):
@@ -293,9 +292,6 @@ class MsgQ:
             sys.stderr.write("[b10-msgq] Routing decode error: %s\n" % err)
             return
 
-#        sys.stdout.write("\t" + pprint.pformat(routingmsg) + "\n")
-#        sys.stdout.write("\t" + pprint.pformat(data) + "\n")
-
         self.process_command(fd, sock, routingmsg, data)
 
     def process_command(self, fd, sock, routing, data):
@@ -357,7 +353,18 @@ class MsgQ:
         if fileno in self.sendbuffs:
             amount_sent = 0
         else:
-            amount_sent = self.__send_data(sock, msg)
+            try:
+                amount_sent = self.__send_data(sock, msg)
+            except socket.error as sockerr:
+                # in the case the other side seems gone, kill the socket
+                # and drop the send action
+                if sockerr.errno == errno.EPIPE:
+                    print("[b10-msgq] SIGPIPE on send, dropping message " +
+                          "and closing connection")
+                    self.kill_socket(fileno, sock)
+                    return
+                else:
+                    raise
 
         # Still something to send
         if amount_sent < len(msg):
@@ -448,12 +455,12 @@ class MsgQ:
 
     def run(self):
         """Process messages.  Forever.  Mostly."""
-        
+
         if self.poller:
             self.run_poller()
         else:
             self.run_kqueue()
-    
+
     def run_poller(self):
         while True:
             try:
@@ -511,7 +518,7 @@ def signal_handler(signal, frame):
 
 if __name__ == "__main__":
     def check_port(option, opt_str, value, parser):
-        """Function to insure that the port we are passed is actually 
+        """Function to insure that the port we are passed is actually
         a valid port number. Used by OptionParser() on startup."""
         intval = int(value)
         if (intval < 0) or (intval > 65535):

src/bin/msgq/tests/Makefile.am

@@ -6,7 +6,7 @@ EXTRA_DIST = $(PYTESTS)
 # required by loadable python modules.
 LIBRARY_PATH_PLACEHOLDER =
 if SET_ENV_LIBRARY_PATH
-LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$$$(ENV_LIBRARY_PATH)
+LIBRARY_PATH_PLACEHOLDER += $(ENV_LIBRARY_PATH)=$(abs_top_builddir)/src/lib/cryptolink/.libs:$(abs_top_builddir)/src/lib/dns/.libs:$(abs_top_builddir)/src/lib/dns/python/.libs:$(abs_top_builddir)/src/lib/cc/.libs:$(abs_top_builddir)/src/lib/config/.libs:$(abs_top_builddir)/src/lib/log/.libs:$(abs_top_builddir)/src/lib/util/.libs:$(abs_top_builddir)/src/lib/exceptions/.libs:$(abs_top_builddir)/src/lib/util/io/.libs:$(abs_top_builddir)/src/lib/datasrc/.libs:$$$(ENV_LIBRARY_PATH)
 endif
 
 # test using command-line arguments, so use check-local target instead of TESTS
@@ -19,7 +19,7 @@ endif
 	for pytest in $(PYTESTS) ; do \
 	echo Running test: $$pytest ; \
 	$(LIBRARY_PATH_PLACEHOLDER) \
-	env PYTHONPATH=$(abs_top_builddir)/src/bin/msgq:$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python \
+	PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/msgq \
 	BIND10_TEST_SOCKET_FILE=$(builddir)/test_msgq_socket.sock \
 	$(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \
 	done

src/bin/resolver/b10-resolver.8

@@ -2,12 +2,12 @@
 .\"     Title: b10-resolver
 .\"    Author: [FIXME: author] [see http://docbook.sf.net/el/author]
 .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
-.\"      Date: February 17, 2011
+.\"      Date: August 17, 2011
 .\"    Manual: BIND10
 .\"    Source: BIND10
 .\"  Language: English
 .\"
-.TH "B10\-RESOLVER" "8" "February 17, 2011" "BIND10" "BIND10"
+.TH "B10\-RESOLVER" "8" "August 17, 2011" "BIND10" "BIND10"
 .\" -----------------------------------------------------------------
 .\" * set default formatting
 .\" -----------------------------------------------------------------
@@ -54,7 +54,7 @@ must be either a valid numeric user ID or a valid user name\&. By default the da
 .PP
 \fB\-v\fR
 .RS 4
-Enabled verbose mode\&. This enables diagnostic messages to STDERR\&.
+Enable verbose mode\&. This sets logging to the maximum debugging level\&.
 .RE
 .SH "CONFIGURATION AND COMMANDS"
 .PP
@@ -77,6 +77,25 @@ string and
 number\&. The defaults are address ::1 port 53 and address 127\&.0\&.0\&.1 port 53\&.
 .PP
 
+
+
+
+
+
+\fIquery_acl\fR
+is a list of query access control rules\&. The list items are the
+\fIaction\fR
+string and the
+\fIfrom\fR
+or
+\fIkey\fR
+strings\&. The possible actions are ACCEPT, REJECT and DROP\&. The
+\fIfrom\fR
+is a remote (source) IPv4 or IPv6 address or special keyword\&. The
+\fIkey\fR
+is a TSIG key name\&. The default configuration accepts queries from 127\&.0\&.0\&.1 and ::1\&.
+.PP
+
 \fIretries\fR
 is the number of times to retry (resend query) after a query timeout (\fItimeout_query\fR)\&. The default is 3\&.
 .PP
@@ -88,7 +107,7 @@ to use directly as root servers to start resolving\&. The list items are the
 \fIaddress\fR
 string and
 \fIport\fR
-number\&. If empty, a hardcoded address for F\-root (192\&.5\&.5\&.241) is used\&.
+number\&. By default, a hardcoded address for l\&.root\-servers\&.net (199\&.7\&.83\&.42 or 2001:500:3::42) is used\&.
 .PP
 
 \fItimeout_client\fR
@@ -121,8 +140,7 @@ BIND 10 Guide\&.
 .PP
 The
 \fBb10\-resolver\fR
-daemon was first coded in September 2010\&. The initial implementation only provided forwarding\&. Iteration was introduced in January 2011\&.
-
+daemon was first coded in September 2010\&. The initial implementation only provided forwarding\&. Iteration was introduced in January 2011\&. Caching was implemented in February 2011\&. Access control was introduced in June 2011\&.
 .SH "COPYRIGHT"
 .br
 Copyright \(co 2010 Internet Systems Consortium, Inc. ("ISC")

src/bin/resolver/b10-resolver.xml

@@ -20,7 +20,7 @@
 <refentry>
 
   <refentryinfo>
-    <date>February 17, 2011</date>
+    <date>August 17, 2011</date>
   </refentryinfo>
 
   <refmeta>
@@ -99,11 +99,14 @@
         </listitem>
       </varlistentry>
 
+<!-- TODO: this needs to be fixed as -v on command line
+should imply stdout or stderr output also -->
+<!-- TODO: can this -v be overidden by configuration or bindctl? -->
       <varlistentry>
         <term><option>-v</option></term>
         <listitem><para>
-          Enabled verbose mode. This enables diagnostic messages to
-          STDERR.
+          Enable verbose mode.
+          This sets logging to the maximum debugging level.
         </para></listitem>
       </varlistentry>
 
@@ -147,6 +150,22 @@ once that is merged you can for instance do 'config add Resolver/forward_address
     </para>
 
     <para>
+<!-- TODO: need more explanation or point to guide. -->
+<!-- TODO: what about a netmask or cidr? -->
+<!-- TODO: document "key" -->
+<!-- TODO: where are the TSIG keys defined? -->
+<!-- TODO: key and from are mutually exclusive? what if both defined? -->
+      <varname>query_acl</varname> is a list of query access control
+      rules. The list items are the <varname>action</varname> string
+      and the <varname>from</varname> or <varname>key</varname> strings.
+      The possible actions are ACCEPT, REJECT and DROP.
+      The <varname>from</varname> is a remote (source) IPv4 or IPv6
+      address or special keyword.
+      The <varname>key</varname> is a TSIG key name.
+      The default configuration accepts queries from 127.0.0.1 and ::1.
+    </para>
+
+    <para>
       <varname>retries</varname> is the number of times to retry
       (resend query) after a query timeout
       (<varname>timeout_query</varname>).
@@ -159,8 +178,10 @@ once that is merged you can for instance do 'config add Resolver/forward_address
       root servers to start resolving.
       The list items are the <varname>address</varname> string
       and <varname>port</varname> number.
-      If empty, a hardcoded address for F-root (192.5.5.241) is used.
+      By default, a hardcoded address for l.root-servers.net
+      (199.7.83.42 or 2001:500:3::42) is used.
     </para>
+<!-- TODO: this is broken, see ticket #1184 -->
 
     <para>
       <varname>timeout_client</varname> is the number of milliseconds
@@ -234,7 +255,8 @@ once that is merged you can for instance do 'config add Resolver/forward_address
       The <command>b10-resolver</command> daemon was first coded in
       September 2010. The initial implementation only provided
       forwarding. Iteration was introduced in January 2011.
-<!-- TODO: document when caching was added -->
+      Caching was implemented in February 2011.
+      Access control was introduced in June 2011.
 <!-- TODO: document when validation was added -->
     </para>
   </refsect1>

src/bin/resolver/resolver.cc

@@ -520,7 +520,8 @@ ResolverImpl::processNormalQuery(const IOMessage& io_message,
     const Client client(io_message);
     const BasicAction query_action(
         getQueryACL().execute(acl::dns::RequestContext(
-                                  client.getRequestSourceIPAddress())));
+                                  client.getRequestSourceIPAddress(),
+                                  query_message->getTSIGRecord())));
     if (query_action == isc::acl::REJECT) {
         LOG_INFO(resolver_logger, RESOLVER_QUERY_REJECTED)
             .arg(question->getName()).arg(qtype).arg(qclass).arg(client);
@@ -539,7 +540,7 @@ ResolverImpl::processNormalQuery(const IOMessage& io_message,
     // ACL passed.  Reject inappropriate queries for the resolver.
     if (qtype == RRType::AXFR()) {
         if (io_message.getSocket().getProtocol() == IPPROTO_UDP) {
-            // Can't process AXFR request receoved over UDP
+            // Can't process AXFR request received over UDP
             LOG_DEBUG(resolver_logger, RESOLVER_DBG_PROCESS, RESOLVER_AXFR_UDP);
             makeErrorMessage(query_message, answer_message, buffer,
                              Rcode::FORMERR());

src/bin/resolver/resolver_log.h

@@ -23,20 +23,20 @@
 /// Defines the levels used to output debug messages in the resolver.  Note that
 /// higher numbers equate to more verbose (and detailed) output.
 
-// Initialization
-const int RESOLVER_DBG_INIT = 10;
+// Initialization and shutdown of the resolver.
+const int RESOLVER_DBG_INIT = DBGLVL_START_SHUT;
 
 // Configuration messages
-const int RESOLVER_DBG_CONFIG = 30;
+const int RESOLVER_DBG_CONFIG = DBGLVL_COMMAND;
 
 // Trace sending and receiving of messages
-const int RESOLVER_DBG_IO = 50;
+const int RESOLVER_DBG_IO = DBGLVL_TRACE_BASIC;
 
 // Trace processing of messages
-const int RESOLVER_DBG_PROCESS = 70;
+const int RESOLVER_DBG_PROCESS = DBGLVL_TRACE_DETAIL;
 
 // Detailed message information
-const int RESOLVER_DBG_DETAIL = 90;
+const int RESOLVER_DBG_DETAIL = DBGLVL_TRACE_DETAIL_DATA;
 
 
 /// \brief Resolver Logger

src/bin/resolver/resolver_messages.mes

@@ -78,7 +78,7 @@ specified, it will appear once for each address.
 % RESOLVER_FORWARD_QUERY processing forward query
 This is a debug message indicating that a query received by the resolver
 has passed a set of checks (message is well-formed, it is allowed by the
-ACL, it is a supported opcode etc.) and is being forwarded to upstream
+ACL, it is a supported opcode, etc.) and is being forwarded to upstream
 servers.
 
 % RESOLVER_HEADER_ERROR message received, exception when processing header: %1
@@ -116,7 +116,7 @@ so is returning a REFUSED response to the sender.
 % RESOLVER_NORMAL_QUERY processing normal query
 This is a debug message indicating that the query received by the resolver
 has passed a set of checks (message is well-formed, it is allowed by the
-ACL, it is a supported opcode etc.) and is being processed the resolver.
+ACL, it is a supported opcode, etc.) and is being processed by the resolver.
 
 % RESOLVER_NOTIFY_RECEIVED NOTIFY arrived but server is not authoritative
 The resolver has received a NOTIFY message.  As the server is not

src/bin/resolver/tests/Makefile.am

@@ -60,6 +60,4 @@ run_unittests_CXXFLAGS += -Wno-unused-parameter
 endif
 endif
 
-
-
 noinst_PROGRAMS = $(TESTS)

src/bin/resolver/tests/resolver_config_unittest.cc

@@ -72,7 +72,8 @@ protected:
                                           IOSocket::getDummyUDPSocket(),
                                           *endpoint));
         client.reset(new Client(*query_message));
-        request.reset(new RequestContext(client->getRequestSourceIPAddress()));
+        request.reset(new RequestContext(client->getRequestSourceIPAddress(),
+                                         NULL));
         return (*request);
     }
     void invalidTest(const string &JSON, const string& name);