Merge #2133

doc/guide/bind10-guide.xml

@@ -1485,134 +1485,154 @@ can use various data source backends.
 
     </section>
 
-    <section>
+    <section id='datasrc'>
       <title>Data Source Backends</title>
 
+      <para>
+        Bind 10 has the concept of data sources. A data source is a place
+        where authoritative zone data reside and where they can be served
+        from. This can be a master file, a database or something completely
+        different.
+      </para>
+
+      <para>
+        Once a query arrives, <command>b10-auth</command> goes through a
+        configured list of data sources and finds the one containing a best
+        matching zone. From the equally good ones, the first one is taken.
+        This data source is then used to answer the query.
+      </para>
+
       <note><para>
-        For the development prototype release, <command>b10-auth</command>
-        supports a SQLite3 data source backend and in-memory data source
-        backend.
+        In the development prototype release, <command>b10-auth</command>
+        can serve data from a SQLite3 data source backend and from master
+        files.
         Upcoming versions will be able to use multiple different
         data sources, such as MySQL and Berkeley DB.
       </para></note>
 
-
       <para>
-        By default, the SQLite3 backend uses the data file located at
+        The configuration is located in data_sources/classes. Each item there
+        represents one RR class and a list used to answer queries for that
+        class. The default contains two classes. The CH class contains a static
+        data source &mdash; one that serves things like
+        <quote>AUTHORS.BIND.</quote>. The IN class contains single SQLite3
+        data source with database file located at
         <filename>/usr/local/var/bind10-devel/zone.sqlite3</filename>.
-        (The full path is what was defined at build configure time for
-        <option>--localstatedir</option>.
-        The default is <filename>/usr/local/var/</filename>.)
-	This data file location may be changed by defining the
-	<quote>database_file</quote> configuration.
       </para>
 
-      <section id="in-memory-datasource">
-	<title>In-memory Data Source</title>
-
-	<para>
-<!--	  How to configure it. -->
-	  The following commands to <command>bindctl</command>
-	  provide an example of configuring an in-memory data
-	  source containing the <quote>example.com</quote> zone
-	  with the zone file named <quote>example.com.zone</quote>:
-
-<!--
-	  <screen>&gt; <userinput> config set Auth/datasources/ [{"type": "memory", "zones": [{"origin": "example.com", "file": "example.com.zone"}]}]</userinput></screen>
--->
-
-          <screen>&gt; <userinput>config add Auth/datasources</userinput>
-&gt; <userinput>config set Auth/datasources[0]/type "<option>memory</option>"</userinput>
-&gt; <userinput>config add Auth/datasources[0]/zones</userinput>
-&gt; <userinput>config set Auth/datasources[0]/zones[0]/origin "<option>example.com</option>"</userinput>
-&gt; <userinput>config set Auth/datasources[0]/zones[0]/file "<option>example.com.zone</option>"</userinput>
-&gt; <userinput>config commit</userinput></screen>
-
-	  The authoritative server will begin serving it immediately
-	  after the zone data is loaded from the master text file.
-	</para>
-
-      </section>
-
-      <section id="in-memory-datasource-with-sqlite3-backend">
-	<title>In-memory Data Source with SQLite3 Backend</title>
-
-	<para>
-<!--	  How to configure it. -->
-	  The following commands to <command>bindctl</command>
-	  provide an example of configuring an in-memory data
-	  source containing the <quote>example.org</quote> zone
-	  with a SQLite3 backend file named <quote>example.org.sqlite3</quote>:
-
-<!--
-	  <screen>&gt; <userinput> config set Auth/datasources/ [{"type": "memory", "zones": [{"origin": "example.org", "file": "example.org.sqlite3", "filetype": "sqlite3"}]}]</userinput></screen>
--->
-
-          <screen>&gt; <userinput>config add Auth/datasources</userinput>
-&gt; <userinput>config set Auth/datasources[1]/type "<option>memory</option>"</userinput>
-&gt; <userinput>config add Auth/datasources[1]/zones</userinput>
-&gt; <userinput>config set Auth/datasources[1]/zones[0]/origin "<option>example.org</option>"</userinput>
-&gt; <userinput>config set Auth/datasources[1]/zones[0]/file "<option>example.org.sqlite3</option>"</userinput>
-&gt; <userinput>config set Auth/datasources[1]/zones[0]/filetype "<option>sqlite3</option>"</userinput>
-&gt; <userinput>config commit</userinput></screen>
-
-	  The authoritative server will begin serving it immediately
-	  after the zone data is loaded from the database file.
-	</para>
-
-      </section>
-
-      <section id="in-memory-datasource-loading">
-	<title>Reloading an In-memory Data Source</title>
-
-	<para>
-	  Use the <command>Auth loadzone</command> command in
-	  <command>bindctl</command> to reload a changed master
-	  file into memory; for example:
+      <para>
+        Each data source has several options. The first one is
+        <varname>type</varname>, which specifies the type of data source to
+        use. Valid types include the ones listed below, but bind10 uses
+        dynamically loaded modules for them, so there may be more in your
+        case. This option is mandatory.
+      </para>
 
-	  <screen>&gt; <userinput>Auth loadzone origin="example.com"</userinput>
-</screen>
+      <para>
+        Another option is <varname>params</varname>. This option is type
+        specific; it holds different data depending on the type
+        above. Also, depending on the type, it could be possible to omit it.
+      </para>
 
-	</para>
+      <para>
+        There are two options related to the so-called cache. If you enable
+        cache, zone data from the data source are loaded into memory.
+        Then, when answering a query, <command>b10-auth</command> looks
+        into the memory only instead of the data source, which speeds
+        answering up. The first option is <varname>cache-enable</varname>,
+        a boolean value turning the cache on and off (off is the default).
+        The second one, <varname>cache-zones</varname>, is a list of zone
+        origins to load into in-memory. Remember that zones in the data source
+        not listed here will not be loaded and will not be available at all.
+      </para>
 
-<!--
+      <section id='datasource-types'>
+        <title>Data source types</title>
         <para>
-          The <varname>file</varname> may be an absolute path to the
-          master zone file or it is relative to the directory BIND 10 is
-          started from.
-	</para>
--->
+          As mentioned, the type used by default is <quote>sqlite3</quote>.
+          It has single configuration option inside <varname>params</varname>
+          &mdash; <varname>database_file</varname>, which contains the path
+          to the sqlite3 file containing the data.
+        </para>
 
+        <para>
+          Another type is called <quote>MasterFiles</quote>. This one is
+          slightly special. The data are stored in RFC1034 master files.
+          Because answering directly from them would be impractical,
+          this type mandates the cache to be enabled. Also, the list of
+          zones (<varname>cache-zones</varname>) should be omitted. The
+          <varname>params</varname> is a dictionary mapping from zone
+          origins to the files they reside in.
+        </para>
       </section>
-      <section id="in-memory-datasource-disabling">
-	<title>Disabling In-memory Data Sources</title>
 
+      <section id='datasrc-examples'>
+        <title>Examples</title>
         <para>
-	By default, the memory data source is disabled; it must be
-	configured explicitly.  To disable all the in-memory zones,
-	specify a null list for <varname>Auth/datasources</varname>:
+          As this is one of the more complex configurations of Bind10,
+          we show some examples. They all assume they start with default
+          configuration.
+        </para>
 
-<!-- TODO: this assumes that Auth/datasources is for memory only -->
+        <para>
+          First, let's disable the static data source
+          (<quote>VERSION.BIND</quote> and friends). As it is the only
+          data source in the CH class, we can remove the whole class.
 
-	  <screen>&gt; <userinput>config set Auth/datasources/ []</userinput>
+          <screen>&gt; <userinput>config remove data_sources/classes CH</userinput>
 &gt; <userinput>config commit</userinput></screen>
-	</para>
+        </para>
 
-	<para>
-          The following example stops serving a specific zone:
+        <para>
+          Another one, let's say our default data source contains zones
+          <quote>example.org.</quote> and <quote>example.net.</quote>.
+          We want them to be served from memory to make the answering
+          faster.
 
-	  <screen>&gt; <userinput>config remove Auth/datasources[<option>0</option>]/zones[<option>0</option>]</userinput>
+          <screen>&gt; <userinput>config set data_sources/classes/IN[0]/cache-enable true</userinput>
+&gt; <userinput>config add data_sources/classes/IN[0]/cache-zones example.org.</userinput>
+&gt; <userinput>config add data_sources/classes/IN[0]/cache-zones example.net.</userinput>
 &gt; <userinput>config commit</userinput></screen>
 
-	  (Replace the list number(s) in
-	  <varname>datasources[<replaceable>0</replaceable>]</varname>
-	  and/or <varname>zones[<replaceable>0</replaceable>]</varname>
-	  for the relevant zone as needed.)
+          Now every time the zone in the data source is changed by the
+          operator, Bind10 needs to be told to reload it, by
+          <screen>&gt; <userinput>Auth loadzone example.org</userinput></screen>
+          You don't need to do this when the zone is modified by
+          XfrIn, it does so automatically.
+        </para>
+
+        <para>
+          Now, the last example is when there are master files we want to
+          serve in addition to whatever is inside the sqlite3 database.
 
-	</para>
+          <screen>&gt; <userinput>config add data_sources/classes/IN</userinput>
+&gt; <userinput>config set data_sources/classes/IN[1]/type MasterFiles</userinput>
+&gt; <userinput>config set data_sources/classes/IN[1]/cache-enable true</userinput>
+&gt; <userinput>config set data_sources/classes/IN[1]/params { "example.org": "/path/to/example.org", "example.com": "/path/to/example.com" }</userinput>
+&gt; <userinput>config commit</userinput></screen>
 
+          Unfortunately, due to current technical limitations, the params must
+          be set as one JSON blob, it can't be edited in
+          <command>bindctl</command>. To reload a zone, you the same command
+          as above.
+        </para>
       </section>
 
+      <note>
+      <para>
+        There's also <varname>Auth/database_file</varname> configuration
+        variable, pointing to a sqlite3 database file. This is no longer
+        used by <command>b10-auth</command>, but it is left in place for
+        now, since other modules use it. Once <command>b10-xfrin</command>,
+        <command>b10-xfrout</command> and <command>b10-ddns</command>
+        are ported to the new configuration, this will disappear. But for
+        now, make sure that if you use any of these modules, the new
+        and old configuration correspond. The defaults are consistent, so
+        unless you tweaked either the new or the old configuration, you're
+        good.
+      </para>
+      </note>
+
     </section>
 
     <section>
@@ -1854,7 +1874,7 @@ http://bind10.isc.org/wiki/ScalableZoneLoadDesign#a7.2UpdatingaZone
 	The administrator doesn't have to do anything for
 	<command>b10-auth</command> to serve the new version of the
 	zone, except for the configuration such as the one described in
-	<xref linkend="in-memory-datasource-with-sqlite3-backend" />.
+	<xref linkend="datasrc" />.
       </para>
     </section>
 
@@ -1965,11 +1985,11 @@ what is XfroutClient xfr_client??
       notify <command>b10-xfrout</command> so that other secondary
       servers will be notified via the DNS NOTIFY protocol.
       In addition, if <command>b10-auth</command> serves the updated
-      zone from its in-memory cache (as described in
-      <xref linkend="in-memory-datasource-with-sqlite3-backend" />),
+      zone (as described in
+      <xref linkend="datasrc" />),
       <command>b10-ddns</command> will also
       notify <command>b10-auth</command> so that <command>b10-auth</command>
-      will re-cache the updated zone content.
+      will re-cache the updated zone content if necessary.
     </para>
 
     <para>
@@ -2548,7 +2568,7 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";</screen>
     </section>
 
     <section id="dhcp4-limit">
-      <title>DHCPv4 Server Limitations</title> 
+      <title>DHCPv4 Server Limitations</title>
       <para>These are the current limitations of the DHCPv4 server
       software. Most of them are reflections of the early stage of
       development and should be treated as <quote>not implemented

src/bin/cfgmgr/plugins/Makefile.am

@@ -3,7 +3,7 @@ SUBDIRS = tests
 EXTRA_DIST = README logging.spec tsig_keys.spec
 
 datasrc.spec: datasrc.spec.pre
-	$(SED) -e "s|@@PKGDATADIR@@|$(pkgdatadir)|" datasrc.spec.pre >$@
+	$(SED) -e "s|@@PKGDATADIR@@|$(pkgdatadir)|;s|@@LOCALSTATEDIR@@|$(localstatedir)|" datasrc.spec.pre >$@
 
 config_plugindir = @prefix@/share/@PACKAGE@/config_plugins
 config_plugin_DATA = logging.spec tsig_keys.spec datasrc.spec

src/bin/cfgmgr/plugins/datasrc.spec.pre.in

@@ -8,6 +8,14 @@
                 "item_type": "named_set",
                 "item_optional": false,
                 "item_default": {
+                    "IN": [
+                        {
+                            "type": "sqlite3",
+                            "params": {
+                                "database_file": "@@LOCALSTATEDIR@@/@PACKAGE@/zone.sqlite3"
+                            }
+                        }
+                    ],
                     "CH": [
                         {
                             "type": "static",

tests/lettuce/configurations/bindctl/bindctl.config.orig

@@ -14,6 +14,9 @@
             "address": "127.0.0.1"
         } ]
     },
+    "data_sources": {
+        "classes": {}
+    },
     "Boss": {
         "components": {
             "b10-cmdctl": { "special": "cmdctl", "kind": "needed" }

tests/lettuce/configurations/bindctl_commands.config.orig

@@ -14,6 +14,9 @@
             "address": "127.0.0.1"
         } ]
     },
+    "data_sources": {
+        "classes": {}
+    },
     "StatsHttpd": {
         "listen_on": [ {
             "port": 47811,