[2776] Some design considerations for sharing a socket

Write some advantages and disadvantages of each method how to solve it.

doc/design/resolver/02-mixed-recursive-authority-setup

@@ -1,7 +1,6 @@
-02-mixed-recursive-authority-setup
+Mixed recursive & authoritative setup
+=====================================
 
-Introduction
-------------
 Ideally we will run the authoritative server independently of the
 recursive resolver.
 
@@ -9,29 +8,129 @@ We need a way to run both an authoritative and a recursive resolver on
 a single platform, listening on the same IP/port. But we need a way to
 run only one of them as well.
 
-We have 3 basic components involved in this mix:
-
-1. Authoritative zones
-2. Cached RRSETs
-3. Non-cached information
+This is mostly the same problem as we have with DDNS packets and xfr-out
+requests, but they aren't that performance sensitive as auth & resolver.
 
 There are a number of possible approaches to this:
 
-a. Make a module that includes all logic. (The BIND 9 module?)
-a'. extract major processing logic of auth into a separate library
-   (maybe loadable module) and allow the resolver to use it, probably
-   as a kind of hook.
-b. Look at authoritative server first, and pass queries to the
-   recursive component.
-c. Make a module that combines authoritative and cache. Queries not
-   found get passed to a resolver, which also has to update the cache.
-d. Have a simple "receptionist" module which knows which zones we are
-   authoritative for and sends all queries to another daemon.
-
-Stephen did some modeling work on this already. We need to understand
-the latency and throughput implications of any of these approaches.
-
-It would be nice to solve the forwarding of packets to DDNS and XfrIn
-with this too, if it worked. Currently, it is not possible to run more
-instances of these modules to divide the load and it is not possible
-to run them without Auth.
+One fat module
+--------------
+
+With some build system or dynamic linker tricks, we create three modules:
+
+ * Stand-alone auth
+ * Stand-alone resolver
+ * Compound module containing both
+
+The user then chooses either one stand alone module, or the compound one,
+depending on the requirements.
+
+Advantages
+~~~~~~~~~~
+
+ * It is easier to switch between processing and ask authoritative questions
+   from within the resolver processing.
+
+Disadvantages
+~~~~~~~~~~~~~
+
+ * The code is not separated (one bugs takes down both, admin can't see which
+   one takes how much CPU).
+ * Bind 9 does this and bind 9 code is a jungle. Maybe it's not just a
+   coincidence.
+ * Limits flexibility -- for example, we can't then decide to make the resolver
+   threaded (or we would have to make sure the auth processing doesn't break
+   with threads, which will be hard).
+
+There's also the idea of putting the auth into a loadable library and the
+resolver could load and use it somehow. But the advantages and disadvantages
+are probably the same.
+
+Auth first
+----------
+
+We do the same as with xfrout and ddns. When a query comes, it is examined and
+if the `RD` bit is set, it is forwarded to the resolver.
+
+Advantages
+~~~~~~~~~~
+
+ * Separate auth and resolver modules
+ * Minimal changes to auth
+ * No slowdown on the auth side
+
+Disadvantages
+~~~~~~~~~~~~~
+
+ * Counter-intuitive asymetric design
+ * Possible slowdown on the resolver side
+ * Resolver needs to know both modes (for running stand-alone too)
+
+There's also the possibility of the reverse -- resolver first. It may make
+more sense for performance (the more usual scenario would probably be a
+high-load resolver with just few low-volume authoritative zones). On the other
+hand, auth already has some forwarding tricks.
+
+Auth with cache
+---------------
+
+This is mostly the same as ``Auth first'', however, the cache is in the auth
+server. If it is in the cache, it is answered right away. If not, it is then
+forwarded to the resolver. The resolver then updates the cache too.
+
+Advantages
+~~~~~~~~~~
+
+ * Probably a good performance
+
+Disadvantages
+~~~~~~~~~~~~~
+
+ * Cache duplication (several auth modules, it doesn't feel like it would work
+   with shared memory without locking).
+ * Cache is probably very different from authoritative zones, it would
+   complicate auth processing.
+ * The resolver needs own copy of cache (to be able to get partial results),
+   probably a different one than the auth server.
+
+Receptionist
+------------
+
+One module does only the listening. It doesn't process the queries itself, it
+only looks into them and forwards them to the processing modules.
+
+Advantages
+~~~~~~~~~~
+
+ * Clean design with separated modules
+ * Easy to run modules stand-alone
+ * Allows for solving the xfrout & ddns forwarding without auth running
+ * Allows for views (different auths with different configurations)
+ * Allows balancing/clustering across multiple machines
+ * Easy to create new modules for different kinds of DNS handling and share
+   port with them too
+
+Disadvantages
+~~~~~~~~~~~~~
+
+ * Need to set up another module (not a problem if we have inter-module
+   dependencies in b10-init)
+ * Possible performance impact
+
+Implementation ideas
+~~~~~~~~~~~~~~~~~~~~
+
+ * Let's have a new TCP transport, where we send not only the DNS messages,
+   but also the source and destination ports and addresses (two reasons --
+   ACLs in target module and not keeping state in the receptionist). It would
+   allow for transfer of a batch of messages at once, to save some calls to
+   kernel (like a length of block of messages, it is read at once, then they
+   are all parsed one by one, the whole block of answers is sent back).
+ * A module creates a listening socket (UNIX by default) on startup and
+   contacts all the receptionists. It sends ACL match for the packets to send
+   to the module and the address of the UNIX socket. All the receptionists
+   connect to the module. This allows for auto-configuring the receptionist.
+ * The queries are sent from the receptionist in batches, the answers are sent
+   back to the receptionist in batches too.
+ * It is possible to fine-tune and use OS-specific tricks (like epoll or
+   sending multiple UDP messages by single call to sendmmsg).