kea/doc/guide/logging.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
<!ENTITY mdash  "&#x2014;" >
]>

  <!-- Note: Please use the following terminology:
       - daemon - one process (e.g. kea-dhcp4)
       - component - one piece of code within a daemon (e.g. libdhcp or hooks)
       - server - currently equal to daemon, but the difference will be more
                  prominent once we add client or relay support
       - logger - one instance of isc::log::Logger
       - structure - an element in config file (e.g. "Dhcp4")

       Do not use:
       - module => daemon
    -->

  <chapter id="logging">
    <title>Logging</title>

    <section>
      <title>Logging Configuration</title>

      <para>
        During its operation Kea may produce many messages. They differ in
        severity (some are more important than others) and source (some are
        produced by specific components, e.g. hooks). It is useful to understand
        which log messages are needed and which are not and configure your
        logging appropriately. For example, debug level messages can be safely
        ignored in a typical deployment. They are, however, very useful when
        debugging a problem.
      </para>

      <para>
        The logging system in Kea is configured through the
        <replaceable>Logging</replaceable> structure in your configuration
        file. All daemons (e.g. DHCPv4 and DHCPv6 servers) will use the
        configuration in the <replaceable>Logging</replaceable> structure to see
        what should be logged and to where. This allows for sharing identical
        logging configuration between daemons.
      </para>

      <section>
        <title>Loggers</title>

        <para>
          Within Kea, a message is logged through an entity called a
          "logger". Different components log messages through different
          loggers, and each logger can be configured independently of
          one another. Some components, in particular the DHCP server
          processes, may use multiple loggers to log messages pertaining
          to different logical functions of the component. For example,
          the DHCPv4 server is using one logger for messages
          pertaining to packet reception and transmission, another
          logger for messages related to lease allocation and so on.
          Some of the libraries used by the Kea servers, e.g. libdhcpsrv
          or libhooks library, use their own loggers.
        </para>

        <para>
          Users implementing hooks libraries (code attached to the server at
          runtime) are responsible for creating the loggers used by those
          libraries. Such loggers should have unique names, different
          from the logger names used by Kea. In this way the
          messages emitted from the hooks library can be distingued from
          messages issued by the core Kea code. Unique names also allow
          the loggers to be configured independently of loggers used
          by Kea.  Whenever it makes sense, a hook library can use multiple
          loggers to log messages pertaining to different logical parts
          of the library.
        </para>

        <para>
          In the <quote>Logging</quote> structure of a configuration file
          you can specify the configuration for zero or more loggers
          (including loggers used by the proprietary hooks libraries). If
          there are no loggers specified, the code will use default values which
          cause Kea to log messages on at least INFO severity to standard
          output.
          <!-- @todo: add reference to section about controlling default
          behavior with env. variables, after #3591 is merged. -->
        </para>

        <para>
          The three most important elements of a logger configuration
          are the <option>name</option> (the component that is
          generating the messages), the <option>severity</option>
          (what to log), and the <option>output_options</option>
          (where to log).
        </para>

        <section>
          <title>name (string)</title>

          <para>
            Each logger in the system has a name, the name being that of the
            component binary file using it to log messages. For instance, if you
            want to configure logging for the DHCPv4 server, you add an entry
            for a logger named <quote>kea-dhcp4</quote>. This configuration will
            then be used by the loggers in the DHCPv4 server, and all the
            libraries used by it (unless a library defines its own logger and
            there is specific logger configuration that applies to that logger).
          </para>

          <para>
            When diagnosing the problem with the server's operation, it is often
            desired to use the DEBUG logging level to obtain the verbose output
            from the server and libraries it uses. However, high verbosity may
            be an overkill for the logging system in cases when the server
            is processing high volume traffic. To mitigate this problem, Kea
            is using multiple loggers, which can be configured independently
            and which are responsible for logging messages from different
            functional parts of the server. If the user, trying to diagnose the
            problem, has a reasonably high confidence that the problem origins
            in a specific function of the server, or the problem is related
            to the specific type of operation, he may enable high verbosity
            only for the relevant logger, thus limiting the debug messages
            to the required minimum.
          </para>

          <para>
            The loggers are associated with a particular library or binary
            of Kea. However, each library or binary may (and usually does)
            include multiple loggers. For example, the DHCPv4 server binary
            contains separate loggers for: packet parsing, for dropped packets,
            for callouts etc. Each logger "derives" its configuration from the
            root logger. In the typical case, the root logger configuration
            is the only logging configuration specified in the configuration
            file. Creating a specific configuration for the selected logger,
            thus overriding the configuration settings specified in the
            root logger configuration, requires putting its configuration
            aside of the root logger's configuration with some of the
            parameters modified.
          </para>

          <para>
            To illustrate this, suppose you are using the DHCPv4 server
            with the root logger <quote>kea-dhcp4</quote> logging at the
            INFO level. In order to enable DEBUG verbosity for the DHCPv4
            packet drops, you must create configuration entry for the
            logger called <quote>kea-dhcp4.bad-packets</quote> and specify
            severity DEBUG for this logger. All other configuration
            parameters may be omited for this logger if the logger should
            use the default values specified in the root logger's
            configuration.
          </para>

        <!-- we don't support asterisk anymore.
        <para>
          One special case is that of a component name of <quote>*</quote>
          (asterisks), which is interpreted as <emphasis>any</emphasis>
          component. You can set global logging options by using this,
          including setting the logging configuration for a library
          that is used by multiple daemons (e.g. <quote>*.config</quote>
          specifies the configuration library code in whatever
          daemon is using it).
        </para> -->

          <para>
            If there are multiple logger specifications in the configuration
            that might match a particular logger, the specification with the
            more specific logger name takes precedence. For example, if there
            are entries for both <quote>kea-dhcp4</quote> and
            <quote>kea-dhcp4.dhcpsrv</quote>, the DHCPv4 server &mdash; and all
            libraries it uses that are not dhcpsrv &mdash; will log messages
            according to the configuration in the first entry
            (<quote>kea-dhcp4</quote>).
          </para>

          <para>
            Currently defined loggers are:
          </para>

        <itemizedlist>
          <listitem>
            <simpara><command>kea-dhcp4</command> - this is the root logger for
            the DHCPv4 server. All components used by the DHCPv4 server inherit
            the settings from this logger if there is no specialized logger
            provided.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.alloc-engine</command> - this is the
            logger used by the lease allocation engine, which is responsible
            for managing leases in the lease database, i.e. create, modify
            and remove DHCPv4 leases as a result of processing messages from
            the clients.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.bad-packets</command> - this is the
            logger used by the DHCPv4 server deamon for logging inbound client
            packets that were dropped or to which the server responded with a
            DHCPNAK. The allows adminstrators to configure a separate log
            output that contains only packet drop and reject entries.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.callouts</command> - this logger is used
            to log messages pertaining to the callouts registration and execution
            for the particular hook point.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.commands</command> - this logger is used
            to log messages relating to the handling of commands received by the
            the DHCPv4 server over the command channel.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.ddns</command> - this logger is used by
            the DHCPv4 server to log messages related to the Client FQDN and
            Hostname option processing. It also includes log messages
            related to the relevant DNS updates.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.dhcp4</command> - this is the logger
            by the DHCPv4 server deamon to log basic operations.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.dhcpsrv</command> - this is a base
            logger for the libdhcpsrv library.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.hooks</command> - this logger is used
            to log messages related to management of hooks libraries, e.g.
            registration and deregistration of the libraries, and to the
            initialization of the callouts execution for various hook points
            within the DHCPv4 server.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.hosts</command> - this logger is used
            within the libdhcpsrv and it logs messages related to the management
            of the DHCPv4 host reservations, i.e. retrieval of the resevations
            and adding new reservations.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.leases</command> - this logger is used
            by the DHCPv4 server to log messages related to the lease allocation.
            The messages include detailed information about the allocated or
            offered leases, errors during the lease allocation etc.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.options</command> - this logger is
            used by the DHCPv4 server to log messages related to processing
            of the options in the DHCPv4 messages, i.e. parsing options,
            encoding options into on-wire format and packet classification
            using options contained in the received packets.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.packets</command> - this logger
            is mostly used to log messages related to transmission of the DHCPv4
            packets, i.e. packet reception and sending a response. Such messages
            include the information about the source and destination IP addresses
            and interfaces used to transmit packets. This logger is also used
            to log messages related to subnet selection, as this selection is
            usually based on the IP addresses and/or interface names, which can
            be retrieved from the received packet, even before the DHCPv4 message
            carried in the packet is parsed.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6</command> - this is the root logger for
            the DHCPv6 server. All components used by the DHCPv6 server inherit
            the settings from this logger if there is no specialized logger
            provided.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.alloc-engine</command> - this is the
            logger used by the lease allocation engine, which is responsible
            for managing leases in the lease database, i.e. create, modify
            and remove DHCPv6 leases as a result of processing messages from
            the clients.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.bad-packets</command> - this is the
            logger used by the DHCPv6 server deamon for logging inbound client
            packets that were dropped.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.callouts</command> - this logger is used
            to log messages pertaining to the callouts registration and execution
            for the particular hook point.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.commands</command> - this logger is used
            to log messages relating to the handling of commands received by the
            the DHCPv6 server over the command channel.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.ddns</command> - this logger is used by
            the DHCPv6 server to log messages related to the Client FQDN option
            processing. It also includes log messages related to the relevant
            DNS updates.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.dhcp6</command> - this is the logger
            used DHCPv6 server deamon to log basic operations.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.dhcpsrv</command> - this is a base
            logger for the libdhcpsrv library.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.hooks</command> - this logger is used
            to log messages related to management of hooks libraries, e.g.
            registration and deregistration of the libraries, and to the
            initialization of the callouts execution for various hook points
            within the DHCPv6 server.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.hosts</command> - this logger is used
            within the libdhcpsrv and it logs messages related to the management
            of the DHCPv6 host reservations, i.e. retrieval of the resevations
            and adding new reservations.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.leases</command> - this logger is used
            by the DHCPv6 server to log messages related to the lease allocation.
            The messages include detailed information about the allocated or
            offered leases, errors during the lease allocation etc.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.options</command> - this logger is
            used by the DHCPv6 server to log messages related to processing
            of the options in the DHCPv6 messages, i.e. parsing options,
            encoding options into on-wire format and packet classification
            using options contained in the received packets.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.packets</command> - this logger
            is mostly used to log messages related to transmission of the DHCPv6
            packets, i.e. packet reception and sending a response. Such messages
            include the information about the source and destination IP addresses
            and interfaces used to transmit packets. This logger is also used
            to log messages related to subnet selection, as this selection is
            usually based on the IP addresses and/or interface names, which can
            be retrieved from the received packet, even before the DHCPv6 message
            carried in the packet is parsed.
            </simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp-ddns</command> - this is the root logger for
            the kea-dhcp-ddns deamon. All components used by this deamon inherit
            the settings from this logger if there is no specialized logger
            provided.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp-ddns.dhcpddns</command> - this is the logger
            used by the kea-dhcp-ddns deamon for logging configuration and global
            event information.  This logger does not specify logging settings
            for libraries used by the deamon.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp-ddns.dhcp-to-d2</command> - this is the logger
            used by the kea-dhcp-ddns deamon for logging information about events
            dealing with receving messages from the DHCP servers and adding them
            to the queue for processing.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp-ddns.d2-to-dns</command> - this is the logger
            used by the kea-dhcp-ddns deamon for logging information about events
            dealing with sending and receiving messages with the DNS servers.
            </simpara>
          </listitem>

        </itemizedlist>

        <para>
          Note that user-defined hook libraries should not use any of those
          loggers, and should define new loggers with names that correspond to
          the libraries using them. Suppose that the user created the library called
          <quote>libpacket-capture</quote> to dump packets received and
          transmitted by the server to the file. The appropriate name for the
          logger could be <command>kea-dhcp4.packet-capture</command>. Note
          that the hook library implementor only specifies the second part
          of this name, i.e. <quote>packet-capture</quote>. The first part is
          a root logger name and is prepended by the Kea logging system.
          It is also important to note that since this new logger is a child
          of a root logger, it inherits the configuration from the root logger,
          unless there is a separate configuration entry for the child logger
          which overrides the default configuration.
        </para>

        <para>
          The list of loggers above excludes any loggers implemented in hooks
          libraries. Please consult the documentation of the specific hooks
          libraries for the names of loggers they define.
        </para>

        <para>Additional loggers may be defined in the future. The easiest
        way to find out the logger name is to configure all logging to go
        to a single destination and look for specific logger names. See
        <xref linkend="logging-message-format"/> for details.</para>
        </section>

        <section>
          <title>severity (string)</title>

        <para>
          This specifies the category of messages logged.
          Each message is logged with an associated severity which
          may be one of the following (in descending order of
          severity):
        </para>

        <itemizedlist>
          <listitem>
            <simpara> FATAL </simpara>
          </listitem>

          <listitem>
            <simpara> ERROR </simpara>
          </listitem>

          <listitem>
            <simpara> WARN </simpara>
          </listitem>

          <listitem>
            <simpara> INFO </simpara>
          </listitem>

          <listitem>
            <simpara> DEBUG </simpara>
          </listitem>
        </itemizedlist>

        <para>

          When the severity of a logger is set to one of these
          values, it will only log messages of that severity, and
          the severities above it. The severity may also be set to
          NONE, in which case all messages from that logger are
          inhibited.

<!-- TODO: worded wrong? If I set to INFO, why would it show DEBUG which is literally below in that list? -->

        </para>

        </section>

        <section>
          <title>output_options (list)</title>

        <para>

          Each logger can have zero or more
          <option>output_options</option>. These specify where log
          messages are sent. These are explained in detail below.

        </para>

        <para>

          The other options for a logger are:

        </para>

        </section>

        <section>
          <title>debuglevel (integer)</title>

        <para>

          When a logger's severity is set to DEBUG, this value
          specifies what debug messages should be printed. It ranges
          from 0 (least verbose) to 99 (most verbose).
        </para>


<!-- TODO: complete this sentence:

          The general classification of debug message types is

TODO; there's a ticket to determine these levels, see #1074

 -->

        <para>

          If severity for the logger is not DEBUG, this value is ignored.

        </para>

        </section>
      </section>

      <section>
        <title>Output Options</title>

        <para>

          The main settings for an output option are the
          <option>destination</option> and a value called
          <option>output</option>, the meaning of which depends on
          the destination that is set.

        </para>

        <section>
          <title>destination (string)</title>

          <para>

            The destination is the type of output. It can be one of:

          </para>

          <itemizedlist>

            <listitem>
              <simpara> console </simpara>
          </listitem>

            <listitem>
              <simpara> file </simpara>
          </listitem>

            <listitem>
              <simpara> syslog </simpara>
            </listitem>

          </itemizedlist>

        </section>

        <section>
          <title>output (string)</title>

        <para>
          This value determines the type of output. There are several
          special values allowed here: <command>stdout</command> (messages
          are printed on standard output), <command>stderr</command>
          (messages are printed on stderr), <command>syslog</command> (messages
          are logged to syslog using default name, <command>syslog:name</command>
          (messages are logged to syslog using specified name). Any other
          value is interpreted as a filename that the logs should be written to.
        </para>

        <para>

          The other options for <option>output_options</option> are:

        </para>

        <section>
          <title>flush (true of false)</title>

          <para>
            Flush buffers after each log message. Doing this will
            reduce performance but will ensure that if the program
            terminates abnormally, all messages up to the point of
            termination are output. Default is true.
          </para>

        </section>

        <section>
          <title>maxsize (integer)</title>

          <para>
            Only relevant when destination is file, this is maximum
            file size of output files in bytes. When the maximum
            size is reached, the file is renamed and a new file opened.
            (For example, a ".1" is appended to the name &mdash;
            if a ".1" file exists, it is renamed ".2",
            etc.)
          </para>

          <para>
            If this is 0, no maximum file size is used.
          </para>

          <note>
            <simpara>
	      Due to a limitation of the underlying logging library
	      (log4cplus), rolling over the log files (from ".1" to
	      ".2", etc) may show odd results: There can be
	      multiple small files at the timing of roll over.  This
	      can happen when multiple processes try to roll
	      over the files simultaneously.
	      Version 1.1.0 of log4cplus solved this problem, so if
	      this or higher version of log4cplus is used to build
	      Kea, it shouldn't happen.  Even for older versions
	      it is normally expected to happen rarely unless the log
	      messages are produced very frequently by multiple
	      different processes.
	    </simpara>
	  </note>

        </section>

        <section>
          <title>maxver (integer)</title>

          <para>
            Maximum number of old log files to keep around when
            rolling the output file. Only relevant when
            <option>output</option> is <quote>file</quote>.
          </para>

        </section>

      </section>

      </section>

      <section>
        <title>Example Logger Configurations</title>

        <para>
          In this example we want to set the global logging to
          write to the console using standard output.
        </para>

<screen><userinput>
"Logging": {
    "loggers": [
        {
            "name": "kea-dhcp4",
            "output_options": [
                {
                    "output": "stdout"
                }
            ],
            "severity": "WARN"
        }
    ]
}
</userinput>
</screen>

<para>In this second example, we want to store debug log messages
in a file that is at most 2MB and keep up to 8 copies of old logfiles.
Once the logfile grows to 2MB, it will be renamed and a new file
file be created.</para>

<screen><userinput>
"Logging": {
    "loggers": [
        {
            "name": "kea-dhcp6",
            "output_options": [
                {
                    "output": "/var/log/kea-debug.log",
                    "maxver": 8,
                    "maxsize": 204800,
                    "flush": true
                }
            ],
            "severity": "DEBUG",
            "debuglevel": 99
        }
   ]
}</userinput></screen>
      </section>

    </section>

    <section id="logging-message-format">
      <title>Logging Message Format</title>

      <para>
          Each message written  to the configured logging
          destinations comprises a number of components that identify
          the origin of the message and, if the message indicates
          a problem, information about the problem that may be
          useful in fixing it.
      </para>

      <para>
          Consider the message below logged to a file:
          <screen>2014-04-11 12:58:01.005 INFO  [kea-dhcp4.dhcpsrv/27456]
    DHCPSRV_MEMFILE_DB opening memory file lease database: type=memfile universe=4</screen>
      </para>

      <para>
        Note: the layout of messages written to the system logging
        file (syslog) may be slightly different.  This message has
        been split across two lines here for display reasons; in the
        logging file, it will appear on one line.
      </para>

      <para>
        The log message comprises a number of components:

          <variablelist>
          <varlistentry>
          <term>2014-04-11 12:58:01.005</term>
<!-- TODO: timestamp repeated even if using syslog? -->
          <listitem><para>
              The date and time at which the message was generated.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>INFO</term>
          <listitem><para>
              The severity of the message.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>[kea-dhcp4.dhcpsrv/27456]</term>
          <listitem><para>
            The source of the message.  This comprises two elements:
            the Kea process generating the message (in this
            case, <command>kea-dhcp4</command>) and the component
            within the program from which the message originated
            (which is the name of the common library used by DHCP server
            implementations). The number after the slash is a process id
            (pid).
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>DHCPSRV_MEMFILE_DB</term>
          <listitem><para>
            The message identification.  Every message in Kea
            has a unique identification, which can be used as an
            index into the <ulink
            url="kea-messages.html"><citetitle>Kea Messages
            Manual</citetitle></ulink> (<ulink
            url="http://kea.isc.org/docs/kea-messages.html"
            />) from which more information can be obtained.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>opening memory file lease database: type=memfile universe=4</term>
          <listitem><para>
              A brief description.
              Within this text, information relating to the condition
              that caused the message to be logged will be included.
              In this example, the information is logged that the in-memory
              lease database backend will be used to store DHCP leases.
          </para></listitem>
          </varlistentry>
          </variablelist>
      </para>

    </section>

    <section>
      <title>Logging During Kea Startup</title>

      <para>
        The logging configuration is specified in the configuration file.
        However, when Kea starts, the file is not read until some way into the
        initialization process.  Prior to that, the logging settings are
        set to default values, although it is possible to modify some
        aspects of the settings by means of environment variables. Note
        that in the absence of any logging configuration in the configuration
        file, the settings of (possibly modified) default configuration will
        persist while the program is running.
      </para>
      <para>
        The following environment variables can be used to control the
        behavio of logging during startup:
      </para>

          <variablelist>
          <varlistentry>
          <term>KEA_LOCKFILE_DIR</term>
          <listitem><para>
              Specifies a directory where the logging system should create its
              lock file. If not specified, it is
              <replaceable>prefix</replaceable>/var/run/kea, where
              <replaceable>prefix</replaceable> defaults to /usr/local.
              This variable must not end
              with a slash. There is one special value: "none", which
              instructs Kea to not create lock file at all. This may cause
              issues if several processes log to the same file.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>KEA_LOGGER_DESTINATION</term>
          <listitem><para>
              Specifies logging output. There are several special values.
              <variablelist>
                  <varlistentry>
                      <term>stdout</term>
                      <listitem><para>
                          Log to standard output.
                      </para></listitem>
                  </varlistentry>
                  <varlistentry>
                      <term>stderr</term>
                      <listitem><para>
                          Log to standard error.
                      </para></listitem>
                  </varlistentry>
                  <varlistentry>
                      <term>syslog<optional>:<replaceable>fac</replaceable></optional></term>
                      <listitem><para>
                          Log via syslog. The optional
                          <replaceable>fac</replaceable> (which is
                          separated from the word "syslog" by a colon)
                          specifies the
                          facility to be used for the log messages. Unless
                          specified, messages will be logged using the
                          facility "local0".
                      </para></listitem>
                  </varlistentry>
              </variablelist>
              Any other value is treated as a name
              of the output file. If not specified otherwise, Kea will log to
              standard output.
          </para></listitem>
          </varlistentry>

          </variablelist>
    </section>

  </chapter>