kea/doc/guide/logging.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
<!ENTITY mdash  "&#x2014;" >
]>

  <!-- Note: Please use the following terminology:
       - daemon - one process (e.g. kea-dhcp4)
       - component - one piece of code within a daemon (e.g. libdhcp or hooks)
       - server - currently equal to daemon, but the difference will be more
                  prominent once we add client or relay support
       - logger - one instance of isc::log::Logger
       - structure - an element in config file (e.g. "Dhcp4")

       Do not use:
       - module => daemon
    -->

  <chapter id="logging">
    <title>Logging</title>

    <section>
      <title>Logging Configuration</title>

      <para>
        During its operation Kea may produce many messages. They differ in
        severity (some are more important than others) and source (some are
        produced by specific components, e.g. hooks). It is useful to understand
        which log messages are needed and which are not and configure your
        logging appropriately. For example, debug level messages can be safely
        ignored in a typical deployment. They are, however, very useful when
        debugging a problem.
      </para>

      <para>
        The logging system in Kea is configured through the
        <replaceable>Logging</replaceable> structure in your configuration
        file. All daemons (e.g. DHCPv4 and DHCPv6 servers) will use the
        configuration in the <replaceable>Logging</replaceable> structure to see
        what should be logged and to where. This allows for sharing identical
        logging configuration between daemons.
      </para>

      <section>
        <title>Loggers</title>

        <para>
          Within Kea, a message is logged through an entity
          called a "logger". Different parts of the code log messages
          through different loggers, and each logger can be configured
          independently of one another. For example there are different
          components that deal with hooks ("hooks" logger) and with
          DHCP engine ("dhcpsrv" logger).
        </para>

        <para>
          In the Logging structure in a configuration file you can
          specify the configuration for zero or more loggers. If there are
          no loggers specified, the code will use default values which
          cause Kea to log messages on at least INFO severity to standard
          output.
          <!-- @todo: add reference to section about controlling default
          behavior with env. variables, after #3591 is merged. -->
        </para>

        <para>
          The three most important elements of a logger configuration
          are the <option>name</option> (the component that is
          generating the messages), the <option>severity</option>
          (what to log), and the <option>output_options</option>
          (where to log).
        </para>

        <section>
          <title>name (string)</title>

          <para>
            Each logger in the system has a name, the name being that of the
            component binary file using it to log messages. For instance, if you
            want to configure logging for the DHCPv4 server, you add an entry
            for a logger named <quote>kea-dhcp4</quote>. This configuration will
            then be used by the loggers in the DHCPv4 server, and all the
            libraries used by it (unless a library defines its own logger and
            there is specific logger configuration that applies to that logger).
          </para>

          <para>
            If you want to specify logging for one specific library within a
            daemon, you set the name to
            <replaceable>daemon.library</replaceable>.  For example, the logger
            used by the code from libdhcpsrv used in kea-dhcp4 binary has the
            full name of <quote>kea-dhcp4.dhcpsrv</quote>. If there is no entry
            in Logging for a particular library, it will use the configuration
            given for the whole daemon.
          </para>

          <para>
            To illustrate this, suppose you want the dhcpsrv library
            to log messages of severity DEBUG, and the rest of the
            DHCPv4 server code to log messages of severity INFO. To achieve
            this you specify two loggers, one with the name
            <quote>kea-dhcp4</quote> and severity INFO, and one with
            the name <quote>kea-dhcp4.dhcpsrv</quote> with severity
            DEBUG. As there are no entries for other libraries,
            they will use the configuration for the daemon
            (<quote>kea-dhcp4</quote>), so giving the desired behavior.
          </para>

        <!-- we don't support asterisk anymore.
        <para>
          One special case is that of a component name of <quote>*</quote>
          (asterisks), which is interpreted as <emphasis>any</emphasis>
          component. You can set global logging options by using this,
          including setting the logging configuration for a library
          that is used by multiple daemons (e.g. <quote>*.config</quote>
          specifies the configuration library code in whatever
          daemon is using it).
        </para> -->

          <para>
            If there are multiple logger specifications in the configuration
            that might match a particular logger, the specification with the
            more specific logger name takes precedence. For example, if there
            are entries for both <quote>kea-dhcp4</quote> and
            <quote>kea-dhcp4.dhcpsrv</quote>, the DHCPv4 server &mdash; and all
            libraries it uses that are not dhcpsrv &mdash; will log messages
            according to the configuration in the first entry
            (<quote>kea-dhcp4</quote>).
          </para>

          <para>
            One final note about the naming. When specifying the daemon name
            within a logger, use the name of the binary file,
            e.g. <quote>kea-dhcp4</quote> for the DHCPv4 server,
            <quote>kea-dhcp6</quote> for the DHCPv6 server, etc. When the
            message is logged, the message will include the name of the process
            (e.g. <quote>kea-dhcp4</quote>) followed by the specific component
            in that process, e.g. <quote>hooks</quote>.  It is possible to
            specify either just the process name (<quote>kea-dhcp4</quote>, will
            apply to everything logged within that process) or process name
            followed by specific logger,
            e.g. <quote>kea-dhcp4.hooks</quote>. That will apply only to
            messages originating from that component.
          </para>

          <para>
            Currently defined loggers are:
          </para>

        <itemizedlist>
          <listitem>
            <simpara><command>kea-dhcp4</command> - this is the root logger for
            the DHCPv4 server. All components used by the DHCPv4 server inherit
            the settings from this logger if there is no specialized logger
            provided.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.bad_packet</command> - this is the
            logger used by the DHCPv4 server deamon for logging inbound client
            packets that were dropped or to which the server responded with a
            NAK.  The allows adminstrators to configure a separate log
            output that contains only packet drop and reject entries.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.dhcp4</command> - this is the logger
            used solely by the DHCPv4 server deamon. This logger does not
            specify logging settings for libraries used by the deamon.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.dhcpsrv</command> - this logger is used
            by the libdhcpsrv library. This covers mostly DHCP engine (the lease
            allocation and renewal process), database operations and
            configuration.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.hooks</command> - this logger is used
            during DHCPv4 hooks operation, i.e. anything related to user
            libraries will be logged using this logger.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp4.hosts</command> - this logger is used
            within the libdhcpsrv and it logs messages related to the management
            of the DHCPv4 host reservations, i.e. retrieval of the resevations
            and adding new reservations.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6</command> - this is the root logger for
            the DHCPv6 server. All components used by the DHCPv6 server inherit
            the settings from this logger if there is no specialized logger
            provided.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.dhcp6</command> - this is the logger
            used solely by the DHCPv6 server deamon. This logger does not
            specify logging settings for libraries used by the daemon.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.dhcpsrv</command> - this logger is used
            by the libdhcpsrv library. This covers mostly DHCP engine (the lease
            allocation and renewal process), database operations and
            configuration.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.hooks</command> - this logger is used
            during DHCPv6 hooks operation, i.e. anything related to user
            libraries will be logged using this logger.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp6.hosts</command> - this logger is used
            within the libdhcpsrv and it logs messages related to the management
            of the DHCPv6 host reservations, i.e. retrieval of the resevations
            and adding new reservations.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp-ddns</command> - this is the root logger for
            the kea-dhcp-ddns deamon. All components used by this deamon inherit
            the settings from this logger if there is no specialized logger
            provided.</simpara>
          </listitem>
          <listitem>
            <simpara><command>kea-dhcp-ddns.dhcpddns</command> - this is the logger
            used solely by the kea-dhcp-ddns deamon. This logger does not
            specify logging settings for libraries used by the deamon.</simpara>
          </listitem>
        </itemizedlist>

        <para>Additional loggers may be defined in the future. The easiest
        way to find out the logger name is to configure all logging to go
        to a single destination and look for specific logger names. See
        <xref linkend="logging-message-format"/> for details.</para>
        </section>

        <section>
          <title>severity (string)</title>

        <para>
          This specifies the category of messages logged.
          Each message is logged with an associated severity which
          may be one of the following (in descending order of
          severity):
        </para>

        <itemizedlist>
          <listitem>
            <simpara> FATAL </simpara>
          </listitem>

          <listitem>
            <simpara> ERROR </simpara>
          </listitem>

          <listitem>
            <simpara> WARN </simpara>
          </listitem>

          <listitem>
            <simpara> INFO </simpara>
          </listitem>

          <listitem>
            <simpara> DEBUG </simpara>
          </listitem>
        </itemizedlist>

        <para>

          When the severity of a logger is set to one of these
          values, it will only log messages of that severity, and
          the severities above it. The severity may also be set to
          NONE, in which case all messages from that logger are
          inhibited.

<!-- TODO: worded wrong? If I set to INFO, why would it show DEBUG which is literally below in that list? -->

        </para>

        </section>

        <section>
          <title>output_options (list)</title>

        <para>

          Each logger can have zero or more
          <option>output_options</option>. These specify where log
          messages are sent. These are explained in detail below.

        </para>

        <para>

          The other options for a logger are:

        </para>

        </section>

        <section>
          <title>debuglevel (integer)</title>

        <para>

          When a logger's severity is set to DEBUG, this value
          specifies what debug messages should be printed. It ranges
          from 0 (least verbose) to 99 (most verbose).
        </para>


<!-- TODO: complete this sentence:

          The general classification of debug message types is

TODO; there's a ticket to determine these levels, see #1074

 -->

        <para>

          If severity for the logger is not DEBUG, this value is ignored.

        </para>

        </section>
      </section>

      <section>
        <title>Output Options</title>

        <para>

          The main settings for an output option are the
          <option>destination</option> and a value called
          <option>output</option>, the meaning of which depends on
          the destination that is set.

        </para>

        <section>
          <title>destination (string)</title>

          <para>

            The destination is the type of output. It can be one of:

          </para>

          <itemizedlist>

            <listitem>
              <simpara> console </simpara>
          </listitem>

            <listitem>
              <simpara> file </simpara>
          </listitem>

            <listitem>
              <simpara> syslog </simpara>
            </listitem>

          </itemizedlist>

        </section>

        <section>
          <title>output (string)</title>

        <para>
          This value determines the type of output. There are several
          special values allowed here: <command>stdout</command> (messages
          are printed on standard output), <command>stderr</command>
          (messages are printed on stderr), <command>syslog</command> (messages
          are logged to syslog using default name, <command>syslog:name</command>
          (messages are logged to syslog using specified name). Any other
          value is interpreted as a filename that the logs should be written to.
        </para>

        <para>

          The other options for <option>output_options</option> are:

        </para>

        <!-- configuration of flush is not supported yet.
        <section>
          <title>flush (true of false)</title>

          <para>
            Flush buffers after each log message. Doing this will
            reduce performance but will ensure that if the program
            terminates abnormally, all messages up to the point of
            termination are output.
          </para>

        </section> -->

        <section>
          <title>maxsize (integer)</title>

          <para>
            Only relevant when destination is file, this is maximum
            file size of output files in bytes. When the maximum
            size is reached, the file is renamed and a new file opened.
            (For example, a ".1" is appended to the name &mdash;
            if a ".1" file exists, it is renamed ".2",
            etc.)
          </para>

          <para>
            If this is 0, no maximum file size is used.
          </para>

          <note>
            <simpara>
	      Due to a limitation of the underlying logging library
	      (log4cplus), rolling over the log files (from ".1" to
	      ".2", etc) may show odd results: There can be
	      multiple small files at the timing of roll over.  This
	      can happen when multiple processes try to roll
	      over the files simultaneously.
	      Version 1.1.0 of log4cplus solved this problem, so if
	      this or higher version of log4cplus is used to build
	      Kea, it shouldn't happen.  Even for older versions
	      it is normally expected to happen rarely unless the log
	      messages are produced very frequently by multiple
	      different processes.
	    </simpara>
	  </note>

        </section>

        <section>
          <title>maxver (integer)</title>

          <para>
            Maximum number of old log files to keep around when
            rolling the output file. Only relevant when
            <option>output</option> is <quote>file</quote>.
          </para>

        </section>

      </section>

      </section>

      <section>
        <title>Example Logger Configurations</title>

        <para>
          In this example we want to set the global logging to
          write to the console using standard output.
        </para>

<screen><userinput>
"Logging": {
    "loggers": [
        {
            "name": "kea-dhcp4",
            "output_options": [
                {
                    "output": "stdout"
                }
            ],
            "severity": "WARN"
        }
    ]
}
</userinput>
</screen>

<para>In this second example, we want to store debug log messages
in a file that is at most 2MB and keep up to 8 copies of old logfiles.
Once the logfile grows to 2MB, it will be renamed and a new file
file be created.</para>

<screen><userinput>
"Logging": {
    "loggers": [
        {
            "name": "kea-dhcp6",
            "output_options": [
                {
                    "output": "/var/log/kea-debug.log",
                    "maxver": 8,
                    "maxsize": 204800,
                    "destination": "file"
                }
            ],
            "severity": "DEBUG",
            "debuglevel": 99
        }
   ]
}</userinput></screen>
      </section>

    </section>

    <section id="logging-message-format">
      <title>Logging Message Format</title>

      <para>
          Each message written  to the configured logging
          destinations comprises a number of components that identify
          the origin of the message and, if the message indicates
          a problem, information about the problem that may be
          useful in fixing it.
      </para>

      <para>
          Consider the message below logged to a file:
          <screen>2014-04-11 12:58:01.005 INFO  [kea-dhcp4.dhcpsrv/27456]
    DHCPSRV_MEMFILE_DB opening memory file lease database: type=memfile universe=4</screen>
      </para>

      <para>
        Note: the layout of messages written to the system logging
        file (syslog) may be slightly different.  This message has
        been split across two lines here for display reasons; in the
        logging file, it will appear on one line.
      </para>

      <para>
        The log message comprises a number of components:

          <variablelist>
          <varlistentry>
          <term>2014-04-11 12:58:01.005</term>
<!-- TODO: timestamp repeated even if using syslog? -->
          <listitem><para>
              The date and time at which the message was generated.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>INFO</term>
          <listitem><para>
              The severity of the message.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>[kea-dhcp4.dhcpsrv/27456]</term>
          <listitem><para>
            The source of the message.  This comprises two elements:
            the Kea process generating the message (in this
            case, <command>kea-dhcp4</command>) and the component
            within the program from which the message originated
            (which is the name of the common library used by DHCP server
            implementations). The number after the slash is a process id
            (pid).
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>DHCPSRV_MEMFILE_DB</term>
          <listitem><para>
            The message identification.  Every message in Kea
            has a unique identification, which can be used as an
            index into the <ulink
            url="kea-messages.html"><citetitle>Kea Messages
            Manual</citetitle></ulink> (<ulink
            url="http://kea.isc.org/docs/kea-messages.html"
            />) from which more information can be obtained.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>opening memory file lease database: type=memfile universe=4</term>
          <listitem><para>
              A brief description.
              Within this text, information relating to the condition
              that caused the message to be logged will be included.
              In this example, the information is logged that the in-memory
              lease database backend will be used to store DHCP leases.
          </para></listitem>
          </varlistentry>
          </variablelist>
      </para>

    </section>

    <section>
      <title>Logging During Kea Startup</title>

      <para>
        The logging configuration is specified in the configuration file.
        However, when Kea starts, the file is not read until some way into the
        initialization process.  Prior to that, the logging settings are
        set to default values, although it is possible to modify some
        aspects of the settings by means of environment variables. Note
        that in the absence of any logging configuration in the configuration
        file, the settings of (possibly modified) default configuration will
        persist while the program is running.
      </para>
      <para>
        The following environment variables can be used to control the
        behavio of logging during startup:
      </para>

          <variablelist>
          <varlistentry>
          <term>KEA_LOCKFILE_DIR</term>
          <listitem><para>
              Specifies a directory where the logging system should create its
              lock file. If not specified, it is
              <replaceable>prefix</replaceable>/var/run/kea, where
              <replaceable>prefix</replaceable> defaults to /usr/local.
              This variable must not end
              with a slash. There is one special value: "none", which
              instructs Kea to not create lock file at all. This may cause
              issues if several processes log to the same file.
          </para></listitem>
          </varlistentry>

          <varlistentry>
          <term>KEA_LOGGER_DESTINATION</term>
          <listitem><para>
              Specifies logging output. There are several special values.
              <variablelist>
                  <varlistentry>
                      <term>stdout</term>
                      <listitem><para>
                          Log to standard output.
                      </para></listitem>
                  </varlistentry>
                  <varlistentry>
                      <term>stderr</term>
                      <listitem><para>
                          Log to standard error.
                      </para></listitem>
                  </varlistentry>
                  <varlistentry>
                      <term>syslog<optional>:<replaceable>fac</replaceable></optional></term>
                      <listitem><para>
                          Log via syslog. The optional
                          <replaceable>fac</replaceable> (which is
                          separated from the word "syslog" by a colon)
                          specifies the
                          facility to be used for the log messages. Unless
                          specified, messages will be logged using the
                          facility "local0".
                      </para></listitem>
                  </varlistentry>
              </variablelist>
              Any other value is treated as a name
              of the output file. If not specified otherwise, Kea will log to
              standard output.
          </para></listitem>
          </varlistentry>

          </variablelist>
    </section>

  </chapter>