<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
<!ENTITY mdash "—" >
]>
<!-- Note: Please use the following terminology:
- daemon - one process (e.g. kea-dhcp4)
- component - one piece of code within a daemon (e.g. libdhcp or hooks)
- server - currently equal to daemon, but the difference will be more
prominent once we add client or relay support
- logger - one instance of isc::log::Logger
- structure - an element in config file (e.g. "Dhcp4")
Do not use:
- module => daemon
-->
<chapter id="logging">
<title>Logging</title>
<section>
<title>Logging Configuration</title>
<para>
During its operation Kea may produce many messages. They differ in
severity (some are more important than others) and source (some are
produced by specific components, e.g. hooks). It is useful to understand
which log messages are needed and which are not and configure your
logging appropriately. For example, debug level messages can be safely
ignored in a typical deployment. They are, however, very useful when
debugging a problem.
</para>
<para>
The logging system in Kea is configured through the
<replaceable>Logging</replaceable> structure in your configuration
file. All daemons (e.g. DHCPv4 and DHCPv6 servers) will use the
configuration in the <replaceable>Logging</replaceable> structure to see
what should be logged and to where. This allows for sharing identical
logging configuration between daemons.
</para>
<section>
<title>Loggers</title>
<para>
Within Kea, a message is logged through an entity called a
"logger". Different components log messages through different
loggers, and each logger can be configured independently of
one another. Some components, in particular the DHCP server
processes, may use multiple loggers to log messages pertaining
to different logical functions of the component. For example,
the DHCPv4 server is using one logger for messages
pertaining to packet reception and transmission, another
logger for messages related to lease allocation and so on.
Some of the libraries used by the Kea servers, e.g. libdhcpsrv
or libhooks library, use their own loggers.
</para>
<para>
Users implementing hooks libraries (code attached to the server at
runtime) are responsible for creating the loggers used by those
libraries. Such loggers should have unique names, different
from the logger names used by Kea. In this way the
messages emitted from the hooks library can be distingued from
messages issued by the core Kea code. Unique names also allow
the loggers to be configured independently of loggers used
by Kea. Whenever it makes sense, a hook library can use multiple
loggers to log messages pertaining to different logical parts
of the library.
</para>
<para>
In the <quote>Logging</quote> structure of a configuration file
you can specify the configuration for zero or more loggers
(including loggers used by the proprietary hooks libraries). If
there are no loggers specified, the code will use default values which
cause Kea to log messages on at least INFO severity to standard
output.
<!-- @todo: add reference to section about controlling default
behavior with env. variables, after #3591 is merged. -->
</para>
<para>
The three most important elements of a logger configuration
are the <option>name</option> (the component that is
generating the messages), the <option>severity</option>
(what to log), and the <option>output_options</option>
(where to log).
</para>
<section>
<title>name (string)</title>
<para>
Each logger in the system has a name, the name being that of the
component binary file using it to log messages. For instance, if you
want to configure logging for the DHCPv4 server, you add an entry
for a logger named <quote>kea-dhcp4</quote>. This configuration will
then be used by the loggers in the DHCPv4 server, and all the
libraries used by it (unless a library defines its own logger and
there is specific logger configuration that applies to that logger).
</para>
<para>
When diagnosing the problem with the server's operation, it is often
desired to use the DEBUG logging level to obtain the verbose output
from the server and libraries it uses. However, high verbosity may
be an overkill for the logging system in cases when the server
is processing high volume traffic. To mitigate this problem, Kea
is using multiple loggers, which can be configured independently
and which are responsible for logging messages from different
functional parts of the server. If the user, trying to diagnose the
problem, has a reasonably high confidence that the problem origins
in a specific function of the server, or the problem is related
to the specific type of operation, he may enable high verbosity
only for the relevant logger, thus limiting the debug messages
to the required minimum.
</para>
<para>
The loggers are associated with a particular library or binary
of Kea. However, each library or binary may (and usually does)
include multiple loggers. For example, the DHCPv4 server binary
contains separate loggers for: packet parsing, for dropped packets,
for callouts etc. Each logger "derives" its configuration from the
root logger. In the typical case, the root logger configuration
is the only logging configuration specified in the configuration
file. Creating a specific configuration for the selected logger,
thus overriding the configuration settings specified in the
root logger configuration, requires putting its configuration
aside of the root logger's configuration with some of the
parameters modified.
</para>
<para>
To illustrate this, suppose you are using the DHCPv4 server
with the root logger <quote>kea-dhcp4</quote> logging at the
INFO level. In order to enable DEBUG verbosity for the DHCPv4
packet drops, you must create configuration entry for the
logger called <quote>kea-dhcp4.bad-packets</quote> and specify
severity DEBUG for this logger. All other configuration
parameters may be omited for this logger if the logger should
use the default values specified in the root logger's
configuration.
</para>
<!-- we don't support asterisk anymore.
<para>
One special case is that of a component name of <quote>*</quote>
(asterisks), which is interpreted as <emphasis>any</emphasis>
component. You can set global logging options by using this,
including setting the logging configuration for a library
that is used by multiple daemons (e.g. <quote>*.config</quote>
specifies the configuration library code in whatever
daemon is using it).
</para> -->
<para>
If there are multiple logger specifications in the configuration
that might match a particular logger, the specification with the
more specific logger name takes precedence. For example, if there
are entries for both <quote>kea-dhcp4</quote> and
<quote>kea-dhcp4.dhcpsrv</quote>, the DHCPv4 server — and all
libraries it uses that are not dhcpsrv — will log messages
according to the configuration in the first entry
(<quote>kea-dhcp4</quote>).
</para>
<para>
Currently defined loggers are:
</para>
<itemizedlist>
<listitem>
<simpara><command>kea-dhcp4</command> - this is the root logger for
the DHCPv4 server. All components used by the DHCPv4 server inherit
the settings from this logger if there is no specialized logger
provided.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.alloc-engine</command> - this is the
logger used by the lease allocation engine, which is responsible
for managing leases in the lease database, i.e. create, modify
and remove DHCPv4 leases as a result of processing messages from
the clients.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.bad-packets</command> - this is the
logger used by the DHCPv4 server deamon for logging inbound client
packets that were dropped or to which the server responded with a
DHCPNAK. The allows adminstrators to configure a separate log
output that contains only packet drop and reject entries.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.callouts</command> - this logger is used
to log messages pertaining to the callouts registration and execution
for the particular hook point.
</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.ddns</command> - this logger is used by
the DHCPv4 server to log messages related to the Client FQDN and
Hostname option processing. It also includes log messages
related to the relevant DNS updates.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.dhcp4</command> - this is the logger
by the DHCPv4 server deamon to log basic operations.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.dhcpsrv</command> - this is a base
logger for the libdhcpsrv library.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.hooks</command> - this logger is used
to log messages related to management of hooks libraries, e.g.
registration and deregistration of the libraries, and to the
initialization of the callouts execution for various hook points
within the DHCPv4 server.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.hosts</command> - this logger is used
within the libdhcpsrv and it logs messages related to the management
of the DHCPv4 host reservations, i.e. retrieval of the resevations
and adding new reservations.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.leases</command> - this logger is used
by the DHCPv4 server to log messages related to the lease allocation.
The messages include detailed information about the allocated or
offered leases, errors during the lease allocation etc.
</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.options</command> - this logger is
used by the DHCPv4 server to log messages related to processing
of the options in the DHCPv4 messages, i.e. parsing options,
encoding options into on-wire format and packet classification
using options contained in the received packets.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp4.packets</command> - this logger
is mostly used to log messages related to transmission of the DHCPv4
packets, i.e. packet reception and sending a response. Such messages
include the information about the source and destination IP addresses
and interfaces used to transmit packets. This logger is also used
to log messages related to subnet selection, as this selection is
usually based on the IP addresses and/or interface names, which can
be retrieved from the received packet, even before the DHCPv4 message
carried in the packet is parsed.
</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6</command> - this is the root logger for
the DHCPv6 server. All components used by the DHCPv6 server inherit
the settings from this logger if there is no specialized logger
provided.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.bad-packets</command> - this is the
logger used by the DHCPv6 server deamon for logging inbound client
packets that were dropped.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.callouts</command> - this logger is used
to log messages pertaining to the callouts registration and execution
for the particular hook point.
</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.ddns</command> - this logger is used by
the DHCPv6 server to log messages related to the Client FQDN option
processing. It also includes log messages related to the relevant
DNS updates.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.dhcp6</command> - this is the logger
used DHCPv6 server deamon to log basic operations.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.alloc-engine</command> - this is the
logger used by the lease allocation engine, which is responsible
for managing leases in the lease database, i.e. create, modify
and remove DHCPv6 leases as a result of processing messages from
the clients.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.dhcpsrv</command> - this is a base
logger for the libdhcpsrv library.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.hooks</command> - this logger is used
to log messages related to management of hooks libraries, e.g.
registration and deregistration of the libraries, and to the
initialization of the callouts execution for various hook points
within the DHCPv6 server.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.hosts</command> - this logger is used
within the libdhcpsrv and it logs messages related to the management
of the DHCPv6 host reservations, i.e. retrieval of the resevations
and adding new reservations.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.leases</command> - this logger is used
by the DHCPv6 server to log messages related to the lease allocation.
The messages include detailed information about the allocated or
offered leases, errors during the lease allocation etc.
</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.options</command> - this logger is
used by the DHCPv6 server to log messages related to processing
of the options in the DHCPv6 messages, i.e. parsing options,
encoding options into on-wire format and packet classification
using options contained in the received packets.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp6.packets</command> - this logger
is mostly used to log messages related to transmission of the DHCPv6
packets, i.e. packet reception and sending a response. Such messages
include the information about the source and destination IP addresses
and interfaces used to transmit packets. This logger is also used
to log messages related to subnet selection, as this selection is
usually based on the IP addresses and/or interface names, which can
be retrieved from the received packet, even before the DHCPv6 message
carried in the packet is parsed.
</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp-ddns</command> - this is the root logger for
the kea-dhcp-ddns deamon. All components used by this deamon inherit
the settings from this logger if there is no specialized logger
provided.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp-ddns.dhcpddns</command> - this is the logger
used by the kea-dhcp-ddns deamon for logging configuration and global
event information. This logger does not specify logging settings
for libraries used by the deamon.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp-ddns.dhcp-to-d2</command> - this is the logger
used by the kea-dhcp-ddns deamon for logging information about events
dealing with receving messages from the DHCP servers and adding them
to the queue for processing.</simpara>
</listitem>
<listitem>
<simpara><command>kea-dhcp-ddns.d2-to-dns</command> - this is the logger
used by the kea-dhcp-ddns deamon for logging information about events
dealing with sending and receiving messages with the DNS servers.
</simpara>
</listitem>
</itemizedlist>
<para>
Note that user-defined hook libraries should not use any of those
loggers, and should define new loggers with names that correspond to
the libraries using them. Suppose that the user created the library called
<quote>libpacket-capture</quote> to dump packets received and
transmitted by the server to the file. The appropriate name for the
logger could be <command>kea-dhcp4.packet-capture</command>. Note
that the hook library implementor only specifies the second part
of this name, i.e. <quote>packet-capture</quote>. The first part is
a root logger name and is prepended by the Kea logging system.
It is also important to note that since this new logger is a child
of a root logger, it inherits the configuration from the root logger,
unless there is a separate configuration entry for the child logger
which overrides the default configuration.
</para>
<para>
The list of loggers above excludes any loggers implemented in hooks
libraries. Please consult the documentation of the specific hooks
libraries for the names of loggers they define.
</para>
<para>Additional loggers may be defined in the future. The easiest
way to find out the logger name is to configure all logging to go
to a single destination and look for specific logger names. See
<xref linkend="logging-message-format"/> for details.</para>
</section>
<section>
<title>severity (string)</title>
<para>
This specifies the category of messages logged.
Each message is logged with an associated severity which
may be one of the following (in descending order of
severity):
</para>
<itemizedlist>
<listitem>
<simpara> FATAL </simpara>
</listitem>
<listitem>
<simpara> ERROR </simpara>
</listitem>
<listitem>
<simpara> WARN </simpara>
</listitem>
<listitem>
<simpara> INFO </simpara>
</listitem>
<listitem>
<simpara> DEBUG </simpara>
</listitem>
</itemizedlist>
<para>
When the severity of a logger is set to one of these
values, it will only log messages of that severity, and
the severities above it. The severity may also be set to
NONE, in which case all messages from that logger are
inhibited.
<!-- TODO: worded wrong? If I set to INFO, why would it show DEBUG which is literally below in that list? -->
</para>
</section>
<section>
<title>output_options (list)</title>
<para>
Each logger can have zero or more
<option>output_options</option>. These specify where log
messages are sent. These are explained in detail below.
</para>
<para>
The other options for a logger are:
</para>
</section>
<section>
<title>debuglevel (integer)</title>
<para>
When a logger's severity is set to DEBUG, this value
specifies what debug messages should be printed. It ranges
from 0 (least verbose) to 99 (most verbose).
</para>
<!-- TODO: complete this sentence:
The general classification of debug message types is
TODO; there's a ticket to determine these levels, see #1074
-->
<para>
If severity for the logger is not DEBUG, this value is ignored.
</para>
</section>
</section>
<section>
<title>Output Options</title>
<para>
The main settings for an output option are the
<option>destination</option> and a value called
<option>output</option>, the meaning of which depends on
the destination that is set.
</para>
<section>
<title>destination (string)</title>
<para>
The destination is the type of output. It can be one of:
</para>
<itemizedlist>
<listitem>
<simpara> console </simpara>
</listitem>
<listitem>
<simpara> file </simpara>
</listitem>
<listitem>
<simpara> syslog </simpara>
</listitem>
</itemizedlist>
</section>
<section>
<title>output (string)</title>
<para>
This value determines the type of output. There are several
special values allowed here: <command>stdout</command> (messages
are printed on standard output), <command>stderr</command>
(messages are printed on stderr), <command>syslog</command> (messages
are logged to syslog using default name, <command>syslog:name</command>
(messages are logged to syslog using specified name). Any other
value is interpreted as a filename that the logs should be written to.
</para>
<para>
The other options for <option>output_options</option> are:
</para>
<section>
<title>flush (true of false)</title>
<para>
Flush buffers after each log message. Doing this will
reduce performance but will ensure that if the program
terminates abnormally, all messages up to the point of
termination are output. Default is true.
</para>
</section>
<section>
<title>maxsize (integer)</title>
<para>
Only relevant when destination is file, this is maximum
file size of output files in bytes. When the maximum
size is reached, the file is renamed and a new file opened.
(For example, a ".1" is appended to the name —
if a ".1" file exists, it is renamed ".2",
etc.)
</para>
<para>
If this is 0, no maximum file size is used.
</para>
<note>
<simpara>
Due to a limitation of the underlying logging library
(log4cplus), rolling over the log files (from ".1" to
".2", etc) may show odd results: There can be
multiple small files at the timing of roll over. This
can happen when multiple processes try to roll
over the files simultaneously.
Version 1.1.0 of log4cplus solved this problem, so if
this or higher version of log4cplus is used to build
Kea, it shouldn't happen. Even for older versions
it is normally expected to happen rarely unless the log
messages are produced very frequently by multiple
different processes.
</simpara>
</note>
</section>
<section>
<title>maxver (integer)</title>
<para>
Maximum number of old log files to keep around when
rolling the output file. Only relevant when
<option>output</option> is <quote>file</quote>.
</para>
</section>
</section>
</section>
<section>
<title>Example Logger Configurations</title>
<para>
In this example we want to set the global logging to
write to the console using standard output.
</para>
<screen><userinput>
"Logging": {
"loggers": [
{
"name": "kea-dhcp4",
"output_options": [
{
"output": "stdout"
}
],
"severity": "WARN"
}
]
}
</userinput>
</screen>
<para>In this second example, we want to store debug log messages
in a file that is at most 2MB and keep up to 8 copies of old logfiles.
Once the logfile grows to 2MB, it will be renamed and a new file
file be created.</para>
<screen><userinput>
"Logging": {
"loggers": [
{
"name": "kea-dhcp6",
"output_options": [
{
"output": "/var/log/kea-debug.log",
"maxver": 8,
"maxsize": 204800,
"flush": true
}
],
"severity": "DEBUG",
"debuglevel": 99
}
]
}</userinput></screen>
</section>
</section>
<section id="logging-message-format">
<title>Logging Message Format</title>
<para>
Each message written to the configured logging
destinations comprises a number of components that identify
the origin of the message and, if the message indicates
a problem, information about the problem that may be
useful in fixing it.
</para>
<para>
Consider the message below logged to a file:
<screen>2014-04-11 12:58:01.005 INFO [kea-dhcp4.dhcpsrv/27456]
DHCPSRV_MEMFILE_DB opening memory file lease database: type=memfile universe=4</screen>
</para>
<para>
Note: the layout of messages written to the system logging
file (syslog) may be slightly different. This message has
been split across two lines here for display reasons; in the
logging file, it will appear on one line.
</para>
<para>
The log message comprises a number of components:
<variablelist>
<varlistentry>
<term>2014-04-11 12:58:01.005</term>
<!-- TODO: timestamp repeated even if using syslog? -->
<listitem><para>
The date and time at which the message was generated.
</para></listitem>
</varlistentry>
<varlistentry>
<term>INFO</term>
<listitem><para>
The severity of the message.
</para></listitem>
</varlistentry>
<varlistentry>
<term>[kea-dhcp4.dhcpsrv/27456]</term>
<listitem><para>
The source of the message. This comprises two elements:
the Kea process generating the message (in this
case, <command>kea-dhcp4</command>) and the component
within the program from which the message originated
(which is the name of the common library used by DHCP server
implementations). The number after the slash is a process id
(pid).
</para></listitem>
</varlistentry>
<varlistentry>
<term>DHCPSRV_MEMFILE_DB</term>
<listitem><para>
The message identification. Every message in Kea
has a unique identification, which can be used as an
index into the <ulink
url="kea-messages.html"><citetitle>Kea Messages
Manual</citetitle></ulink> (<ulink
url="http://kea.isc.org/docs/kea-messages.html"
/>) from which more information can be obtained.
</para></listitem>
</varlistentry>
<varlistentry>
<term>opening memory file lease database: type=memfile universe=4</term>
<listitem><para>
A brief description.
Within this text, information relating to the condition
that caused the message to be logged will be included.
In this example, the information is logged that the in-memory
lease database backend will be used to store DHCP leases.
</para></listitem>
</varlistentry>
</variablelist>
</para>
</section>
<section>
<title>Logging During Kea Startup</title>
<para>
The logging configuration is specified in the configuration file.
However, when Kea starts, the file is not read until some way into the
initialization process. Prior to that, the logging settings are
set to default values, although it is possible to modify some
aspects of the settings by means of environment variables. Note
that in the absence of any logging configuration in the configuration
file, the settings of (possibly modified) default configuration will
persist while the program is running.
</para>
<para>
The following environment variables can be used to control the
behavio of logging during startup:
</para>
<variablelist>
<varlistentry>
<term>KEA_LOCKFILE_DIR</term>
<listitem><para>
Specifies a directory where the logging system should create its
lock file. If not specified, it is
<replaceable>prefix</replaceable>/var/run/kea, where
<replaceable>prefix</replaceable> defaults to /usr/local.
This variable must not end
with a slash. There is one special value: "none", which
instructs Kea to not create lock file at all. This may cause
issues if several processes log to the same file.
</para></listitem>
</varlistentry>
<varlistentry>
<term>KEA_LOGGER_DESTINATION</term>
<listitem><para>
Specifies logging output. There are several special values.
<variablelist>
<varlistentry>
<term>stdout</term>
<listitem><para>
Log to standard output.
</para></listitem>
</varlistentry>
<varlistentry>
<term>stderr</term>
<listitem><para>
Log to standard error.
</para></listitem>
</varlistentry>
<varlistentry>
<term>syslog<optional>:<replaceable>fac</replaceable></optional></term>
<listitem><para>
Log via syslog. The optional
<replaceable>fac</replaceable> (which is
separated from the word "syslog" by a colon)
specifies the
facility to be used for the log messages. Unless
specified, messages will be logged using the
facility "local0".
</para></listitem>
</varlistentry>
</variablelist>
Any other value is treated as a name
of the output file. If not specified otherwise, Kea will log to
standard output.
</para></listitem>
</varlistentry>
</variablelist>
</section>
</chapter>