[1198] Add reordered message file and utility to do this

Added utility to put messages in a message file into alphabetical
order and, for ease of further editing, the data source message file
in alphabetical order.

src/lib/datasrc/datasrc_messages.mes

@@ -78,6 +78,9 @@ different TTL values. This isn't allowed on the wire and is considered
 an error, so we set it to the lowest value we found (but we don't modify the
 database). The data in database should be checked and fixed.
 
+% DATASRC_DATABASE_FOUND_CNAME search in datasource %1 for %2/%3/%4 resulted in CNAME %5
+(TBD)
+
 % DATASRC_DATABASE_FOUND_DELEGATION Found delegation at %2 in %1
 When searching for a domain, the program met a delegation to a different zone
 at the given domain name. It will return that one instead.
@@ -105,17 +108,14 @@ domain name, class and type.
 The data returned by the database backend contained data for the given domain
 name and class, but not for the given type.
 
+% DATASRC_DATABASE_FOUND_NXRRSET_NSEC search in datasource %1 for %2/%3/%4 resulted in RRset %5
+(TBD)
+
 % DATASRC_DATABASE_FOUND_RRSET search in datasource %1 resulted in RRset %2
 The data returned by the database backend contained data for the given domain
 name, and it either matches the type or has a relevant type. The RRset that is
 returned is printed.
 
-% DATASRC_DATABASE_FOUND_CNAME search in datasource %1 for %2/%3/%4 resulted in CNAME %5
-(TBD)
-
-% DATASRC_DATABASE_FOUND_NXRRSET_NSEC search in datasource %1 for %2/%3/%4 resulted in RRset %5
-(TBD)
-
 % DATASRC_DATABASE_ITERATE iterating zone %1
 The program is reading the whole zone, eg. not searching for data, but going
 through each of the RRsets there.
@@ -133,6 +133,41 @@ were found to be different. This isn't allowed on the wire and is considered
 an error, so we set it to the lowest value we found (but we don't modify the
 database). The data in database should be checked and fixed.
 
+% DATASRC_DATABASE_UPDATER_COMMIT updates committed for '%1/%2' on %3
+Debug information.  A set of updates to a zone has been successfully
+committed to the corresponding database backend.  The zone name,
+its class and the database name are printed.
+
+% DATASRC_DATABASE_UPDATER_CREATED zone updater created for '%1/%2' on %3
+Debug information.  A zone updater object is created to make updates to
+the shown zone on the shown backend database.
+
+% DATASRC_DATABASE_UPDATER_DESTROYED zone updater destroyed for '%1/%2' on %3
+Debug information.  A zone updater object is destroyed, either successfully
+or after failure of, making updates to the shown zone on the shown backend
+database.
+
+% DATASRC_DATABASE_UPDATER_ROLLBACK zone updates roll-backed for '%1/%2' on %3
+A zone updater is being destroyed without committing the changes.
+This would typically mean the update attempt was aborted due to some
+error, but may also be a bug of the application that forgets committing
+the changes.  The intermediate changes made through the updater won't
+be applied to the underlying database.  The zone name, its class, and
+the underlying database name are shown in the log message.
+
+% DATASRC_DATABASE_UPDATER_ROLLBACKFAIL failed to roll back zone updates for '%1/%2' on %3: %4
+A zone updater is being destroyed without committing the changes to
+the database, and attempts to rollback incomplete updates, but it
+unexpectedly fails.  The higher level implementation does not expect
+it to fail, so this means either a serious operational error in the
+underlying data source (such as a system failure of a database) or
+software bug in the underlying data source implementation.  In either
+case if this message is logged the administrator should carefully
+examine the underlying data source to see what exactly happens and
+whether the data is still valid.  The zone name, its class, and the
+underlying database name as well as the error message thrown from the
+database module are shown in the log message.
+
 % DATASRC_DATABASE_WILDCARD constructing RRset %3 from wildcard %2 in %1
 The database doesn't contain directly matching domain, but it does contain a
 wildcard one which is being used to synthesize the answer.
@@ -493,12 +528,12 @@ enough information for it.  The code is 1 for error, 2 for not implemented.
 % DATASRC_SQLITE_CLOSE closing SQLite database
 Debug information. The SQLite data source is closing the database file.
 
-% DATASRC_SQLITE_CONNOPEN Opening sqlite database file '%1'
-The database file is being opened so it can start providing data.
-
 % DATASRC_SQLITE_CONNCLOSE Closing sqlite database
 The database file is no longer needed and is being closed.
 
+% DATASRC_SQLITE_CONNOPEN Opening sqlite database file '%1'
+The database file is being opened so it can start providing data.
+
 % DATASRC_SQLITE_CREATE SQLite data source created
 Debug information. An instance of SQLite data source is being created.
 
@@ -601,38 +636,3 @@ data source.
 % DATASRC_UNEXPECTED_QUERY_STATE unexpected query state
 This indicates a programming error. An internal task of unknown type was
 generated.
-
-% DATASRC_DATABASE_UPDATER_CREATED zone updater created for '%1/%2' on %3
-Debug information.  A zone updater object is created to make updates to
-the shown zone on the shown backend database.
-
-% DATASRC_DATABASE_UPDATER_DESTROYED zone updater destroyed for '%1/%2' on %3
-Debug information.  A zone updater object is destroyed, either successfully
-or after failure of, making updates to the shown zone on the shown backend
-database.
-
-%DATASRC_DATABASE_UPDATER_ROLLBACK zone updates roll-backed for '%1/%2' on %3
-A zone updater is being destroyed without committing the changes.
-This would typically mean the update attempt was aborted due to some
-error, but may also be a bug of the application that forgets committing
-the changes.  The intermediate changes made through the updater won't
-be applied to the underlying database.  The zone name, its class, and
-the underlying database name are shown in the log message.
-
-%DATASRC_DATABASE_UPDATER_ROLLBACKFAIL failed to roll back zone updates for '%1/%2' on %3: %4
-A zone updater is being destroyed without committing the changes to
-the database, and attempts to rollback incomplete updates, but it
-unexpectedly fails.  The higher level implementation does not expect
-it to fail, so this means either a serious operational error in the
-underlying data source (such as a system failure of a database) or
-software bug in the underlying data source implementation.  In either
-case if this message is logged the administrator should carefully
-examine the underlying data source to see what exactly happens and
-whether the data is still valid.  The zone name, its class, and the
-underlying database name as well as the error message thrown from the
-database module are shown in the log message.
-
-% DATASRC_DATABASE_UPDATER_COMMIT updates committed for '%1/%2' on %3
-Debug information.  A set of updates to a zone has been successfully
-committed to the corresponding database backend.  The zone name,
-its class and the database name are printed.

tools/reorder_message_file.py

@@ -0,0 +1,196 @@
+# Copyright (C) 2011  Internet Systems Consortium, Inc. ("ISC")
+#
+# Permission to use, copy, modify, and/or distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+# AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+# PERFORMANCE OF THIS SOFTWARE.
+
+# Reorder Message File
+#
+# Reads a message file into memory, then outputs it with the messages and
+# associated descriptions in alphabetical order.
+#
+# Invocation:
+# The code is invoked using the command line:
+#
+# python reorder.py message_file
+#
+# Output is written to stdout.
+
+import sys
+
+def removeEmptyLeadingTrailing(lines):
+    """
+    Removes leading and trailing empty lines.
+
+    A list of strings is passed as argument, some of which may be empty.
+    This function removes from the start and end of the list a contiguous
+    sequence of empty lines and returns the result.  Embedded sequences of
+    empty lines are not touched.
+
+    Parameters:
+    lines List of strings to be modified.
+
+    Return:
+    Input list of strings with leading/trailing blank line sequences
+    removed.
+    """
+
+    retlines = []
+
+    # Dispose of degenerate case of empty array
+    if len(lines) == 0:
+        return retlines
+
+    # Search for first non-blank line
+    start = 0
+    while start < len(lines):
+        if len(lines[start]) > 0:
+            break
+        start = start + 1
+
+    # Handle case when entire list is empty
+    if start >= len(lines):
+        return retlines
+
+    # Search for last non-blank line
+    finish = len(lines) - 1
+    while finish >= 0:
+        if len(lines[finish]) > 0:
+            break
+        finish = finish - 1
+
+    retlines = lines[start:finish + 1]
+    return retlines
+
+
+def canonicalise_message_line(line):
+    """
+    Given a line known to start with the '%' character (i.e. a line
+    introducing a message), canonicalise it by ensuring that the result
+    is of the form '%<single-space>MESSAGE_IDENTIFIER<single-space>text'.
+
+    Parameters:
+    line - input line.  Known to start with a '%' and to have leading
+           and trailing spaces removed.
+
+    Return:
+    Canonicalised line.
+    """
+    # Cope with degenerate case of a single "%"
+    if len(line) == 1:
+        return line
+
+    # Get the rest of the line
+    line = line[1:].lstrip()
+
+    # Extract the first word (the message ID)
+    words = line.split()
+    message_line = "% " + words[0]
+    
+    # ... and now the rest of the line
+    if len(line) > len(words[0]):
+        message_line = message_line + " " + line[len(words[0]):].lstrip()
+
+    return message_line
+
+
+def make_dict(lines):
+    """
+    Split the lines into segments starting with the message definition and
+    place into a dictionary.
+
+    Parameters:
+    lines - list of lines containing the text of the message file (less the
+            header).
+
+    Returns:
+    dictionary - map of the messages, keyed by the line that holds the message
+                 ID.
+    """
+
+    dictionary = {}
+
+    message_key = canonicalise_message_line(lines[0])
+    message_lines = [message_key]
+    index = 1;
+    while index < len(lines):
+        if lines[index].startswith("%"):
+            # Start of new message
+            dictionary[message_key] = removeEmptyLeadingTrailing(message_lines)
+            message_key = canonicalise_message_line(lines[index])
+            message_lines = [message_key]
+        else:
+            message_lines.append(lines[index])
+        
+        index = index + 1
+
+    dictionary[message_key] = removeEmptyLeadingTrailing(message_lines)
+
+    return dictionary
+
+        
+def print_dict(dictionary):
+    """
+    Prints the dictionary with a blank line between entries.
+
+    Parameters:
+    dicitionary - Map holding the message dictionary
+    """
+    count = 0
+    for msgid in sorted(dictionary):
+
+        # Blank line before all entries but the first
+        if count > 0:
+            print("")
+        count = count + 1
+
+        # ... and the entry itself.
+        for l in dictionary[msgid]:
+            print(l.strip())
+
+
+def processFile(filename):
+    """
+    Processes a file by reading it and searching for the first line starting
+    with the '%' sign.  Everything before that line is treated as the file
+    header and is copied to the output with leading and trailing spaces removed.
+    After that, each message block is read and stored for later sorting.
+
+    Parameters:
+    filename     Name of the message file to process
+    """
+    lines = open(filename).read().splitlines();
+
+    # Search for the first line starting with the percent character.  Everything
+    # before it is considered the file header and is copied to the output with
+    # leading and trailing spaces removed.
+    index = 0
+    while index < len(lines):
+        if lines[index].startswith("%"):
+            break
+        print(lines[index].strip())
+        index = index + 1
+
+    # Now put the remaining lines into the message dictionary
+    dictionary = make_dict(lines[index:])
+
+    # ...and print it
+    print_dict(dictionary)
+
+
+# Main program
+if __name__ == "__main__":
+
+    # Read the files and load the data
+    if len(sys.argv) != 2:
+        print "Usage: python reorder.py message_file"
+    else:
+        processFile(sys.argv[1])