[Git][NTPsec/ntpsec][master] Add documentation of the Python code.

[Eric S. Raymond]

Eric S. Raymond pushed to branch master at NTPsec / ntpsec


Commits:
73f83ccc by Eric S. Raymond at 2016-12-23T13:40:25-05:00
Add documentation of the Python code.

- - - - -


1 changed file:

- devel/tour.txt


Changes:

=====================================
devel/tour.txt
=====================================
--- a/devel/tour.txt
+++ b/devel/tour.txt
@@ -408,4 +408,97 @@ a loadable Python module.  Here are a few useful generalizations:
   You will usually be able to model your implementation on code that
   is already there.
 
+== The Python tools ==
+
+Project policy is that (a) anything that does not have to be written
+in C shouldn't be, and (b) our preferred non-C language is Python.
+Most of the auxiliary tools have already been moved.  This section
+describes how they fit together.
+
+== The pylib/ntp library
+
+The most important structural thing about the python tools is the
+layering of the three most important ones - ntpq, ntpdig, and ntpmon. 
+These are front ends to a back-end library of Python service routines that
+installs as 'ntp' and lives in the source tree at pylib/. 
+
+=== ntpq and ntpmon ===
+
+ntpq and ntpmon are front ends to back-end class called ControlSession
+that lives in ntp.packet.
+
+ntpq proper is mostly one big instance of a class derived from
+Python's cmd.Cmd. That command interpreter, the Ntpq class, manages an
+instance of a back-end ControlSession class.  ControlSession speaks
+the Mode 6 control protocol recognized by the daemon.
+
+The cmd.Cmd methods are mostly pretty thin
+wrappers around calls to eight methods of ControlSession
+corresponding to each of the implemented Mode 6 request types.
+
+Within ControlSession, those methods turn into wrappers around
+doquery() calls.  doquery() encapsulates "send a request, get a
+response" and includes all the response fragment reassembly, retry,
+and time-out/panic logic.
+
+ntpmon is simpler.  It's a basic TUI modeled on Unix top(1), mutt(1)
+and similar programs.  It just calls some of the ControlSession
+methods repeatedly, formating what it gets back as a live display.
+
+The code for making the actual displays in htpq and ntpmon mostly
+doesn't live in the front end.  It's in ntp.util, well separated from
+both the command interpreter and the protocol back end so it can be
+re-used.
+
+=== ntpdig ===
+
+ntpdig also uses the pylib library, but doesn't speak Mode 6.
+Instead, it builds and interprets time-synchronization packets 
+using some of the same machinery.
+
+=== MRU reporting ===
+
+The mrulist() method in ControlSession is more complex than the rest of the
+back-end code put together except do_query() itself.  It is the one part
+that was genuinely difficult to write, as opposed to merely having high
+friction because the C it was translated from was so grotty.
+
+The way that part of the protocol works is a loop that does two
+layers of segment reassembly.  The lower layer is the vanilla UDP
+fragment reassembly encapsulated in do_query() and shared with the
+other request types.
+
+In order to avoid blocking for long periods of time, and in order to
+be cleanly interruptible by control-C, the upper layer does a sequence
+of requests for MRU spans, which are multi-frag sequences of
+ASCIIizations of MRU records, oldest to newest.  The spans include
+sequence metadata intended to allow you to stitch them together on the
+fly in O(n) time.
+
+There is also a direct mode thsat makes the individual spans available
+as they come in.  This may be useful for getting partial data from
+very heavily-loaded servers.
+
+A further interesting complication is use of a nonce to foil DDoSes by
+source-address spoofing.  The mrulist() code begins by requesting a
+nonce from ntpd, which it then replays between span requets to
+convince ntpd that the address it's firehosing all that MRU data at is
+the same one that asked for the nonce. To foil replay attacks, the
+nonce is timed out; you haveto re-request another every 16 seconds
+(the code does this automatically).
+
+The Python code does not replicate the old C logic for stitching
+together the MRU spans; that looked pretty fragile in the presence of
+span dropouts (we don't know that those can ever happen, but we don't
+know that they can't, either).  Instead, just brute-forces the problem
+- accumulates all the MRU spans until either the protocol marker for
+the end of the last one or ^C interrupting the span-read loop, and
+then quicksorts the list before handing it up to the front end for
+display.
+
+There's a keyboard-interrupt catcher *inside* the mrulist() method.
+That feels like a layering violation, but nobody has come up with a
+better way to partition things.  Under the given constraints there may
+not be one.
+
 // end



View it on GitLab: https://gitlab.com/NTPsec/ntpsec/commit/73f83ccc25143cabc79e5ce6e2aee010886a65a6
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ntpsec.org/pipermail/vc/attachments/20161223/7a1d6de3/attachment.html>