Yellow alert to all hands - please intensify review and live testing

[Eric S. Raymond]

The NTPsec code is, of necessity, entering a risky time.  Everyone
should go to heightened scrutiny and more frequent live testing of new
versions.

The following expands on some explanation I gave Hal Murray in a
recent reply.

While writing nts.c I realized that the NTS support is going to
require some significant changes to the packet-handling infrastructure
in the rest of the code.  These will not be safe changes; they could
easily introduce subtle and serious bugs.

Hence the need for other people to eyeball the code I will be pushing
in the near future more carefully than usual, and to stay on top of
changes to the repository head, and to do more live-testing, and to be
prepared to bisect if something breaks.

The good news is I don't expect the risky period to last long - I
expect in the not-too distant-future to be able to tell everyone they
can relax to normal levels of caution.  So that everyone understands
why, let me explain what's going on.

While working on the stub module for NTS support I got my nose rubbed
in the fact that the C code in ntpd is going to have to do real things
with NTP packet extension fields (because that's where NTS cookies are
carried).  That knocks over some dominos.

The first one to fall is that a job Daniel (wisely) ducked after the Great
Refactoring of the protocol engine has to get done.  That is unifying
the old ("insane", struct pkt) with the new ("sane", struct
parsed_pkt), because only the latter has actual support for extensions
other than the MAC field.

The second to fall is that the ugly type punning around refid fields -
sometimes treated as char refid[4], sometimes as uint32_t - has to
end.  That's a prerequisite for moving everything to the parsed-packet
representation.

I have a patch ready fixing all the refid uses - I'm rechecking it now
and expect to push it soon after I send this mail.  But doing so will
take us into the risky time.

Then I must do the rest of the spadework to unify those packet
representations.  That patch series is going to be messy
and somewhat dangerous.

Once done with that, however, we'll be back to where the *rest* of the
NTS changes are not very risky.  I'll notify the devel list when
everyone can stand down a bit. It will be days, not weeks.

What comes after:

Once I get those packet representations unified, I need to write
primitives for adding extension fields to a packet and extracting
them.

After *that*, it will be time to modify the ntpd config parser to
supply all the configuration info needed for the NTP client and server
sides.

Then I need to write everything about ntsked except the actual packet
processing.  A baby C framework for a TCP/IP listener, basically. I'm
hoping I'll be able to swipe this from somebody else's code.

You notice I'm talking about everything but the NTS packet processing
itself.  Hal Murray and James Browning and Gary Miller and others have
developed better understanding of that than I have yet - in part
because I've been preserving my ignorance in order to be able to
review the first-cut implementation with fresh eyes.

I, on the other hand, have been more intimate with the config parser
and packet-bashing and the other bits of C infrastructure NTS needs to
use than the rest of you have.  It's efficient for me to focus on
that, as well as keeping an eye on architectural simplicity.

So I'm making it my job to define the NTS code's interface to the rest
of that infrastructure, and then line up all the ducks outside nts.c
in neat rows. While that's going on, others can be filling in the
RFC-driven bits in nts.c.  My goal for what's there now was just to
define enough of it so the people who have been studying NTS more
closely than I have so far can be productive.

Have fun and stay safe out there.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

"Guard with jealous attention the public liberty.  Suspect every one
who approaches that jewel.  Unfortunately, nothing will preserve it
but downright force.  Whenever you give up that force, you are
inevitably ruined."	-- Patrick Henry, speech of June 5 1788