Thinking forward

[Eric S. Raymond]

This is a lightly-edited version of a briefing I just gave Hal Murray
off-list. Some of it's been discussed on the Signal channel.
Everybody else should should know what's going on, too

Mark and I have been thinking strategically about the medium and
long-term future of this project.  The era during which we could make
major gains by code removal is pretty clearly drawing to a close.
Going forward frrom 1.0 we are likely to have to proceed by adding
code more than subtracting it.

But we have about concluded that it's not really a good path forward
to add a lot of complexity to the C for either performance or other
reasons.  Instead, we are now seriously entertaining the idea of
stripping the C codebase down to the bare minimum that will still
work, isolating the platform dependencies - and then moving the whole
codebase to a language with better correctness guarantees and better
concurrency support.

Of course, the major point of the move would be to get to a place
where buffer overruns and wild-pointer bugs are impossible.  But
another effect would be to get us the use of concurrency primitives
that are much easier and safer to use.  We could use these, in
particular, to replace the rather alarming kludge that is the current
asynch-DNS lookup code.

This wouldn't have been practical starting from 227KLOC of grubby,
#ifdef-encrusted C. But we're now down to 66KLOC of much cleaner C and
likely to drop a few KLOC more (in particular, from moving ntpdig to Python).
Moving to another language, even if we had to do it by hand-translation,
is probably within the limit of practicality now.

And we probably wouldn't have to do it by hand.  The two candidate
languages we're considering, Go and Rust, have mechanical C translators.
The Rust one, called "corrode", is rumored to be production-quality.
The Go one was written to translate the Go compiler from C, and is advertised
to only translate C written in a restricted style. Which seems to mean
excluding unions and some kinds of gotos that are a bad idea anyway.

Which ties directly into the reason I've been pretty silent for the last
week. I've been learning Go - writing a replacement for David A. Wheeler's
sloccount utility. It has been quite the experience, and has left me
with a good feeling about the feasibility of moving our codebase to Go.

Mark is encouraging this research, though he quite rightly wants me to
evaluate Rust just as thoroughly before we make any major decisions.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

You know why there's a Second Amendment?  In case the government fails to
follow the first one.
         -- Rush Limbaugh, in a moment of unaccustomed profundity 17 Aug 1993