Using Go for NTPsec

[Eric S. Raymond]

Hal Murray <halmurray at sonic.net>:
> Thanks for taking the time to explain things to me.

Why would I not?  It's *good* to have someone on the projects who's sufficiently
smart and stubborn to question my premises - please never stop doing that!  I know
more about the problems around getting to a memory-safe language after this
conversation than I did before it, and that is excellemt.

> It would be interesting to peer into your brain and mine to figure out how 
> much of our reasoning was making a choice because we like it at first glance, 
> and then collecting arguments to justify that choice.

I wanted to like Rust. I really did.  Read the Rust book through
twice, was very excited by the degree of provable correctness it
seems to offer.  Then I tried programming in it.

Here's what I did.  At the time I had recently been hearing some
serious bitching about IRC servers not scaling well.  So, as an experiment, I
decided to try writing the frame for an IRC server in both Rust and Go.
Not the protocol state machine, but the wrapper that accepts connections,
passes incoming lines to a processing hook, and then ships the output of the
processing hook to all connected clients.

It was almost a trivial exercise in Go.  About 60 lines IIRC.  Looked
like good code, compiled.  To be fair, I didn't test it - I was
looking to scope complexity, not actually start a production
server. Yet.

Then I tried it in Rust.  I couldn't get it done.  At all.  The
friction cost of writing even tiny amounts of code was horrifying -
and I'm a guy who normally eats new languages like candy.

I gave up the experiment when I discovered that Rust had no equivalent
of poll(2)/select(2). This toy project literally *could not be done*.
It could now; they've since standardized (as much as Rust-world is
capable of standardizing, see below) on a crate called tokio that
provides equivalent abstractions.  But this experience left me with a
very bad taste in my mouth about the barrier to entry.

I also got a serious scare about API stability, because I learned how
the Rust community makes decisions about core language and library
features.  Do you remember Conway's Law? Organizations design systems
that mirror their own communication structure.  And, it could be added
today, mirror the affordances of their communications tools.

The good news is Rust crates are the best downloadable-module system
I've ever seen.  The bad news is that Rust culture is all about using
their decentralizing power power to swarm-attack library-API design
without having any benevolent dictator with the social authority to
say "Enough. Stop."

The result is that Rust is years away from being able to define a
stable set of API commitments.  It may never be able to do so at all,
at least not within its current sociology.  I'm keeping an eye on it
for signs that somebody is successfully exerting enough control to
stabilize the language and API, but at time of writing there seems to
be no prospect of this.

The compiler is more stable than the libraries, but only because it's
not modular. It *has* to have a lead maintainer and a small core group
and a test suite in a way that the library collection does not. These
things have a braking effect on feature churn.

By contrast: You know the joke about a helicopter being a collection
of parts flying in loose formation?  The Rust "standard" API is a
collection of crates flying in loose formation.  It has fuzzy,
unstable membership, stuff shows up and drops out as the Rust
community explores the design space.

Thus: Tokio is the thing now, but if somebody else comes up with a
roughly equivalent thing that the community decides is "better", Tokio
could fall out of leading-edge use and then bit-rot as it's abandoned.
There's nothing in Rust's institutional design to prevent this - no
group of core devs that owns and defends a core standard library.

This is great strategy when your language is in the explore phase.
The Rust community has optimized itself to be an exploration engine.
But it is utterly freaking *terrifying* if you have decadal-scale
maintainability and toolchain-stability concerns.

I discussed this experiment on my blog.  Rust fanboys showed up in
comments to berate me for Not Getting It. Also some core Rust devs
showed up, claimed to be embarrassed by the fanboys, and allowed that
Rust isn't ready for the kind of deployment I do.

I wasn't only thinking about NTPsec.  There's GPSD too.  It's smaller;
I probably would have moved GPSD first.

(I never considered moving GPSD to Go very seriously, because it has
an unusual property: the data flow is simple enough that dynamic
allocation and GC are very little help.)

> I really really really like type checking.  Rust seems interesting.  I want to 
> know more.

Rust's type-checking is certainly one of its strong suits.  Strictly better than
Go's in the sense of foreclosing more errors.  I'd say the other is what they
call "zero-overhead abstractions". Rust may have a more expressive type system
than Go, too - there are some tricky issues around evaluating that.

Unfortunately, I found the price for these virtues too high.  I wish it had
been otherwise.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>