Is it time to plan a move to Go?

[Eric S. Raymond]

Hal Murray <hmurray at megapathdsl.net>:
> > Thoughts?
> 
> Is Go well supported on ARM?  (how about powerpc which is other-endian?)

According to the Go documentation, both are fully supported.

> > * libntp is a problem - ideally we want to have the Python extension
> >   that wraps it call compile Go rather than C, but that could be
> >   difficult to arrange. 
> 
> Why do we need a python extension?  Can't we convert ntpclients to Go?  (and 
> drop python)

Yeah, that'd be be kind of a mess.  The problem is this: if you're
measuring by richness of type structure, dymamicity, and range of
support libraries *for our purposes*, Go > C, but Python > Go.

This has consequences. One is that C -> Go translation is pretty easy;
outside of dynamic allocation the idioms are quite similar, and near
it Go's idioms are simpler.  This means eyeball translation by a
skilled programmer can be relatively fast and have a tolerably low
defect rate.  It's the exact same reason I described Go as an easy
step up for experienced C programmers.

But for the same reasons, Python -> Go is *not* easy. I've studied
this in detail because I may yet have to move reposurgeon out of
Python for performance reasons, and Go looks like by far the best
place to land. Tough landing, though - stuffing code from a language
further over on the dynamic-typing/unlimited-extents axis into one
closer to static-typing/limited-extents is always going to be
difficult.

We might well end up all Go all the time.  But if we have to wait on
translating all the Python to Go to get there, fielding the Go ntpd
will take much longer.  Thus, as a transitional measure we're going to
either need to learn how to do Python extensions over Go or live with
a three-language implementation in which C persists all the way through
to all the Python getting moved.

There is this - https://github.com/go-python/gopy - to tell us the Go crowd
is aware of the problem and it's being worked on, and also this:
https://github.com/google/grumpy .  Early-stage code; neither is a
complete solution to the problem we're going to have.

Anyway, I'm not sure I'd want to get rid of Python if I could.  Go's
design tradeoffs are a *really* good match for ntpd, but Python is in
many ways a better fit for directly user-facing code like our clients.
The features that make Python > Go have more impact there.

But yes, this is the right kind of question to be asking.

> > * Not hard to see how to qualify most of the translation, but refclock
> >   drivers we don't have test hardware for are a source of worry. Can
> >   we do anything other than be meticulous and hope? 
> 
> Maybe that would be a good time to split the refclocks out to separate 
> programs.

Wouldn't make the problem go away.  Whether they're inline of ntpd or
separated, either they have to be translated to Go or C persists in the
language mix.

> Here is a quarter baked idea.
> 
> Write a dumb NTP server in Go.  The idea is to move ntpd to a different port 
> number so the new code can answer client requests.  This assumes that a 
> server can get all the info it needs from the kernel.  That may be bogus, but 
> I think it's worth a try.  We could write similar code in c and compare 
> performance.

Don't we already *have* similar code in C, namely ntpd itself?  I guess
I'm not grasping what you mean by "dumb".

> Should this happen before or after we have real crypto working?  If crypto 
> uses lots of CPU cycles the difference between c and Go won't matter as much.

I don't think the Go/C performance difference will matter under *any* plausible
scenario, not given what I've read about Go oerformance and reasonable
inferences about how its internals work.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.