Using Go for NTPsec

[Eric S. Raymond]

Hal Murray <halmurray at sonic.net>:
> More topics for this discussion:
> 
> What platforms is the new environment supported on?  See my reply to Sanjeev 
> Gupta's message.
>
> As far as I can tell, we don't have a list of supported/tested platforms.  Is 
> there an official we page that describes what we support?  (I'm expecting 
> something like POSIX-enough.)

From: https://golang.org/doc/install/source#environment

The valid combinations of $GOOS and $GOARCH are:

$GOOS	$GOARCH
aix	ppc64
android	386
android	amd64
android	arm
android	arm64
darwin	amd64
darwin	arm64
dragonfly	amd64
freebsd	386
freebsd	amd64
freebsd	arm
illumos	amd64
ios	arm64
js	wasm
linux	386
linux	amd64
linux	arm
linux	arm64
linux	ppc64
linux	ppc64le
linux	mips
linux	mipsle
linux	mips64
linux	mips64le
linux	riscv64
linux	s390x
netbsd	386
netbsd	amd64
netbsd	arm
openbsd	386
openbsd	amd64
openbsd	arm
openbsd	arm64
plan9	386
plan9	amd64
plan9	arm
solaris	amd64
windows	386
windows	amd64

Note the Windows support.

In addition, I believe they have RISC-V support in progress.

> In addition to the barrier to entry, there is also the barrier to re-entry.
>   Friends mentioned it as a disadvantage of python.  It's a pain to work on 
> code that you haven't worked on for a while.
>   I can confirm that.  I find ntpq hard to work on.

I do too, though I remember the C version as being even more
difficult.

I'm pretty sure this is a problem with the ntpq code, not with Python
- Python in general has a reputation for being *easy* to read six
months later, which I think is deserved.  It's one of the first things
I noticed when I started coding in Python back in 1998 or so.

On the other hand, I know from reading patches other people shipped
for the original Python version of reposurgeon that if you go all-out
trying to write the most compact code possible, especiall using
comprehensions and other functional-programming idioms, you can
produce Python that is actually hard to read.  You have to exert extra
cleverness and effort, but it is possible.

Maybe this is what your informants bumped into.  I know that in its
Python days there were a few times I had to ask reposurgeon
co-developers to, er, curb their enthusiasm a little - usually after I
rejected a patch that made my eyes cross.  It was always possible to
express what they were after in similar code, Python is good that way
- they just had to want to do it...

I know from reading what the Go devs have written that they made a
conscious choice to design simple syntax and a small set of
first-class operations, deliberately trading away expressiveness, in
order to make the language easier to read and improve long-term
maintainability.

I don't know if they had been influenced by having seen overly-clever
Python, but Go is designed as if they were trying to prevent exactly
that failure mode.

I'm not myself entirely certain this was a good tradeoff!  But I think
they achieved what they were after - I find Go very easy to read.  I
have yet to encounter a Go equivalent of excessively-clever Python.

BTW, I will not strongly assert that this is an advantage over Rust,
because I never grokked Rust well enough to make a really fair
comparison.  But Rust requires a lot of ceremony in the form of
lifetime declarations that neither Python nor Go does; a priori this
probably does put it at a readability disadvantage.

> There is a bug in our ntpq, or was not-that-long-ago and I'm pretty sure it 
> hasn't been fixed.  The problem is recovering from lost packets.  If it's 
> collecting a batch of packets and one gets lost, it returns none rather than a 
> partial clump up to the lost packet.  A couple layers up when it should be 
> dropping the size of the clump it goes off on another path and just tries 
> again.

I'm aware.  Some time back I spent a day hunting for that bug.  I
couldn't find it.  That's a nasty thicket of code in there.

> It would be interesting to poke around and see if that bug was in
> the original C version or if the bug was introduced when translating
> to python or while working on the python version.

We've had this conversation before.  I think we determined that it is
in the C original. Which would make sense - I did a very literal
translation.

> You use "memory safe" often enough without actually discussing what
> it covers.  I sometimes wonder if there are other properties we
> should be considering.

I mean that in Go it is not possible to overwrite memory outside of
a data structure or buffer by deliberately or accidentally computing
a wild pointer.  This of course closes off a very large class of
exploits.

It is still possible to abort a Go program by chasing a null pointer.
But the potential for DDoSes is low because the standard library is
all memory-safe code.  You have to work harder to find a code defect
that will pass a zero pointer than you would in C.  I've only ever
gotten there by accidentally dereferencing a pointer-valued variable
that had been declared but not yet initialized.

There is a stricter sense of "memory-safe" that means you can never
generate an invalid pointer at all.  Rust is almost there, but not
quite.  It has static null-pointer safety (you normally have to
initialize a pointer to a valid target location at declaration time)
but not dynamic safety. That would require a runtime check, and
Rust makes a point of not having a runtime.

> I put near-optimal performance high on the list.  My attempt at some
> simple Rust timing code got surprisingly high numbers for reading
> the clock.  It may have been loop overhead.  Manually replacing an
> iterator style loop (the clean way) with an ugly by-hand counter got
> somethig reasonable.  I haven't tried to figure out what is/was
> going on.  (yet)

That's a surprising result.  One of Rust's most touted advantages is
low overhead for close-to-the-metal code.  I wouldn't have expected it
to compile fat code from an iterator.

> > I gave up the experiment when I discovered that Rust had no equivalent of
> > poll(2)/select(2). 
> 
> I poked around a bit and didn't find anything that looked good.  I
> assume you wanted to wait for network data.  Did you consider a
> thread per socket?

I did, but I was trying to write a simple proof of concept.  Not a good
time to introduce threading.

> My experience is that my experiment of similar trivial size was a
> success.  Was that because I picked a problem area that was easier
> in some measure, or maybe I just got lucky and all the library
> support I needed was already there.

My money would be on the latter.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>