GC timing

[Eric S. Raymond]

Hal Murray <halmurray at sonic.net>:
> If you pass in a buffer, there is no reason to allocate anything in the case 
> of a server processing a request so this whole discussion is a wild goose 
> chase.

It's a little more complicated than that, because I was describing the
lowest-level recvfrom() in the socket library.

Even if we use that for the dumb phase 1 translation, it might be
right in phase 2 to move to the higher-level interfaces in the ipv4
and ipv6 libraries. Some of those dynamically allocate buffers.

In thinking this through, I want to be reasonably sure that not just
the phase 1 but phase 2 implementation isn't going to take a sync
accuracy hit. Thus, I'm trying to make worst-plausible-case
assumptions.

> Two orders of magnitude is 10K packets/second.  That's in the interesting 
> range.  Our current code can do 400K non-auth packets/second.  That's only 40% 
> of wire speed on a gigabit link.  NTS is 90K packets/sec, 25% of wire speed.
> 
> Memory speeds are ballpark of 20 GB/sec.  So if you have a GB of headroom, it 
> will take at least 50 ms just to scan that.
> 
> NIST servers are running 10E10 packets/day.  That's averaging 115K 
> packets/second.
> 
> https://tf.nist.gov/general/pdf/2818.pdf

A NIST-grade server can afford to dedicate a lot of RAM and then set
the GOGC knob to a high value that trades having a bigger working set
for a longer expected interval between GCs.

You can, actually, pick the mimimum GC interval you think is tolerable
and get it by pushing GOGC high enough.  Of course if you don't have
enough physical RAM memory-retrieval times will blow up due to
swapping, but these days putting a bunch of terabytes of RAM in a
datacenter-grade server isn't even unusual any more

115K 90-byte packets/second will fill a GB in about a tenth of a second.
Drop N terabytes of RAM on the problem and now you're looking at memory-full
intervals of about N * 100 seconds.  It's not going to take a large N
to make latency spikes a rare and minor blip in the traffic. 
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>