Thoughts on networking and threads...

[Hal Murray]

I've been thinking about how to make ntpd serve lots and lots of clients.

I think that requires the server to be multi-threaded, especially if we want 
to support NTS.

It takes a microsecond or 2 for the basic network activity, and another 
microsecond for the server to fill in the response information.  NTS adds 
ballpark of 40 microseconds.

This looks like a classic example of an easily parallelizable problem.  Server 
hardware comes with lots of cores.  12 and 16 are typical.  Seems like a good 
match.

An interesting article here:
  The SO_REUSEPORT socket option
  https://lwn.net/Articles/542629/

I've been experimenting with a simple echo server.  On a good day, with a bit 
of tailwind, I can echo 1,000,000 packets per second on a PC with 4 cores 
running at 3.60GHz.

Eric: How is your Go translator going?  I'd like to try some basic networking 
experiments.

----------

I think we should split ntpd into 3 chunks: client, server, and mode6.  
Currently, they are all tangled up at the receive packet processing.  I don't 
mean 3 separate programs, but won't rule that out.  The main idea is to 
understand where they interact.

There is a draft RFC to have the client use something other than port 123.  
Splitting the client and server gets that.

Using threads will clean up the packet processing at the cost of adding locks. 
 I think the locks will be simple -- after we understand where they interact.

The client can stay single threaded.  I haven't looked carefully, but the 
server doesn't need much data from the client side so I'm pretty sure we can 
make a clean handoff with a simple lock.  The server only reads that data.

It's possible that it might be easy and clean to split the client and server 
into separate programs.  The client feeds a lot of into to the kernel where 
the server can get it back.  I'll have to check on what that covers.

It will probably help to rearrange the header files, something like network, 
utility, client, and server.

If we start using threads, it will be easy to give each refclock a thread.  
Again, the critical step will be understanding how it interacts with the main 
client side thread.  For example, the server side doesn't need to know 
anything about the peer struct.

Humm.  I wonder if the client side gets cleaner if we have a thread per 
server.  That's probably a big step toward fixing the 1 second polling mess 
that keeps laptops from power saving.

I haven't thought much about mode6.  I'm willing to discuss anything.  We 
could shift to TCP to avoid the DDoS amplification issues.  If we stick with 
UDP, we need a new port number.  The current code is mostly ASCII on the wire. 
 Should we fix the rest of the binary data?  Note that there are 2 parts to 
mode6, the client (peers) and the server (mrulist).

--------------

When I started typing this, I was focused on the socket level interface.  The current receive dispatching seems pretty ugly to me.  Splitting the client and server was a way to clean that up.  And I wanted many threads for the server.  But the list above kept getting longer as I typed things in.

Maybe we should start from scratch.  Why bother with a Go translator?

Maybe we should go help with chrony.



-- 
These are my opinions.  I hate spam.