Updated to-do list?

[Hal Murray]

[Context is a multi-threaded Go echo server]

> I can manage to do that.

I'll send you the C code off list.

Here is an outline of the big picture:

Linux and FreeBSD have a SO_REUSEPORT option.  The idea is that you can open 
several sockets on the same port number.  The kernel will hash on source IP 
address and source port.  (so packets for a connection go to the same thread 
-- the thread can save state for a connection)

The main server thread sets up worker threads
  Each worker thread opens a socket
  The worker threads bump counters.
  The main thread will read/collect and print the info.

The worker threads have an option to spin for N microseconds between recvfrom 
and sendto.  The idea is to simulate crypto work.

The limiting factor on packet throughput is the kernel thread
  That thread puts arriving packets on the right socket queue
    and wakes up a waiting thread if there is one
  So larger packets reduce the load on the kernel thread for a given traffic 
load in bits/second (rather than packets/second).

Handwave...
  The CPU load for a NTP server is roughly constant in terms of bits/second.

So you want to assign the kernel thread to one core (leaving the other 
hyperthread idle) and disable interrupt coalescing.
Then fill up the rest of the CPUs on the chip with worker threads.

It's roughly a microsecond per packet for the kernel thread,
a microsecond each for recvfrom and sendto
and (handwave) a microsecond for basic NTP server work.

--------

Up a level...

There is a companion client side that runs several worker threads sending to 
the server.
You can run it on multiple old/slow PCs to get enough traffic to saturate the 
target server.

There is an option to send N extra packets, like ping -l.  I call them "in 
flight".

Things are setup assuming that no packets are lost.  So you have to be sane on 
picking combinations of multiple clients and extra packets in flight.

---------

Up another level...

I have some hackish scripts that
  fire up a server
  fire up several clients
  read/reset the server stats
  wait a bit
  read server stats
  print results
loop for various CPU load/delays or packet lengths


-- 
These are my opinions.  I hate spam.