Fuzz, Numbers

Mon Jan 13 06:46:43 UTC 2020

and without 'limited' on ~5kpps I have 8-10% CPU regardless minitoring 
enabled/disabled. About 1% on 1000pps.
(Hardware is old MS-9258 server, CPU Quad CPU Q940, FreeBSD 12.1)

As I see many limited queries really sourced from NAT, and we cannot 
determine whether they are correct or not. So for production server 
better not have 'limited' or have limited to tens queries per second. 
And maybe limit only ip:source port, not only 'per ip' because we have 
different source ports from NAT and identical port on "dumb" clients. 
But we cannot set such settings.. To protect against participation in 
DDoS, you can use traffic restriction with a firewall to 1-5Mbit/s. 
Every 1k queries takes <1Mbps of bandwidth.

For those who want to process hundreds of thousands of requests per 
second (like 'national standard' servers) you can use multithreading and 
multiply power of server. As I know professional solutions like Meinberg 
Lantime can run multithreading, but no opensource daemons can do it. 
NTPPoll community have poses about good expirience with 
https://github.com/mlichvar/rsntp (look at 
https://community.ntppool.org/t/can-i-incrase-number-of-threads-to-use-in-ntpd-proccess/1159/20) 
.

Maybe when there will be absolutely nothing to do you can write some 
proxy-balancer that solves this task as official utility :)

Have a nice day!

--
Mike Yurlov

09.01.2020 13:52, Mike Yurlov via devel пишет:
> Hi, Hal!
>
>
> I build ntpd from latest sources tonight. CPU load drops from 18-20% 
> average to 5-6% on my ~3-4k pps. Looks perfect!
> If you get race with "init before config read", you can create build 
> option for the init size of the mrulist.
>
> Here the stats from nigth to 13:00 (GMT+3):
> recieded 173 647 480 packets, 3.1kpps average (real from 2.5 to 6kpps 
> i see on network interface),
> 1.8% bad, 21% ratelimited, 77% processed
>
>
> ntpq> sysstats
> uptime:                 55394
> sysstats reset:         55394
> packets received:       173647480
> current version:        76272783
> older version:          57692039
> control requests:       1516
> bad length or format:   3287409
> authentication failed:  3955
> declined:               3199
> restricted:             388
> rate limited:           36398991
> KoD responses:          0
> processed for time:     133953537
>
> ntpq> monstats
>
> enabled:                2
> hash slots in use:      158963
> addresses in use:       290909
> peak addresses:         290909
> maximum addresses:      290909
> reclaim above count:    600
> reclaim maxage:         250
> reclaim minage:         240
> kilobytes:              25000
> maximum kilobytes:      25000
> alloc: exists:          133311968
> alloc: new:             290909
> alloc: recycle old:     35498556
> alloc: recycle full:    1162596
> alloc: none:            150665
> age of oldest slot:     240
>
>
> Some request strange and I don't know is this NAT or not.
>
> This one looks like many clients over NAT
> 13:17:31.160400 IP 90.188.255.3.42962 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:17:31.312476 IP 90.188.255.3.51241 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:17:31.482878 IP 90.188.255.3.55666 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:17:31.570783 IP 90.188.255.3.38018 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:17:31.596582 IP 90.188.255.3.36581 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:17:31.776522 IP 90.188.255.3.42962 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:17:31.928548 IP 90.188.255.3.51241 > x.x.x.x.123: NTPv4, Client, 
> length 48
>
> But than it looks like woodpecker :)
> 13:19:24.257556 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:24.917559 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:25.533525 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:26.157515 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:26.769554 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:27.381551 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:28.001559 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:28.617574 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:29.237470 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:29.853630 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:30.469565 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:31.081622 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:31.705618 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:32.321652 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:32.945589 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:33.025639 IP 90.188.255.3.46163 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:33.573548 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:33.661612 IP 90.188.255.3.46163 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:34.193647 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:34.273687 IP 90.188.255.3.46163 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:34.809651 IP 90.188.255.3.39114 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:19:34.897663 IP 90.188.255.3.46163 > x.x.x.x.123: NTPv4, Client, 
> length 48
>
> many clients look buggy or installed behind firewall. It request 3-5 
> times once per second, do 1-2 sec pause and repeat cycle. ntpd 
> ratelimit it and reply once on every cycle, but it send request again 
> and again. Many such clients make ~100k requests per day. I think to 
> answer to such requests are a waste of hardware resources and network 
> bandwidth worldwide.
>
> 13:27:02.246352 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:02.246384 IP x.x.x.x.123 > 77.222.101.171.123: NTPv4, Server, 
> length 48
> 13:27:02.278056 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:03.245720 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:04.246223 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:06.840038 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:06.840064 IP x.x.x.x.123 > 77.222.101.171.123: NTPv4, Server, 
> length 48
> 13:27:06.869703 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:07.840540 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:08.841967 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:11.440866 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:11.440883 IP x.x.x.x.123 > 77.222.101.171.123: NTPv4, Server, 
> length 48
> 13:27:11.480807 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:12.442444 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:13.437732 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:16.012160 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
> 13:27:16.012188 IP x.x.x.x.123 > 77.222.101.171.123: NTPv4, Server, 
> length 48
> 13:27:16.048975 IP 77.222.101.171.123 > x.x.x.x.123: NTPv4, Client, 
> length 48
>
> Such clients suggest that a mrulist is still needed.
>
> And of cource several times per dat I recieve definitely flood with 
> hundreds similar requests per second from one ip.
>
>
> -- 
> Mike
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
>