ntpq meets log files

[James Browning]

On Mon, Mar 8, 2021 at 2:28 PM Hal Murray via devel <devel at ntpsec.org>
wrote:

> I'm poking around in ntp_control because I want to add the CPU time that
> ntpd
> has used for some tests I'm trying to run.
>
> We have our share of crufty code, but this is the stuff that annoys me the
> most.  What should be simple turns into a pain in the ass.
>
> -------------
>
> We have several log files that record statistics hourly.  After they write
> out
> a line, they clear the counters.  ntpq can't ask for the totals since
> restart
> - the data is gone.
>
> James has recently fixed part of that.  ntpq sysstats now prints 2 columns.
>
> I'd like to open this area up for discussion.
>
> I'd like the counters behind all log files to keep the totals too.  There
> are
> 2 ways to implement that.  One is for the active counters to be the
> totals.
> The hourly logic would maintain a second copy that holds the value to be
> subtracted from the totals to get the this-hour values.  The other is for
> the
> active counters to hold the this-hour values.  To get the totals you would
> add
> the values from the second copy.  I think I prefer the active counters to
> hold
> the totals but I have not strong preferences.
>
> We need to keep in mind how to make this work in a multi threaded
> environment.
>

I think you would need to deglobalize some variables first.


> It would be neat to have 2 more columns - packets per second.  I don't
> know if
> that will fit.
>
> I'd like to have a ntpq command for each log file.  I like the 2 column
> approach.  Currently, the slots in each column have a separate name.  We
> could
> compress that by returning something like:
>   foo=23,45678
> that is 2 values for one name.
>

Not until the ramp to 2.0, That screams here lie incompatible changes.


> ntpq could save a copy of the data so it could print the since-xxx values.
>
> We could get last-day values by saving 24 copies of the last hour data.
>
> There is at least one comment in ntpq.py saying, roughy, this table should
> be
> in ntpd so that we don't have to edit here too when something changes.
> "rv 0"
> returns a subset of a big table marked "default".  We could easily add
> more
> flags, one for each log file.
>
> There is a slight complication.  We need the name that comes back with a
> variable to be a meaningful description.


So, drop the Millsian object notation (or whatever it is called) for
something closer to JSON? Protobuf need not apply.


> What's the best way to clean up this mess?  We discussed this before, but
> I
> don't remember any consensus, or anybody stepping up to do their favorite
> way.
>

I messed up a little which is where the CV_NAME and CASE_ macros came from.


> The current approach needs 3 (I think) tables.  One to define a set of
> tags,
> one to translate the name of each variable to a tag, and the 3rd to print
> out
> the data for that tag.
>

4+ tables, at least 3 long lists of preprocessor macros, and too much
copy/paste code.


> I propose we dump the tags and put everything for each slot into struct.
> What does it need?
>   name that ntpq uses to ask for this slot
>   flags
>   pointer to value to be printed
>   routine to print it
>
> We can setup macros to build the struct.
>
> Does that make sense?
>

Not really, but I am not very effective. I would like to be able to
implement something like the following, but it is probably too fussy...

For example, after butchering all the following it occurred to me that one
could just ad a CB (callback) or HOOK value. I mean it is not like there is
a sudden lack of space for them.

```py
clock_var_e = {
    'name': { RO|DEF|QSTR, 'clockname' },
    'timecode': { RO|DEF|QSTR, 'p_lastcode' },
    'poll': { RO|DEF|UINT, 'polls' },
    'noreply': { RO|DEF|UINT, 'noresponse' },
    'badformat': { RO|DEF|UINT, 'badformat' },
    'baddata': { RO|DEF|UINT, 'baddata' },
    'flags': { RO|DEF|UINT, 'flags' },
}

clock_var_h = {
    'fudgetime1': { RO|DEF, clock_do_fudgetime1 },
    'fudgetime2': { RO|DEF, clock_do_fudgetime2 },
    'stratum': { RO|DEF, clock_do_stratum },
    'refid': { RO|DEF, clock_do_refid },
    'device': { RO|DEF, clock_do_device },
    'clock_var_list': { RO, clock_do_varlist },
}

def read_clockstatus(buffer, statusmask):
    if not (REFCLOCK)
        ctl_error(CERR_BADASSOC)
        return

    if res_id == 0:
        peer = findpeerbyassoc(res_associd);
    else:
        if sys_vars.sys_peer and
FLAG_REFCLOCK&sys_vars.sys_peer.cfg.flags)):
            peer = sys_vars.sys_peer;
        else:
            for (peer = peer_list; peer; peer = peer->p_link):
                if (FLAG_REFCLOCK & peer->cfg.flags)
                    break;
    if not (peer and peer.flags&FLAG_REFCLOCK)
        ctl_error(CERR_BADASSOC)
        return
    refclock_control(&peer->srcadr, NULL, &cs);

    qset = set()
    gotvar = False
    iset = set(qstr.split(','))
    for key in iset:
        if key in clock_var_e or key in clock_var_h:
            qset = qset.add(key)
        else:
            ctl_error(CERR_BADVALUE if key == '' else
                          CERR_UNKNOWNVAR);
            return

    if gotvar:
        // This used to be putclock
        if id in clock_var_e:
            handler = handlers[clock_var_e[id][0] & FMASK]
            handler(id, clock_var_e[id][1])
        elif id in clock_var_h:
            clock_var_h[id][1](id)
    else:
        for key in clock_var_e:
            if DEF&clock_var_e[key]:
                handler = handlers[clock_var_e[id][0] & FMASK]
                handler(id, clock_var_e[id][1])
        for key in clock_var_h:
            if id in clock_var_h and DEF & clock_var_e[id][0]:
                clock_var_h[id][1](id)
    flushpkt()
```
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ntpsec.org/pipermail/devel/attachments/20210309/8c90547f/attachment.htm>