 in HTML

> It's not working for me in either the man pages or the web pages.

Please point me at a particular instance; I'll see what I can do to
trace the problem.

> The web page for ntpq has a bunch of links to other programs at the top.  So 
> do several other web pages for programs.  I assume that's a bug, but don't 
> know what the right fix is.

Dave Mills designed it that way.  It wouldn't be difficult to change, but
I'm trying to avoid moving away from his look and feel before we have
him recruited/co-opted.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Sun Nov 22 05:02:44 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sat, 21 Nov 2015 21:02:44 -0800
Subject: Testing
In-Reply-To: Message from "Eric S. Raymond" 
 of "Sat, 21 Nov 2015 23:42:04 EST." <20151122044204.GA21002@thyrsus.com>
Message-ID: <20151122050244.B7864406057@ip-64-139-1-69.sjc.megapath.net>

esr at thyrsus.com said:
> Sadly, I don't know of any good tutorials on this.  It is a swamp full of
> razor blades for even very experienced C programmers.  If you have a
> particular exporession you want me to analyze I might be able to say
> something useful. 

The old code was:

        int chars;
        char req_buf[CTL_MAX_DATA_LEN];

                req = req_buf;
                req_end = req_buf + sizeof(req_buf);
#define REQ_ROOM        (req_end - req)

                        chars = strlen(buf);
                        if (REQ_ROOM - chars < 1)
                                break;

The new code is:

        size_t chars;

                        if (REQ_ROOM <= chars)
                                break;

The old if didn't work after chars changed to size_t.

-- 
These are my opinions.  I hate spam.

From fallenpegasus at gmail.com  Sun Nov 22 05:08:22 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Sun, 22 Nov 2015 05:08:22 +0000
Subject: Testing
In-Reply-To: <20151122044204.GA21002@thyrsus.com>
References: <20151122024047.2D6C2406057@ip-64-139-1-69.sjc.megapath.net>
 <20151122044204.GA21002@thyrsus.com>
Message-ID: 

On Sat, Nov 21, 2015 at 8:42 PM Eric S. Raymond  wrote:

>
> The second (capture test data for every bug where it makes sense) is
> already
> in my goal set.  The first (apply TESTFRAME to other programs rather than
> just
> ntp) is going to be tricky and I don't have a detailed technical plan for
> it
> yet.  I'll look at this again after TESTFRAME lands.
>

It can be done with a LD preload that precatches the standard library
and/or syscalls.

It has the downside of being very OS specific: pick ahead of time Linux or
one of the *BSDs.

A project to start from for it as inspiration could be libeatmydata,
written by my friend Stewart Smith:
https://flamingspork.com/projects/libeatmydata/

And it's close relative, libhostile, maintained by my friend LinuxJedi:
https://github.com/libhostile/libhostile

BTW, giving the libhostile treatment to ntpsec will likely prove to be
amusing.

> > We don't get any compiler warnings.  Is that because our code is clean or
> > because we don't have enough flags turned on?
>
> One of the first things I did after the fork point was clean up all
> warnings.
> I'm a bit obsessive about that, having exoperienced too many situations
> where
> serious errors got lost in the clutter.
>
> We still build with -Wall, so the code really is clean that way.
>

There are additional warnings that are not turned on with -Wall.  They can
be turned on with -Wextra, which may have caught this because it enables
*-Wsign**-compare and **-Wsign**-conversion.*

..m
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From hmurray at megapathdsl.net  Sun Nov 22 05:27:27 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sat, 21 Nov 2015 21:27:27 -0800
Subject: doc quirks
In-Reply-To: Message from "Eric S. Raymond" 
 of "Sat, 21 Nov 2015 23:49:07 EST." <20151122044907.GB21002@thyrsus.com>
Message-ID: <20151122052727.8A13F406057@ip-64-139-1-69.sjc.megapath.net>

[+foo+]

> Yes, that is supposed to map to const-width font -  or  in HTML

from docs/includes/ntpq-body.txt

+-w+, +--wide+::

from build/docs/ntpq.html

-w, --wide

from build/ntpq/ntpq.1

.RE
.PP
\-w, \-\-wide
.RS 4

It's possible that something is broken on my asciidoc setup.

This system is Fedora 22
It says asciidoc.noarch 8.6.8-6.fc21 is installed.
That fc21 looks fishy, but I'm pretty sure it is the latest.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Sun Nov 22 05:50:16 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sat, 21 Nov 2015 21:50:16 -0800
Subject: doc quirks
In-Reply-To: Message from "Eric S. Raymond" 
 of "Sat, 21 Nov 2015 23:49:07 EST." <20151122044907.GB21002@thyrsus.com>
Message-ID: <20151122055016.3ED5D406057@ip-64-139-1-69.sjc.megapath.net>

> Dave Mills designed it that way.  It wouldn't be difficult to change, but
> I'm trying to avoid moving away from his look and feel before we have him
> recruited/co-opted. 

Here is a sample ntpq page from NTP Classic, starting after the cartoon:

More Help
[HR]
Synopsis
ntpq [-46dinp] [-c command] [host] [...]
Description

The ntpq utility program is used to ...

Here is the top of our ntpq.html:

More Help
[HR]
Program Manual Pages
[HR]
    ntpd - Network Time Protocol (NTP) daemon
    ntpdig - Simple Network Time Protocol (SNTP) client
    ntpq - standard NTP query program
    ntpfrob - frob the local clock hardware
    ntpkeygen - generate public and private keys
    ntpleapfetch(8) fetch and manage leap-offset file
    ntpdsim(8) - Network Time Protocol (NTP) simulator
    ntptime - read and set kernel time variables
    ntptrace - trace a chain of NTP servers back to the primary source
    Site Map
(There are links behind most of those text strings.)
[HR]
Synopsis
[HR]
ntpq [-46dinpw] [-c command] [host] [...]

Description
[HR]
The ntpq utility program is used to ...

-------------

docs/ntpq.txt includes:
  include::includes/manual.txt[]
It's also included by ntpdsim, ntpkeygen, ntpleapfetch, and sitemap

That's the source of the extra clump of links.  It looks reasonable if I 
delete that line, but I'm not sure that's the appropriate fix.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Sun Nov 22 08:13:49 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sun, 22 Nov 2015 00:13:49 -0800
Subject: What's the relation between GitLab/issues and devel/TODO?
Message-ID: <20151122081349.96665406057@ip-64-139-1-69.sjc.megapath.net>

Where do we keep track of bugs?

There are 2 bugs that should be on our list.

The first is the obscure DNS mixup I ran into a while ago.
It's in ensure_workresp_empty_slot in libntp/work_thread.c
Appending to an array used as a fifo to make more room
doesn't work if the finger is pointing into the middle of the array.

There is similar code when passing names to the worker thread.
I think that one doesn't get triggered because the finger isn't
pointing into the middle when it fills up.

I think the right solution is to replace the array with a linked list.
It needs a lock and such.

A hack good-enough solution would be to make the array bigger.

Chris has been thinking about a major cleanup/simplification
of that whole area.  That seems right to me.  (If he doesn't do it, I will.)

The trigger case is a DNS name that does a lookup using packets
followed by 5 names that use the local /etc/hosts.
It didn't fail every time, but wasn't hard to tickle.

The second bug is a screwup in ntpq/mrulist.
The mrulist command may take a long time to collect all the data.
^C is supposed to stop collecting and print out what it has.

Instead, it sometimes gives an error message and bails.
Ctrl-C will stop MRU retrieval and display partial results.
1305 (0 updates) ^C./glypnod/ntpq/ntpq: select fails: Interrupted system 
call***Server returns unknown error code -1

It doesn't happen all the time, but it isn't hard to tickle, at least in my 
setup.

-- 
These are my opinions.  I hate spam.

From esr at thyrsus.com  Sun Nov 22 13:55:41 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 08:55:41 -0500
Subject: Testing
In-Reply-To: <20151122050244.B7864406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151122044204.GA21002@thyrsus.com>
 <20151122050244.B7864406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151122135541.GB25127@thyrsus.com>

Hal Murray :
> 
> esr at thyrsus.com said:
> > Sadly, I don't know of any good tutorials on this.  It is a swamp full of
> > razor blades for even very experienced C programmers.  If you have a
> > particular exporession you want me to analyze I might be able to say
> > something useful. 
> 
> The old code was:
> 
>         int chars;
>         char req_buf[CTL_MAX_DATA_LEN];
> 
>                 req = req_buf;
>                 req_end = req_buf + sizeof(req_buf);
> #define REQ_ROOM        (req_end - req)
> 
>                         chars = strlen(buf);
>                         if (REQ_ROOM - chars < 1)
>                                 break;
> 
> The new code is:
> 
>         size_t chars;
> 
>                         if (REQ_ROOM <= chars)
>                                 break;
> 
> The old if didn't work after chars changed to size_t.

Right.  That's a good change.  Any time you eliminate a subtraction you
eliminate a potential overflow point. I'm still not happy with this
code, because REQ_ROOM looks like a constant but conceals another
subtraction.

What happened here, I think, is that because of the signed to unsigned
conversion results of the subtraction that would have gone to less than zero
turned into a large unsigned value due to modular wraparound.

Hm.  Maybe I need to *write* a tutorial on this.  The trouble is that while I
probably know enough to do it, my knowledge is tacit and reactive rather
than explicit.

In the mean time, I'm going to follow Mark's advice and experiment with -Wextra.
Maybe that will flush out similar instances.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Sun Nov 22 14:05:04 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 09:05:04 -0500
Subject: What's the relation between GitLab/issues and devel/TODO?
In-Reply-To: <20151122081349.96665406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151122081349.96665406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151122140504.GC25127@thyrsus.com>

Hal Murray : 
> Where do we keep track of bugs?

The bugtracker will usually be the best place, now that we have one.
devel/TODO may still be a better place for internal feature requests
and "it would be nice" notes that don't rise to the level of bugs.

> There are 2 bugs that should be on our list.

Please add these to the bugtracker.  For PR reasons, if you know that
a bug is inherited from Classic, so mark it.  I think that's true
of the first, perhaps not of the second.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Sun Nov 22 14:11:26 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 09:11:26 -0500
Subject: doc quirks
In-Reply-To: <20151122055016.3ED5D406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151122044907.GB21002@thyrsus.com>
 <20151122055016.3ED5D406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151122141126.GD25127@thyrsus.com>

Hal Murray :
> 
> > Dave Mills designed it that way.  It wouldn't be difficult to change, but
> > I'm trying to avoid moving away from his look and feel before we have him
> > recruited/co-opted. 
> 
> Here is a sample ntpq page from NTP Classic, starting after the cartoon:
> 
> More Help
> [HR]
> Synopsis
> ntpq [-46dinp] [-c command] [host] [...]
> Description
> 
> The ntpq utility program is used to ...
> 
> Here is the top of our ntpq.html:
> 
> More Help
> [HR]
> Program Manual Pages
> [HR]
>     ntpd - Network Time Protocol (NTP) daemon
>     ntpdig - Simple Network Time Protocol (SNTP) client
>     ntpq - standard NTP query program
>     ntpfrob - frob the local clock hardware
>     ntpkeygen - generate public and private keys
>     ntpleapfetch(8) fetch and manage leap-offset file
>     ntpdsim(8) - Network Time Protocol (NTP) simulator
>     ntptime - read and set kernel time variables
>     ntptrace - trace a chain of NTP servers back to the primary source
>     Site Map
> (There are links behind most of those text strings.)
> [HR]
> Synopsis
> [HR]
> ntpq [-46dinpw] [-c command] [host] [...]
> 
> Description
> [HR]
> The ntpq utility program is used to ...
> 
> -------------
> 
> docs/ntpq.txt includes:
>   include::includes/manual.txt[]
> It's also included by ntpdsim, ntpkeygen, ntpleapfetch, and sitemap
> 
> That's the source of the extra clump of links.  It looks reasonable if I 
> delete that line, but I'm not sure that's the appropriate fix.

It will be, once we don't have adhering to Dave Mills's present ideas about
correct page design as a major goal of the Web UI. That is most
likely to occur after we make the pilgrimage to Delaware and I get
a chance to have a nice long chat with him about it.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Sun Nov 22 14:27:07 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 09:27:07 -0500
Subject: doc quirks
In-Reply-To: <20151122052727.8A13F406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151122044907.GB21002@thyrsus.com>
 <20151122052727.8A13F406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151122142707.GE25127@thyrsus.com>

Hal Murray :
> 
> [+foo+]
> 
> > Yes, that is supposed to map to const-width font -  or  in HTML
> 
> from docs/includes/ntpq-body.txt
> 
> +-w+, +--wide+::
> 
> from build/docs/ntpq.html
> 
> 
> -w, --wide
> 
> 
> from build/ntpq/ntpq.1
> 
> .RE
> .PP
> \-w, \-\-wide
> .RS 4
> 
> It's possible that something is broken on my asciidoc setup.

On the Web side this was a CSS error which I just fixed.  I don't know what to
do about the man page rendering, though; that looks like a bug in the a2x
stylesheets.  I'll look for a way to report it upstream.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Sun Nov 22 20:25:33 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sun, 22 Nov 2015 12:25:33 -0800
Subject: doc quirks
In-Reply-To: Message from "Eric S. Raymond" 
 of "Sun, 22 Nov 2015 09:11:26 EST." <20151122141126.GD25127@thyrsus.com>
Message-ID: <20151122202533.10E97406057@ip-64-139-1-69.sjc.megapath.net>

esr at thyrsus.com said:
> It will be, once we don't have adhering to Dave Mills's present ideas about
> correct page design as a major goal of the Web UI. That is most likely to
> occur after we make the pilgrimage to Delaware and I get a chance to have a
> nice long chat with him about it. 

We aren't on the same wavelength yet.  Let me try again.

The cartoon at the top is reproduced reasonably closely.  (It is missing the 
last edit.)

I'm trying to point out some cruft that is NOT in the version from NTP 
Classic.

It looks good to me if I remove the includes/manual.txt, but I want to make 
sure that you didn't expect that to do something important.

-- 
These are my opinions.  I hate spam.

From esr at thyrsus.com  Sun Nov 22 21:16:45 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 16:16:45 -0500
Subject: doc quirks
In-Reply-To: <20151122202533.10E97406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151122141126.GD25127@thyrsus.com>
 <20151122202533.10E97406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151122211645.GA22918@thyrsus.com>

Hal Murray :
> 
> esr at thyrsus.com said:
> > It will be, once we don't have adhering to Dave Mills's present ideas about
> > correct page design as a major goal of the Web UI. That is most likely to
> > occur after we make the pilgrimage to Delaware and I get a chance to have a
> > nice long chat with him about it. 
> 
> We aren't on the same wavelength yet.  Let me try again.
> 
> The cartoon at the top is reproduced reasonably closely.  (It is missing the 
> last edit.)
> 
> I'm trying to point out some cruft that is NOT in the version from NTP 
> Classic.
> 
> It looks good to me if I remove the includes/manual.txt, but I want to make 
> sure that you didn't expect that to do something important.

Oh.  In that case, remove it.  That must have been a slip of someone's
finger while Nalette and I were converting the HTML to asciidoc.
-- 
		Eric S. Raymond

From chrisj at ntpsec.org  Mon Nov 23 01:21:37 2015
From: chrisj at ntpsec.org (Chris Johns)
Date: Mon, 23 Nov 2015 12:21:37 +1100
Subject: What's the relation between GitLab/issues and devel/TODO?
In-Reply-To: <20151122081349.96665406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151122081349.96665406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <56526A21.3040305@ntpsec.org>

On 22/11/2015 7:13 PM, Hal Murray wrote:
> 
> Chris has been thinking about a major cleanup/simplification
> of that whole area.  That seems right to me.  (If he doesn't do it, I will.)

I need to get a stretch of time to do this but other issues are
distracting me. I will try look at the code this week.

Chris

From hmurray at megapathdsl.net  Mon Nov 23 02:53:37 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sun, 22 Nov 2015 18:53:37 -0800
Subject: feature 'CRYPTO' does not exist - bind at least one method to it
Message-ID: <20151123025337.2A4EB406057@ip-64-139-1-69.sjc.megapath.net>

That's from the top of ./waf build

I don't know what it's trying to tell me.  It seems to continue building so 
I'm ignoring it.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Mon Nov 23 03:05:17 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 23 Nov 2015 03:05:17 +0000
Subject: feature 'CRYPTO' does not exist - bind at least one method to it
In-Reply-To: <20151123025337.2A4EB406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151123025337.2A4EB406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151123030517.GA29884@darkbeer.org>

On 2015-11-22 18:53 -0800, Hal Murray wrote:
> 
> That's from the top of ./waf build
> 
> I don't know what it's trying to tell me.  It seems to continue building so 
> I'm ignoring it.

Fixed, I meant to put this in use=

Thanks.

Amar.

From esr at snark.thyrsus.com  Mon Nov 23 03:41:22 2015
From: esr at snark.thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 22:41:22 -0500 (EST)
Subject: The new website directory
Message-ID: <20151123034122.1A059C00A6@snark.thyrsus.com>

You wrote this:

>I converted the WWW repo to waf and put it on oldtimer in
>
>  /data/git/www-asciidoc.git
>
>I will hookup buildbot to build and then install to www.ntpsec.org after every
>commit.
>
>Please take a look at the repo to make sure it works for everyone here.  It
>should.  Usage is simple:
>
>  # waf configure --prefix=/path/to/www/
>  # waf install
>
>You don't need to install it to view it just go to the build/ dir and open
>index.html it will work fine.

And then this:

>In order to see the site add:
>
>140.211.9.65   www.ntpsec.org  nwww
>
>To your hosts file.
>
>The source is built and installed after every commit via Buildbot:
>
>  https://buildbot.ntpsec.org/#/builders/13
>
>I will enable it after I'm given the all-clear.  This will fix our SSL issues.

You didn't specify an URL to clone from.  I had to figure out that it
was ssh://esr at oldtimer.ntpsec.org/data/git/www-asciidoc.git

One I cloned it, I found that the README still said

>Edit cycle:
>
>1. Modify the asciidoc.
>
>2. Run 'rebuild' to generate HTML from the masters.
>
>3. Commit everything, then push.
>
>The github.io site ignores the .txt files and makes the HTML visible.
>
>The HTML is normally write-locked to remind you not to modify it.

even though there is no rebuild script there any more.  Your change
may have been well-intentioned, but because you failed to document
it properly you have changed a setup I knew how to use into a setup
I don't know how to use.

I am left with the following questions:

1. If I modify a .txt file and push it, will the corresponding *.html
   file be created and published?

2. If not, what rebuild step must I perform?

3. Why did the site favicon stop being displayed?  What can be done to
   fix that?

4. Why is waf involved at all?

5. What problem was this solving?  Why were you making apparently
   gratuitous changes to the website publishing machinery when what
   0.9.1 most needs from you is for all the unit tests to work?   
-- 
		Eric S. Raymond

The right of self-defense is the first law of nature: in most
governments it has been the study of rulers to confine this right
within the narrowest limits possible.  Wherever standing armies
are kept up, and when the right of the people to keep and bear
arms is, under any color or pretext whatsoever, prohibited,
liberty, if not already annihilated, is on the brink of
destruction." 
	-- Henry St. George Tucker (in Blackstone's Commentaries)

From fallenpegasus at gmail.com  Mon Nov 23 04:09:20 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Mon, 23 Nov 2015 04:09:20 +0000
Subject: The new website directory
In-Reply-To: <20151123034122.1A059C00A6@snark.thyrsus.com>
References: <20151123034122.1A059C00A6@snark.thyrsus.com>
Message-ID: 

Hi Eric, everyone:

The website canonical repo is the one on gitlab.  I asked Amar to use the
gitlab push hooks so that pushes to that repo would rebuild the HTML and
put the results on the web server.

We had to move our web pages off GitHub Pages to our own web server, to
solve the problem of bricking someone's browser against our site if they
manually typed "https://ntpsec.org".

I myself find it useful to have WAF rebuild the HTML, because I don't have
asciidoc installed on every machine I may be using when editing the site.

Plus I was noticing random changes in the site stemming from me having
slightly different versions of asciidoc installed than other team members.
Changing it to being built via the push hooks solved that problem.

Open a bug on the issue tracker about the favicon, and Amar will be able to
fix it.

Thanks!

..m

On Sun, Nov 22, 2015 at 7:41 PM Eric S. Raymond 
wrote:

> You wrote this:
>
> >I converted the WWW repo to waf and put it on oldtimer in
> >
> >  /data/git/www-asciidoc.git
> >
> >I will hookup buildbot to build and then install to www.ntpsec.org after
> every
> >commit.
> >
> >Please take a look at the repo to make sure it works for everyone here.
> It
> >should.  Usage is simple:
> >
> >  # waf configure --prefix=/path/to/www/
> >  # waf install
> >
> >You don't need to install it to view it just go to the build/ dir and open
> >index.html it will work fine.
>
> And then this:
>
> >In order to see the site add:
> >
> >140.211.9.65   www.ntpsec.org  nwww
> >
> >To your hosts file.
> >
> >The source is built and installed after every commit via Buildbot:
> >
> >  https://buildbot.ntpsec.org/#/builders/13
> >
> >I will enable it after I'm given the all-clear.  This will fix our SSL
> issues.
>
> You didn't specify an URL to clone from.  I had to figure out that it
> was ssh://esr at oldtimer.ntpsec.org/data/git/www-asciidoc.git
>
> One I cloned it, I found that the README still said
>
> >Edit cycle:
> >
> >1. Modify the asciidoc.
> >
> >2. Run 'rebuild' to generate HTML from the masters.
> >
> >3. Commit everything, then push.
> >
> >The github.io site ignores the .txt files and makes the HTML visible.
> >
> >The HTML is normally write-locked to remind you not to modify it.
>
> even though there is no rebuild script there any more.  Your change
> may have been well-intentioned, but because you failed to document
> it properly you have changed a setup I knew how to use into a setup
> I don't know how to use.
>
> I am left with the following questions:
>
> 1. If I modify a .txt file and push it, will the corresponding *.html
>    file be created and published?
>
> 2. If not, what rebuild step must I perform?
>
> 3. Why did the site favicon stop being displayed?  What can be done to
>    fix that?
>
> 4. Why is waf involved at all?
>
> 5. What problem was this solving?  Why were you making apparently
>    gratuitous changes to the website publishing machinery when what
>    0.9.1 most needs from you is for all the unit tests to work?
> --
>                 Eric S. Raymond
>
> The right of self-defense is the first law of nature: in most
> governments it has been the study of rulers to confine this right
> within the narrowest limits possible.  Wherever standing armies
> are kept up, and when the right of the people to keep and bear
> arms is, under any color or pretext whatsoever, prohibited,
> liberty, if not already annihilated, is on the brink of
> destruction."
>         -- Henry St. George Tucker (in Blackstone's Commentaries)
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From esr at thyrsus.com  Mon Nov 23 04:58:41 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Sun, 22 Nov 2015 23:58:41 -0500
Subject: The new website directory
In-Reply-To: 
References: <20151123034122.1A059C00A6@snark.thyrsus.com>

Message-ID: <20151123045841.GA32744@thyrsus.com>

Mark Atwood :
> Hi Eric, everyone:
> 
> The website canonical repo is the one on gitlab.  I asked Amar to use the
> gitlab push hooks so that pushes to that repo would rebuild the HTML and
> put the results on the web server.

So, I should ignore the repo at

    ssh://esr at oldtimer.ntpsec.org/data/git/www-asciidoc.git

?

> We had to move our web pages off GitHub Pages to our own web server, to
> solve the problem of bricking someone's browser against our site if they
> manually typed "https://ntpsec.org".
> 
> I myself find it useful to have WAF rebuild the HTML, because I don't have
> asciidoc installed on every machine I may be using when editing the site.

Do I have to run waf when I want to push a change to the site?

> Plus I was noticing random changes in the site stemming from me having
> slightly different versions of asciidoc installed than other team members.
> Changing it to being built via the push hooks solved that problem.
> 
> Open a bug on the issue tracker about the favicon, and Amar will be able to
> fix it.

Done.

The README still needs to describe correct procedure?
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Mon Nov 23 05:32:18 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sun, 22 Nov 2015 21:32:18 -0800
Subject: doc quirks
In-Reply-To: Message from "Eric S. Raymond" 
 of "Sun, 22 Nov 2015 09:27:07 EST." <20151122142707.GE25127@thyrsus.com>
Message-ID: <20151123053218.CB48C406057@ip-64-139-1-69.sjc.megapath.net>

> On the Web side this was a CSS error which I just fixed.

Thanks.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Mon Nov 23 05:42:11 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Sun, 22 Nov 2015 21:42:11 -0800
Subject: doc quirks
In-Reply-To: Message from "Eric S. Raymond" 
 of "Sun, 22 Nov 2015 16:16:45 EST." <20151122211645.GA22918@thyrsus.com>
Message-ID: <20151123054211.26892406057@ip-64-139-1-69.sjc.megapath.net>

[cruft on top of web pages]

> Oh.  In that case, remove it.  That must have been a slip of someone's
> finger while Nalette and I were converting the HTML to asciidoc. 

I figured out what is going on.  You and Nalette did the right thing.

The problem is that I run my browser with javascript disabled.  There are a 
few places where the web pages from NTP Classic use javascript to include 
stuff that is replicated on several pages.  I didn't see that stuff that was 
included that way.

Of course, I didn't catch that until I had pushed the "fix", so I pushed an 
unfix.

I did the unfix by hand.  Is there a way to undo changes in git?  Or get it 
to make a change that undoes a particular commit?

-- 
These are my opinions.  I hate spam.

From esr at thyrsus.com  Mon Nov 23 05:50:11 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 23 Nov 2015 00:50:11 -0500
Subject: doc quirks
In-Reply-To: <20151123054211.26892406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151122211645.GA22918@thyrsus.com>
 <20151123054211.26892406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151123055011.GA1354@thyrsus.com>

Hal Murray :
> I did the unfix by hand.  Is there a way to undo changes in git?  Or get it 
> to make a change that undoes a particular commit?

Yes.  Look up "git revert".
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Wed Nov 25 01:33:20 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Tue, 24 Nov 2015 17:33:20 -0800
Subject: How do I find out which test is failing?
Message-ID: <20151125013320.E87BB406057@ip-64-139-1-69.sjc.megapath.net>

After a git pull a few minutes ago:

Waf: Leaving directory `/home/murray/ntpsec/play/glypnod'
execution summary 
  tests that pass 1/2 
    /home/murray/ntpsec/play/glypnod/tests/test_ntpdig 
  tests that fail 1/2 
    /home/murray/ntpsec/play/glypnod/tests/test_libntp 
'build' finished successfully (2m40.091s)

There is no info in the normal printout.  (Or I'm not looking in the right 
place.)

[343/366] Compiling tests/common/tests_main.c
[344/366] Linking glypnod/tests/test_ntpdig
[345/366] Processing glypnod/tests/test_ntpdig
[346/366] Compiling tests/libntp/calyearstart.c

[355/366] Compiling tests/common/caltime.c
[356/366] Linking glypnod/tests/test_libntp
[357/366] Processing glypnod/tests/test_libntp
[358/366] Compiling ntpwait/ntpwait

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Wed Nov 25 01:35:39 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 01:35:39 +0000
Subject: How do I find out which test is failing?
In-Reply-To: <20151125013320.E87BB406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151125013320.E87BB406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151125013539.GA73900@darkbeer.org>

On 2015-11-24 17:33 -0800, Hal Murray wrote:
> 
> After a git pull a few minutes ago:
> 
> Waf: Leaving directory `/home/murray/ntpsec/play/glypnod'
> execution summary 
>   tests that pass 1/2 
>     /home/murray/ntpsec/play/glypnod/tests/test_ntpdig 
>   tests that fail 1/2 
>     /home/murray/ntpsec/play/glypnod/tests/test_libntp 
> 'build' finished successfully (2m40.091s)

  waf check -v

It's probably the prettydate test I just pushed a fix for that.

Amar.

From hmurray at megapathdsl.net  Wed Nov 25 02:26:00 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Tue, 24 Nov 2015 18:26:00 -0800
Subject: How do I find out which test is failing?
In-Reply-To: Message from Amar Takhar 
 of "Wed, 25 Nov 2015 01:35:39 GMT." <20151125013539.GA73900@darkbeer.org>
Message-ID: <20151125022600.33205406057@ip-64-139-1-69.sjc.megapath.net>

verm at darkbeer.org said:
>   waf check -v

Thanks.

Please try to set things up so the details get saved somewhere so that I see 
what happened after a test fails.  I'd like to be setup that way in case we 
ever get a test that is flaky rather than solid.  You might as well print it 
out if you detect a failure.

> It's probably the prettydate test I just pushed a fix for that.

Thanks.  It's happy now.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Wed Nov 25 02:30:54 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 02:30:54 +0000
Subject: How do I find out which test is failing?
In-Reply-To: <20151125022600.33205406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151125013539.GA73900@darkbeer.org>
 <20151125022600.33205406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151125023054.GA74708@darkbeer.org>

On 2015-11-24 18:26 -0800, Hal Murray wrote:
> 
> verm at darkbeer.org said:
> >   waf check -v
> 
> Thanks.
> 
> Please try to set things up so the details get saved somewhere so that I see 
> what happened after a test fails.  I'd like to be setup that way in case we 
> ever get a test that is flaky rather than solid.  You might as well print it 
> out if you detect a failure.

Can you open a ticket for this please and assign it to me?

Thanks.

Amar.

From fallenpegasus at gmail.com  Wed Nov 25 05:00:58 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 05:00:58 +0000
Subject: checking in
Message-ID: 

Hi everyone.

Sorry for going silent for half a week, I'm helping to build Yet Another
501c6 open source project foundation to wrap around Yet Another big
multi-company open source project.  When it announces foundation formation,
I'll say what it is.

I'll backtrack this evening and tomorrow morning through list emails to
catch up.

Also, this is a good time for everyone to chime in with what things you've
got working this past week, and what you thing you will be hacking on next
week.  ("Next week" being modulo the US Thanksgiving Holiday).

Thanks!

..m
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From esr at thyrsus.com  Wed Nov 25 05:27:19 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 00:27:19 -0500
Subject: checking in
In-Reply-To: 
References: 
Message-ID: <20151125052719.GA22989@thyrsus.com>

Mark Atwood :
> Also, this is a good time for everyone to chime in with what things you've
> got working this past week, and what you thing you will be hacking on next
> week.  ("Next week" being modulo the US Thanksgiving Holiday).

Fixing small bugs opportunistically.

Working on TESTFRAME.  About 75% of the capture side is done, but the last
three intercepts are the hard ones.  There's some strange stuff going on
in the swings-both-ways IPv4/IPv6 networking code that I'm going to need
to grok before I can do the packet-in/packet-out intercepts.
-- 
		Eric S. Raymond

From chrisj at ntpsec.org  Wed Nov 25 07:23:12 2015
From: chrisj at ntpsec.org (Chris Johns)
Date: Wed, 25 Nov 2015 18:23:12 +1100
Subject: checking in
In-Reply-To: 
References: 
Message-ID: <565561E0.5020001@ntpsec.org>

On 25/11/2015 4:00 PM, Mark Atwood wrote:
> Also, this is a good time for everyone to chime in with what things
> you've got working this past week, and what you thing you will be
> hacking on next week.  ("Next week" being modulo the US Thanksgiving
> Holiday).

Working on the threading. I have been attempting to figure out
scheduled_sleep and dnsworker_ctx. It seems to back off DNS after a
signal is received. Any insights would be appreciated. Also been getting
some back ground info on pools from Hal, again related to DNS and so
threading.

Chris

From hmurray at megapathdsl.net  Wed Nov 25 08:54:09 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Wed, 25 Nov 2015 00:54:09 -0800
Subject: Another lesson on compare warnings please ...
Message-ID: <20151125085409.AB79B406057@ip-64-139-1-69.sjc.megapath.net>

In ntpd/refclock_palisade.h
        short           rpt_cnt;        /* TSIP packet length so far */
        char            rpt_buf[BMAX];  /* packet assembly buffer */

>From ntpd/refclock_palisade.c
                else if (up->rpt_cnt > BMAX)

That code is correct, but I would have used sizeof(rpt_buf) rather than BMAX. 
 With a little bit of poking around, you can figure out that rpt_buf is the 
buffer.  With BMAX, you have to go look in another file to see that it's the 
size of the buffer.

But that gets:
../ntpd/refclock_palisade.c:1415:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   else if (up->rpt_cnt > sizeof(up->rpt_buf)) {

Both BMAX and sizeof work if I change the short to u_int.

So what is the type of sizeof, and what is the type of a literal constant?  I expected them to be the same.

What is the best way to do this sort of thing?

Do I get a segfault if I botch something and get an underflow on a unsigned int and then use it as an index with a 64 bit pointer?

-- 
These are my opinions.  I hate spam.

From joel at rtems.org  Wed Nov 25 14:28:38 2015
From: joel at rtems.org (Joel Sherrill)
Date: Wed, 25 Nov 2015 08:28:38 -0600
Subject: Another lesson on compare warnings please ...
In-Reply-To: <20151125085409.AB79B406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151125085409.AB79B406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: 

On Nov 25, 2015 2:54 AM, "Hal Murray"  wrote:
>
>
> In ntpd/refclock_palisade.h
>         short           rpt_cnt;        /* TSIP packet length so far */
>         char            rpt_buf[BMAX];  /* packet assembly buffer */
>
> From ntpd/refclock_palisade.c
>                 else if (up->rpt_cnt > BMAX)
>
> That code is correct, but I would have used sizeof(rpt_buf) rather than
BMAX.
>  With a little bit of poking around, you can figure out that rpt_buf is
the
> buffer.  With BMAX, you have to go look in another file to see that it's
the
> size of the buffer.
>
> But that gets:
> ../ntpd/refclock_palisade.c:1415:24: warning: comparison between signed
and unsigned integer expressions [-Wsign-compare]
>    else if (up->rpt_cnt > sizeof(up->rpt_buf)) {
>
> Both BMAX and sizeof work if I change the short to u_int.
>
> So what is the type of sizeof, and what is the type of a literal
constant?  I expected them to be the same.

Use size_t.

> What is the best way to do this sort of thing?
>
> Do I get a segfault if I botch something and get an underflow on a
unsigned int and then use it as an index with a 64 bit pointer?
>
>
> --
> These are my opinions.  I hate spam.
>
>
>
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From verm at darkbeer.org  Wed Nov 25 14:55:15 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 14:55:15 +0000
Subject: checking in
In-Reply-To: 
References: 
Message-ID: <20151125145515.GA86577@darkbeer.org>

On 2015-11-25 05:00 +0000, Mark Atwood wrote:
> Also, this is a good time for everyone to chime in with what things you've got
> working this past week, and what you thing you will be hacking on next week. ??
> ("Next week" being modulo the US Thanksgiving Holiday).

Working on the new test suite alongside converting the current one.  I'm far 
enough in that I'm getting a clear idea of how it should be re-organised for 
easier usage.

Along with this I've:

  * Started work on automated coverage reporting which will go on the web.
  * Setup docs.ntpsec.org (not active, today hopefully)
    - Builds HTML NTP docs
    - I will be adding the manpages eventually
  * Building gprof reports into waf for the testsuite and regular binaries.

I'm about 50-60% done the testsuite it's hard to tell since many tests are 
obsolete and several are going to have to be moved around / split up.  Hopefully 
it will be done within a week or two.

Amar.

From fallenpegasus at gmail.com  Wed Nov 25 17:06:37 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 17:06:37 +0000
Subject: checking in
In-Reply-To: <20151125145515.GA86577@darkbeer.org>
References: 
 <20151125145515.GA86577@darkbeer.org>
Message-ID: 

thank you Amar.

please focus primarily on the testsuite.

..m

On Wed, Nov 25, 2015, 6:55 AM Amar Takhar  wrote:

> On 2015-11-25 05:00 +0000, Mark Atwood wrote:
> > Also, this is a good time for everyone to chime in with what things
> you've got
> > working this past week, and what you thing you will be hacking on next
> week.
> > ("Next week" being modulo the US Thanksgiving Holiday).
>
> Working on the new test suite alongside converting the current one.  I'm
> far
> enough in that I'm getting a clear idea of how it should be re-organised
> for
> easier usage.
>
> Along with this I've:
>
>   * Started work on automated coverage reporting which will go on the web.
>   * Setup docs.ntpsec.org (not active, today hopefully)
>     - Builds HTML NTP docs
>     - I will be adding the manpages eventually
>   * Building gprof reports into waf for the testsuite and regular binaries.
>
> I'm about 50-60% done the testsuite it's hard to tell since many tests are
> obsolete and several are going to have to be moved around / split up.
> Hopefully
> it will be done within a week or two.
>
>
> Amar.
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From verm at darkbeer.org  Wed Nov 25 17:13:20 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 17:13:20 +0000
Subject: checking in
In-Reply-To: 
References: 
 <20151125145515.GA86577@darkbeer.org>

Message-ID: <20151125171320.GB88317@darkbeer.org>

On 2015-11-25 17:06 +0000, Mark Atwood wrote:
> thank you Amar.
> 
> please focus primarily on the testsuite.

I'm not sure what you're asking.  With exception to the docs -- which is an open 
ticket this is all testsuite work.

Amar.

From fallenpegasus at gmail.com  Wed Nov 25 17:35:27 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 17:35:27 +0000
Subject: checking in
In-Reply-To: <20151125171320.GB88317@darkbeer.org>
References: 
 <20151125145515.GA86577@darkbeer.org>

 <20151125171320.GB88317@darkbeer.org>
Message-ID: 

Ah, I misunderstood.

So, the state we are in now is:
- All the tests run?
- All the test pass?

This is great news!

Regarding "Working on the new test suite alongside converting the current
one."

Why a new test suite?  What are you converting the current one into?

..m

On Wed, Nov 25, 2015 at 9:13 AM Amar Takhar  wrote:

> On 2015-11-25 17:06 +0000, Mark Atwood wrote:
> > thank you Amar.
> >
> > please focus primarily on the testsuite.
>
> I'm not sure what you're asking.  With exception to the docs -- which is
> an open
> ticket this is all testsuite work.
>
>
> Amar.
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From verm at darkbeer.org  Wed Nov 25 17:42:10 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 17:42:10 +0000
Subject: checking in
In-Reply-To: 
References: 
 <20151125145515.GA86577@darkbeer.org>

 <20151125171320.GB88317@darkbeer.org>

Message-ID: <20151125174210.GA89109@darkbeer.org>

On 2015-11-25 17:35 +0000, Mark Atwood wrote:
> Ah, I misunderstood.
> 
> So, the state we are in now is:
> - All the tests run?
> - All the test pass?

When you say "all" yes all the ones converted run and pass -- with exception to a 
few which I will get back to later they've been commented out.

> Regarding "Working on the new test suite alongside converting the current one."
> 
> Why a new test suite??? What are you converting the current one into?

The old test suite used Google Test which is C++.  The development model of the 
project changed to one incompatible with the project.  Right now I am converting 
it all to C using Unity.

The 'new' test suite involves re-arranging it into a more suitable format to 
edit and test.  We need to be setup better for regression testing as well.

It's complicated since the C++ code uses classes and function overloading which 
is going to require a lot of refactoring.  I'm leaving these ones to the end 

>From the old C++ tests it's about 60% complete conversion wise.  I'll hopefully 
have this done 'soon' barring any strangeness which I've already hit..

Amar.

From joel at rtems.org  Wed Nov 25 18:17:53 2015
From: joel at rtems.org (Joel Sherrill)
Date: Wed, 25 Nov 2015 12:17:53 -0600
Subject: checking in
In-Reply-To: 
References: 
Message-ID: 

I have been attempting to build on CentOS/RHEL 6 and 7. I filed tickets
based on experiences on each

CentOS 6 does not have an RPM for libevent2. Thus without special work by
each person building NTPsec, it won't build. AFAIK this distribution is the
latest RHEL approved for use on DoD networks. I could not find a STIG for
RHEL 7 so any organization which requires the DISA STIG to be applied is
stuck at 6.

My CentOS 7 was missing a package and the build didn't disable the
reference clocks that depended on it. Ticket filed.

The CentOS 6 issue raises some questions. Did we intend to support this OS?
Long term, it highlights the problem of assuming a newer external support
library is going to be part of a LTS OS distribution. I don't know if this
was on the master list of supported OSes and checked when libevent2 as a
dependency was discussed. I don't recall having a checklist OSes to ensure
we checked all.

--joel
On Nov 24, 2015 11:01 PM, "Mark Atwood"  wrote:

> Hi everyone.
>
> Sorry for going silent for half a week, I'm helping to build Yet Another
> 501c6 open source project foundation to wrap around Yet Another big
> multi-company open source project.  When it announces foundation formation,
> I'll say what it is.
>
> I'll backtrack this evening and tomorrow morning through list emails to
> catch up.
>
> Also, this is a good time for everyone to chime in with what things you've
> got working this past week, and what you thing you will be hacking on next
> week.  ("Next week" being modulo the US Thanksgiving Holiday).
>
> Thanks!
>
> ..m
>
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From dtpoirot at gmail.com  Wed Nov 25 18:33:27 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Wed, 25 Nov 2015 12:33:27 -0600
Subject: checking in
In-Reply-To: 
References: 

Message-ID: <00d101d127af$c6ff5f80$54fe1e80$@gmail.com>

I am using the following in Ubuntu 14.04 LTS:

sudo apt-get update

sudo apt-get install bison libevent-2.0-5 libevent-dev libcap2 libcap-dev libssl1.0.0 libssl-dev libreadline6 libreadline6-dev pps-tools asciidoc

git clone https://github.com/ntpsec/ntpsec.git

cd ntpsec

/usr/bin/python2.7 /home/user/work/ntpsec/waf clean

/usr/bin/python2.7 /home/user/work/ntpsec/waf configure

/usr/bin/python2.7 /home/user/work/ntpsec/waf build check

From: devel [mailto:devel-bounces at ntpsec.org] On Behalf Of Joel Sherrill
Sent: Wednesday, November 25, 2015 12:18 PM
To: Mark Atwood
Cc: devel at ntpsec.org
Subject: Re: checking in

I have been attempting to build on CentOS/RHEL 6 and 7. I filed tickets based on experiences on each 

CentOS 6 does not have an RPM for libevent2. Thus without special work by each person building NTPsec, it won't build. AFAIK this distribution is the latest RHEL approved for use on DoD networks. I could not find a STIG for RHEL 7 so any organization which requires the DISA STIG to be applied is stuck at 6.

My CentOS 7 was missing a package and the build didn't disable the reference clocks that depended on it. Ticket filed.

The CentOS 6 issue raises some questions. Did we intend to support this OS? Long term, it highlights the problem of assuming a newer external support library is going to be part of a LTS OS distribution. I don't know if this was on the master list of supported OSes and checked when libevent2 as a dependency was discussed. I don't recall having a checklist OSes to ensure we checked all.

--joel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fallenpegasus at gmail.com  Wed Nov 25 19:03:55 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 19:03:55 +0000
Subject: States of tests
In-Reply-To: <20151125174210.GA89109@darkbeer.org>
References: 
 <20151125145515.GA86577@darkbeer.org>

 <20151125171320.GB88317@darkbeer.org>

 <20151125174210.GA89109@darkbeer.org>
Message-ID: 

On Wed, Nov 25, 2015 at 9:42 AM Amar Takhar  wrote:

> On 2015-11-25 17:35 +0000, Mark Atwood wrote:
> > Ah, I misunderstood.
> >
> > So, the state we are in now is:
> > - All the tests run?
> > - All the test pass?
>
> When you say "all" yes all the ones converted run and pass -- with
> exception to a
> few which I will get back to later they've been commented out.
>

Ok.  Why were they commented out?  Did we comment them out, or were they
commented out in NTP Classic?

|> Why a new test suite?  What are you converting the current one into?

>
> The old test suite used Google Test which is C++.  The development model
> of the
> project changed to one incompatible with the project.  Right now I am
> converting
> it all to C using Unity.
>

Ok.  Expand that a bit for me, please.   The development model of the
Google Test project changed to one incompatible with the NTPsec project?
Or the development model of the NTPsec project changed to one incompatible
with the Google Test project?

I am inclined agree to want the tests in standard C99 C instead of C++, but
also it needs to bring a value-add, such as making it easier for non-core
contributors to write new tests and run them.

> The 'new' test suite involves re-arranging it into a more suitable format
> to
> edit and test.  We need to be setup better for regression testing as well.
>
> It's complicated since the C++ code uses classes and function overloading
> which
> is going to require a lot of refactoring.  I'm leaving these ones to the
> end
>
> From the old C++ tests it's about 60% complete conversion wise.  I'll
> hopefully
> have this done 'soon' barring any strangeness which I've already hit..
>
>

Ok.  Do please email to devel at ntpsec.org each time you convert another
test, and what the test was that you converted.

I'm pushing really hard on the tests because getting all the tests running
is a good marker for 0.9.1, and also once we have that working, I'm going
to ask you to go back and start the work to waf-ify the fork point and
forward, so we can get bisect to work.

Thanks!

..m
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fallenpegasus at gmail.com  Wed Nov 25 19:16:32 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 19:16:32 +0000
Subject: regarding RHEL6, libevent2, and DOD STIG
In-Reply-To: 
References: 

Message-ID: 

On Wed, Nov 25, 2015 at 10:17 AM Joel Sherrill  wrote:

> CentOS 6 does not have an RPM for libevent2. Thus without special work by
> each person building NTPsec, it won't build. AFAIK this distribution is the
> latest RHEL approved for use on DoD networks. I could not find a STIG for
> RHEL 7 so any organization which requires the DISA STIG to be applied is
> stuck at 6.
>

What do other packages that have to run on DoD RHEL6 systems do?  Do you
have any contacts in DOD contractors who already have to face/fix this
issue?

If there is no clean way to do this, then there is no clean way to do it,
and we will just have to bake that "special work" into the build
instructions or build scripts.  What does the special work involve?

We don't have to rush to fix this, it doesnt have to be in 0.9.1 or 0.9.2,
but it probably should be in 1.0.0.

I will email some contacts at RH about if it's possible to backport a
package into official RHEL6/CentOS6.

..m
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From verm at darkbeer.org  Wed Nov 25 19:17:24 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 19:17:24 +0000
Subject: States of tests
In-Reply-To: 
References: 
 <20151125145515.GA86577@darkbeer.org>

 <20151125171320.GB88317@darkbeer.org>

 <20151125174210.GA89109@darkbeer.org>

Message-ID: <20151125191724.GA90630@darkbeer.org>

On 2015-11-25 19:03 +0000, Mark Atwood wrote:
> 
> Ok.?? Why were they commented out??? Did we comment them out, or were they
> commented out in NTP Classic?

It's 3 tests, I commented them out because I may have a better understanding of 
why they aren't working after the rest of the conversion is done.  It's a waste 
of time trying to figure it out now.  It is 3 tests of 97.

I will fix them at the end.

> Ok.?? Expand that a bit for me, please. ?? The development model of the Google
> Test project changed to one incompatible with the NTPsec project??? Or the
> development model of the NTPsec project changed to one incompatible with the
> Google Test project?

The first.  they recommend you include the entire project within your project in 
order to use it.  Also, there are some quirks running it with C code that did 
not exist before -- not back in 2010.

> I am inclined agree to want the tests in standard C99 C instead of C++, but
> also it needs to bring a value-add, such as making it easier for non-core
> contributors to write new tests and run them.

This is one of the main reasons why I am doing this I stated that in my first 
email: to make it easier to maintain and do regression tests.

> Ok.?? Do please email to devel at ntpsec.org??each time you convert another test,
> and what the test was that you converted.

If you look at the commit logs you can see that I'm converting each batch one by 
one.  If you subscribe to vc at ntpsec.org you will see the commit logs.  The name 
of the tests and files speak for themselves.  Documentation for these tests 
already exist at:

  https://support.ntp.org/bin/view/Dev/NtpdFunctionMap
  https://support.ntp.org/bin/view/Dev/UnitTestingNotes

I will be creating better documentation for this at the end which will be 
combined with the new test harness.

> I'm pushing really hard on the tests because getting all the tests running is a
> good marker for 0.9.1, and also once we have that working, I'm going to ask you
> to go back and start the work to waf-ify the fork point and forward, so we can
> get bisect to work.

I have explained what we need to do in order to get this to work it is far more 
than just the build system.

We need to coalesce commits and re-play them over the fork point with 
pre-commit building to ensure they work.

The current build system as-is can be used for this as long as we rename all the 
defines and do the file moves right at the start point.

Just doing the build system does not ensure each commit actually builds which is 
the whole point of making bisect useful.

If it's easier think of it this way.  The development done now was done to 
figure out what we need to build on our platforms.  Now that we know this we 
push the build system to the front and put everything on top of it.  With what 
we have now this ensures every commit will actually build which is something 
that was not possible before since we did not know what we needed

Amar.

From hmurray at megapathdsl.net  Wed Nov 25 19:21:20 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Wed, 25 Nov 2015 11:21:20 -0800
Subject: States of tests
Message-ID: <20151125192120.4A25440605C@ip-64-139-1-69.sjc.megapath.net>

fallenpegasus at gmail.com said:
> I am inclined agree to want the tests in standard C99 C instead of C++, but
> also it needs to bring a value-add, such as making it easier for non-core
> contributors to write new tests and run them. 

Core contributors too.  Some of us don't know anything about C++.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Wed Nov 25 20:24:34 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Wed, 25 Nov 2015 12:24:34 -0800
Subject: Tests broken...
Message-ID: <20151125202434.7904040605C@ip-64-139-1-69.sjc.megapath.net>

[175/204] Compiling tests/libntp/a_md5encrypt.c
../tests/libntp/a_md5encrypt.c:30:25: error: initializer element is not constant
 const int totalLength = packetLength + keyIdLength + digestLength;

I can't figure out what's going on.  It looks reasonable to me.

  const int packetLength = 16;
  const int keyIdLength = 4;
  const int digestLength = 16;
  const int totalLength = packetLength + keyIdLength + digestLength;

In case it matters, that's from a Fedora 22 system.
  gcc (GCC) 5.1.1 20150618 (Red Hat 5.1.1-4)

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Wed Nov 25 20:38:53 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 20:38:53 +0000
Subject: Tests broken...
In-Reply-To: <20151125202434.7904040605C@ip-64-139-1-69.sjc.megapath.net>
References: <20151125202434.7904040605C@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151125203853.GA92298@darkbeer.org>

On 2015-11-25 12:24 -0800, Hal Murray wrote:
> [175/204] Compiling tests/libntp/a_md5encrypt.c
> ../tests/libntp/a_md5encrypt.c:30:25: error: initializer element is not constant
>  const int totalLength = packetLength + keyIdLength + digestLength;
> 
> I can't figure out what's going on.  It looks reasonable to me.
> 
>   const int packetLength = 16;
>   const int keyIdLength = 4;
>   const int digestLength = 16;
>   const int totalLength = packetLength + keyIdLength + digestLength;
> 
> In case it matters, that's from a Fedora 22 system.
>   gcc (GCC) 5.1.1 20150618 (Red Hat 5.1.1-4)

It's because packetLength, keyIdLength, digestLength could be redefined using 
#define so it may not be constant.  It wants it to be:

  const int totalLength = 36; or 16 + 4 + 16 this way it cannot be changed.

That's quite pedantic I like it.  I didn't even get a warning under CLang 3.4.1.

I'll put in a temp fix and mark it for looking at later on.

Amar.

From fallenpegasus at gmail.com  Wed Nov 25 20:57:16 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 20:57:16 +0000
Subject: Tests broken...
In-Reply-To: <20151125203853.GA92298@darkbeer.org>
References: <20151125202434.7904040605C@ip-64-139-1-69.sjc.megapath.net>
 <20151125203853.GA92298@darkbeer.org>
Message-ID: 

| It's because packetLength, keyIdLength, digestLength could be redefined
using
#define so it may not be constant.  It wants it to be:

That can't be right.  #define is done by the preprocessor, so what the
compiler sees *is*
  const int totalLength = 16 + 4 + 16;

Hal, I suggest seting the flag in the compiler to dump the output of the
prepropressor, and find the expression, and see if it's a constant
expression or not.

..m

On Wed, Nov 25, 2015 at 12:38 PM Amar Takhar  wrote:

> On 2015-11-25 12:24 -0800, Hal Murray wrote:
> > [175/204] Compiling tests/libntp/a_md5encrypt.c
> > ../tests/libntp/a_md5encrypt.c:30:25: error: initializer element is not
> constant
> >  const int totalLength = packetLength + keyIdLength + digestLength;
> >
> > I can't figure out what's going on.  It looks reasonable to me.
> >
> >   const int packetLength = 16;
> >   const int keyIdLength = 4;
> >   const int digestLength = 16;
> >   const int totalLength = packetLength + keyIdLength + digestLength;
> >
> > In case it matters, that's from a Fedora 22 system.
> >   gcc (GCC) 5.1.1 20150618 (Red Hat 5.1.1-4)
>
> It's because packetLength, keyIdLength, digestLength could be redefined
> using
> #define so it may not be constant.  It wants it to be:
>
>   const int totalLength = 36; or 16 + 4 + 16 this way it cannot be changed.
>
> That's quite pedantic I like it.  I didn't even get a warning under CLang
> 3.4.1.
>
> I'll put in a temp fix and mark it for looking at later on.
>
>
> Amar.
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From verm at darkbeer.org  Wed Nov 25 21:05:28 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Wed, 25 Nov 2015 21:05:28 +0000
Subject: Tests broken...
In-Reply-To: 
References: <20151125202434.7904040605C@ip-64-139-1-69.sjc.megapath.net>
 <20151125203853.GA92298@darkbeer.org>

Message-ID: <20151125210528.GA92751@darkbeer.org>

On 2015-11-25 20:57 +0000, Mark Atwood wrote:
> #define so it may not be constant.?? It wants it to be:
> 
> That can't be right. ??#define is done by the??preprocessor, so what the compiler
> sees *is*??
> ????const int totalLength = 16 + 4 + 16;
> 
> Hal, I suggest seting the flag in the compiler to dump the output of the
> prepropressor, and find the expression, and see if it's a constant expression
> or not.

Making the change I suggested fixed the problem.

The bottom answer here explains why we're getting the error quite nicely:

  http://stackoverflow.com/questions/3025050/error-initializer-element-is-not-constant-when-trying-to-initialize-variable-w

This is not the first time I've run into it but it is the first time it's 
happened by default.

Amar.

From esr at thyrsus.com  Wed Nov 25 21:22:32 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 16:22:32 -0500
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: 
References: 

Message-ID: <20151125212232.GA31609@thyrsus.com>

Joel Sherrill :
> I have been attempting to build on CentOS/RHEL 6 and 7. I filed tickets
> based on experiences on each
> 
> CentOS 6 does not have an RPM for libevent2. Thus without special work by
> each person building NTPsec, it won't build. AFAIK this distribution is the
> latest RHEL approved for use on DoD networks. I could not find a STIG for
> RHEL 7 so any organization which requires the DISA STIG to be applied is
> stuck at 6.

This is a little bad, but not unrecoverable.  Only ntpdig requires libevent2;
ntpd, in particular, builds without it.  If we have to document that as a
limitation I don't think we're going to lose customers over it.

I have changed the ticket to have the 'build' label and asked Amar to change the
build so it fails gracefully in the absence of libevent2, making everything
but ntpdig.

> My CentOS 7 was missing a package and the build didn't disable the
> reference clocks that depended on it. Ticket filed.

This can be fixed with a relatively minor build system tweak.  I have added
that request to the ticket.

> The CentOS 6 issue raises some questions. Did we intend to support this OS?
> Long term, it highlights the problem of assuming a newer external support
> library is going to be part of a LTS OS distribution. I don't know if this
> was on the master list of supported OSes and checked when libevent2 as a
> dependency was discussed. I don't recall having a checklist OSes to ensure
> we checked all.

I guess now is a good time for me to come clean about one of the
assumptions behind my original technical planning.

I never pushed having a checklist of OSes to support for the same
reason I advocated coding to a strict POSIX.1-2001/C99 baseline.
(Joel, this discussion took place well before you were on board.)

Experience with GPSD had persuaded me that the traditional Unix
strategy of keeping a lot of porting code around just in case led to
carrying a lot of historical baggage that had outlasted its time.  I
believed having a large a-priori set of Unix variants to support would
pull in that direction as well, and it was not the direction I thought
we should go.  I believed our overwhelming priority should be reducing
attack surface by discarding code.

See the project motto: "Perfection is achieved, not when there is
nothing more to add, but when there is nothing left to take away."
This choice was not just me being cute - I viewed it as a direct
implication of our security focus.

In accordance, I deliberately went to the opposite extreme - to be as
aggressive and strict in platform requirements as I thought I could
get away with, in order to throw away as much code as possible,
and then patch around the resulting porting issues.

So far this strategy has had exactly two real downsides:

1. We've had to simulate POSIX clock_gettime() for MAC OS X using a native call.

2. The libevent2 dependency blocks ntpdig on CentOS 6 and older NetBSDs as well.

If these are the worst problems we have as a result - and that looks likely
at this point - I'm going to claim a strategic victory.  We are, as far as I
know, good on all *current* LTSes.

Now here's the controversial part:

In truth, I was anticipating something like the libevent problem
(failure to insta-port to older releases) with an attitude almost like
hope.  That is, if we hadn't run into *anything* like it, I'd consider
that I hadn't pushed quite hard enough towards "nothing left to take
away".  While I would have lived with that outcome, I would have
considered it suboptimal.

In effect, I was hoping for a sort of optimal pain level, enough to signal
strong success at the clean-up-and-pare-down without seriously messing
up functionality.  I think we have that.

I'll admit to having been a little quiet and sneaky about that
success criterion; I didn't want the arguments I'd get if I made it
explicit.

Finally, I will observe that if this let-it-break-a-little stance seems
like a dicy or over-bold choice to have made, some reflection on why
Classic NTP stagnated may be in order.  Fear of breaking anything can
be a trap; among other reasons you want a system architect around is to
exercise judgment about when it's time to bust out, judgment that takes
into account the whole-life-cycle perspective. Including considerations
like reducing downstream maintainance costs, attack surface and
defect rates.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Wed Nov 25 21:31:30 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 16:31:30 -0500
Subject: States of tests
In-Reply-To: 
References: 
 <20151125145515.GA86577@darkbeer.org>

 <20151125171320.GB88317@darkbeer.org>

 <20151125174210.GA89109@darkbeer.org>

Message-ID: <20151125213130.GB31609@thyrsus.com>

Mark Atwood :
> I'm pushing really hard on the tests because getting all the tests running
> is a good marker for 0.9.1, and also once we have that working, I'm going
> to ask you to go back and start the work to waf-ify the fork point and
> forward, so we can get bisect to work.

*blink*

This is why you are the right PM, not me - I had not actually thought
that far forward.  Yes, I got that all unit tests is the right
completion marker for 0.9.1, but, despite all our discussion of it,
hadn't quite gotten to the pro-bisection replay as being the right
*next* thing.  Too focused on TESTFRAME.

Duh.  Yes.  Will execute.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Wed Nov 25 21:32:56 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 16:32:56 -0500
Subject: States of tests
In-Reply-To: <20151125192120.4A25440605C@ip-64-139-1-69.sjc.megapath.net>
References: <20151125192120.4A25440605C@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151125213256.GC31609@thyrsus.com>

Hal Murray :
> 
> fallenpegasus at gmail.com said:
> > I am inclined agree to want the tests in standard C99 C instead of C++, but
> > also it needs to bring a value-add, such as making it easier for non-core
> > contributors to write new tests and run them. 
> 
> Core contributors too.  Some of us don't know anything about C++.

And some of us are constantly struggling to know less about it. :-)
-- 
		Eric S. Raymond

From dtpoirot at gmail.com  Wed Nov 25 21:53:08 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Wed, 25 Nov 2015 15:53:08 -0600
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re: checking in))
Message-ID: <005701d127cb$ac4bace0$04e306a0$@gmail.com>

Speaking of requirements and such...

Is there a candidate list of OSes to be targeted? You can put just about
anything on a VM these days!  ;-)

- dan

From esr at thyrsus.com  Wed Nov 25 23:11:57 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 18:11:57 -0500
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re: checking
 in))
In-Reply-To: <005701d127cb$ac4bace0$04e306a0$@gmail.com>
References: <005701d127cb$ac4bace0$04e306a0$@gmail.com>
Message-ID: <20151125231157.GB2456@thyrsus.com>

Daniel Poirot :
> Speaking of requirements and such...
> 
> Is there a candidate list of OSes to be targeted? You can put just about
> anything on a VM these days!  ;-)

We don't have a really elaborate one, partly for reasons I explained in my
recent long reply to Joel.  We've been focused on our main development
platforms, which happen to be Debian-family Linuxes and recent FreeBSD,
and on POSIX/C99 compliance.

You should talk with Amar about this; he presently runs the VM farm.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Wed Nov 25 23:16:10 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Wed, 25 Nov 2015 15:16:10 -0800
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re: checking 
 in))
In-Reply-To: Message from "Daniel Poirot" 
 of "Wed, 25 Nov 2015 15:53:08 CST." <005701d127cb$ac4bace0$04e306a0$@gmail.com>
Message-ID: <20151125231610.DD32840605C@ip-64-139-1-69.sjc.megapath.net>

dtpoirot at gmail.com said:
> Is there a candidate list of OSes to be targeted?

For Linux, OS isn't good enough.  You also have to specify the distro.  I 
think I've seen something similar for one of the *BSDs, but I forget the 
details.

For all OSes/distros, you also need to specify the version.  Most 
OSes/distros support 2 versions, or 2.5 depending on how you count.  There is 
normally something like:
  The current released version
  The previous released version
  The in-progress next version.

There is often a bit of overlap for a short while after a new version is 
released.  For example, on Fedora 3 versions are normally supported for a 
month or so before support for the now old-old version gets dropped.

> You can put just about anything on a VM these days!  ;-) 

I'm not familiar with running VMs.  Is xen on Linux/Fedora reasonably easy to 
setup and learn?  (and reasonably solit)  Can I boot from any handy CD?  Or 
do things work a lot better if the guest kernal knows it will be running on 
VM?

-- 
These are my opinions.  I hate spam.

From fallenpegasus at gmail.com  Wed Nov 25 23:18:09 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 15:18:09 -0800
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: <20151125212232.GA31609@thyrsus.com>
References: 

 <20151125212232.GA31609@thyrsus.com>
Message-ID: 

Ive reached out to contacts of contacts at "Red Hat Consulting for the Public Sector? about how they deal with libevent2?s absence in RHEL6 on DOD systems. ?Unfortunately, its now past 6pm in Raleigh and in DC on the US Thanksgiving Holiday.

Mayhap I get some suggestions back next week.

..m

On November 25, 2015 at 1:23:02 PM, Eric S. Raymond (esr at thyrsus.com) wrote:
> Joel Sherrill :
> > I have been attempting to build on CentOS/RHEL 6 and 7. I filed tickets
> > based on experiences on each
> >
> > CentOS 6 does not have an RPM for libevent2. Thus without special work by
> > each person building NTPsec, it won't build. AFAIK this distribution is the
> > latest RHEL approved for use on DoD networks. I could not find a STIG for
> > RHEL 7 so any organization which requires the DISA STIG to be applied is
> > stuck at 6.
>  
> This is a little bad, but not unrecoverable. Only ntpdig requires libevent2;
> ntpd, in particular, builds without it. If we have to document that as a
> limitation I don't think we're going to lose customers over it.
>  
> I have changed the ticket to have the 'build' label and asked Amar to change the
> build so it fails gracefully in the absence of libevent2, making everything
> but ntpdig.
>  
> > My CentOS 7 was missing a package and the build didn't disable the
> > reference clocks that depended on it. Ticket filed.
>  
> This can be fixed with a relatively minor build system tweak. I have added
> that request to the ticket.
>  
> > The CentOS 6 issue raises some questions. Did we intend to support this OS?
> > Long term, it highlights the problem of assuming a newer external support
> > library is going to be part of a LTS OS distribution. I don't know if this
> > was on the master list of supported OSes and checked when libevent2 as a
> > dependency was discussed. I don't recall having a checklist OSes to ensure
> > we checked all.
>  
> I guess now is a good time for me to come clean about one of the
> assumptions behind my original technical planning.
>  
> I never pushed having a checklist of OSes to support for the same
> reason I advocated coding to a strict POSIX.1-2001/C99 baseline.
> (Joel, this discussion took place well before you were on board.)
>  
> Experience with GPSD had persuaded me that the traditional Unix
> strategy of keeping a lot of porting code around just in case led to
> carrying a lot of historical baggage that had outlasted its time. I
> believed having a large a-priori set of Unix variants to support would
> pull in that direction as well, and it was not the direction I thought
> we should go. I believed our overwhelming priority should be reducing
> attack surface by discarding code.
>  
> See the project motto: "Perfection is achieved, not when there is
> nothing more to add, but when there is nothing left to take away."
> This choice was not just me being cute - I viewed it as a direct
> implication of our security focus.
>  
> In accordance, I deliberately went to the opposite extreme - to be as
> aggressive and strict in platform requirements as I thought I could
> get away with, in order to throw away as much code as possible,
> and then patch around the resulting porting issues.
>  
> So far this strategy has had exactly two real downsides:
>  
> 1. We've had to simulate POSIX clock_gettime() for MAC OS X using a native call.
>  
> 2. The libevent2 dependency blocks ntpdig on CentOS 6 and older NetBSDs as well.
>  
> If these are the worst problems we have as a result - and that looks likely
> at this point - I'm going to claim a strategic victory. We are, as far as I
> know, good on all *current* LTSes.
>  
> Now here's the controversial part:
>  
> In truth, I was anticipating something like the libevent problem
> (failure to insta-port to older releases) with an attitude almost like
> hope. That is, if we hadn't run into *anything* like it, I'd consider
> that I hadn't pushed quite hard enough towards "nothing left to take
> away". While I would have lived with that outcome, I would have
> considered it suboptimal.
>  
> In effect, I was hoping for a sort of optimal pain level, enough to signal
> strong success at the clean-up-and-pare-down without seriously messing
> up functionality. I think we have that.
>  
> I'll admit to having been a little quiet and sneaky about that
> success criterion; I didn't want the arguments I'd get if I made it
> explicit.
>  
> Finally, I will observe that if this let-it-break-a-little stance seems
> like a dicy or over-bold choice to have made, some reflection on why
> Classic NTP stagnated may be in order. Fear of breaking anything can
> be a trap; among other reasons you want a system architect around is to
> exercise judgment about when it's time to bust out, judgment that takes
> into account the whole-life-cycle perspective. Including considerations
> like reducing downstream maintainance costs, attack surface and
> defect rates.
> --
> Eric S. Raymond
>  

From joel at rtems.org  Wed Nov 25 23:33:43 2015
From: joel at rtems.org (Joel Sherrill)
Date: Wed, 25 Nov 2015 17:33:43 -0600
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: 
References: 

 <20151125212232.GA31609@thyrsus.com>

Message-ID: 

FWIW I am OK with disabling what requires libevent2 when it isn't available
as long as the waf configure ends with a very clear message of what had to
be disabled.

And hopefully, we can build a table of platforms supporting everything,
missing X, missing y, etc. and file tickets that are referenced in
configure output.
On Nov 25, 2015 5:18 PM, "Mark Atwood"  wrote:

> Ive reached out to contacts of contacts at "Red Hat Consulting for the
> Public Sector? about how they deal with libevent2?s absence in RHEL6 on DOD
> systems.  Unfortunately, its now past 6pm in Raleigh and in DC on the US
> Thanksgiving Holiday.
>
> Mayhap I get some suggestions back next week.
>
> ..m
>
>
> On November 25, 2015 at 1:23:02 PM, Eric S. Raymond (esr at thyrsus.com)
> wrote:
> > Joel Sherrill :
> > > I have been attempting to build on CentOS/RHEL 6 and 7. I filed tickets
> > > based on experiences on each
> > >
> > > CentOS 6 does not have an RPM for libevent2. Thus without special work
> by
> > > each person building NTPsec, it won't build. AFAIK this distribution
> is the
> > > latest RHEL approved for use on DoD networks. I could not find a STIG
> for
> > > RHEL 7 so any organization which requires the DISA STIG to be applied
> is
> > > stuck at 6.
> >
> > This is a little bad, but not unrecoverable. Only ntpdig requires
> libevent2;
> > ntpd, in particular, builds without it. If we have to document that as a
> > limitation I don't think we're going to lose customers over it.
> >
> > I have changed the ticket to have the 'build' label and asked Amar to
> change the
> > build so it fails gracefully in the absence of libevent2, making
> everything
> > but ntpdig.
> >
> > > My CentOS 7 was missing a package and the build didn't disable the
> > > reference clocks that depended on it. Ticket filed.
> >
> > This can be fixed with a relatively minor build system tweak. I have
> added
> > that request to the ticket.
> >
> > > The CentOS 6 issue raises some questions. Did we intend to support
> this OS?
> > > Long term, it highlights the problem of assuming a newer external
> support
> > > library is going to be part of a LTS OS distribution. I don't know if
> this
> > > was on the master list of supported OSes and checked when libevent2 as
> a
> > > dependency was discussed. I don't recall having a checklist OSes to
> ensure
> > > we checked all.
> >
> > I guess now is a good time for me to come clean about one of the
> > assumptions behind my original technical planning.
> >
> > I never pushed having a checklist of OSes to support for the same
> > reason I advocated coding to a strict POSIX.1-2001/C99 baseline.
> > (Joel, this discussion took place well before you were on board.)
> >
> > Experience with GPSD had persuaded me that the traditional Unix
> > strategy of keeping a lot of porting code around just in case led to
> > carrying a lot of historical baggage that had outlasted its time. I
> > believed having a large a-priori set of Unix variants to support would
> > pull in that direction as well, and it was not the direction I thought
> > we should go. I believed our overwhelming priority should be reducing
> > attack surface by discarding code.
> >
> > See the project motto: "Perfection is achieved, not when there is
> > nothing more to add, but when there is nothing left to take away."
> > This choice was not just me being cute - I viewed it as a direct
> > implication of our security focus.
> >
> > In accordance, I deliberately went to the opposite extreme - to be as
> > aggressive and strict in platform requirements as I thought I could
> > get away with, in order to throw away as much code as possible,
> > and then patch around the resulting porting issues.
> >
> > So far this strategy has had exactly two real downsides:
> >
> > 1. We've had to simulate POSIX clock_gettime() for MAC OS X using a
> native call.
> >
> > 2. The libevent2 dependency blocks ntpdig on CentOS 6 and older NetBSDs
> as well.
> >
> > If these are the worst problems we have as a result - and that looks
> likely
> > at this point - I'm going to claim a strategic victory. We are, as far
> as I
> > know, good on all *current* LTSes.
> >
> > Now here's the controversial part:
> >
> > In truth, I was anticipating something like the libevent problem
> > (failure to insta-port to older releases) with an attitude almost like
> > hope. That is, if we hadn't run into *anything* like it, I'd consider
> > that I hadn't pushed quite hard enough towards "nothing left to take
> > away". While I would have lived with that outcome, I would have
> > considered it suboptimal.
> >
> > In effect, I was hoping for a sort of optimal pain level, enough to
> signal
> > strong success at the clean-up-and-pare-down without seriously messing
> > up functionality. I think we have that.
> >
> > I'll admit to having been a little quiet and sneaky about that
> > success criterion; I didn't want the arguments I'd get if I made it
> > explicit.
> >
> > Finally, I will observe that if this let-it-break-a-little stance seems
> > like a dicy or over-bold choice to have made, some reflection on why
> > Classic NTP stagnated may be in order. Fear of breaking anything can
> > be a trap; among other reasons you want a system architect around is to
> > exercise judgment about when it's time to bust out, judgment that takes
> > into account the whole-life-cycle perspective. Including considerations
> > like reducing downstream maintainance costs, attack surface and
> > defect rates.
> > --
> > Eric S. Raymond
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From hmurray at megapathdsl.net  Thu Nov 26 00:13:52 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Wed, 25 Nov 2015 16:13:52 -0800
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: Message from "Eric S. Raymond" 
 of "Wed, 25 Nov 2015 16:22:32 EST." <20151125212232.GA31609@thyrsus.com>
Message-ID: <20151126001352.AD08340605C@ip-64-139-1-69.sjc.megapath.net>

> This is a little bad, but not unrecoverable.  Only ntpdig requires
> libevent2; ntpd, in particular, builds without it.  If we have to document
> that as a limitation I don't think we're going to lose customers over it. 

I agree with your general approach, but it's more complicated than that.

libevent2 is currently linked with ntpkeygen
I just tried removing it and it still builds.  I'll push the fix (or send an 
update) shortly.

ntpdig is the ntpdate replacement.  It's often used as part of the boot time 
startup tangle.  CentOS 7 does that.  I'm pretty sure we can work out a clean 
alternative, but we haven't done that yet.  Even if they like it, it's not 
obvious to me how long it takes for a conservative organization to merge in 
that sort of change.  They could continue to use the old ntpdate.

-- 
These are my opinions.  I hate spam.

From esr at thyrsus.com  Thu Nov 26 01:34:51 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 20:34:51 -0500
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: 
References: 

 <20151125212232.GA31609@thyrsus.com>

Message-ID: <20151126013451.GA3896@thyrsus.com>

Joel Sherrill :
> FWIW I am OK with disabling what requires libevent2 when it isn't available
> as long as the waf configure ends with a very clear message of what had to
> be disabled.

Yes, that is clearly the right thing.

> And hopefully, we can build a table of platforms supporting everything,
> missing X, missing y, etc. and file tickets that are referenced in
> configure output.

Hal has been agitating for a features vs. test-status matrix for a while.
I agree it would be a good idea, but I'm not well-positioned to do anything
about it myself - I don't do enough cross-platform testing. I think either Hal
or Amar will have to take the lead on this.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Thu Nov 26 02:12:01 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 21:12:01 -0500
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: <20151126001352.AD08340605C@ip-64-139-1-69.sjc.megapath.net>
References:  <20151125212232.GA31609@thyrsus.com>
 <20151126001352.AD08340605C@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151126021201.GB3896@thyrsus.com>

Hal Murray :
> 
> > This is a little bad, but not unrecoverable.  Only ntpdig requires
> > libevent2; ntpd, in particular, builds without it.  If we have to document
> > that as a limitation I don't think we're going to lose customers over it. 
> 
> I agree with your general approach, but it's more complicated than that.
> 
> libevent2 is currently linked with ntpkeygen
> I just tried removing it and it still builds.  I'll push the fix (or send an 
> update) shortly.

Just pulled it - thanks.

> ntpdig is the ntpdate replacement.  It's often used as part of the boot time 
> startup tangle.  CentOS 7 does that.

Are you sure? I did a check on major distributions in the debate around dropping
ntpdate.  I'm pretty sure I checked Fedora and they were launching with ntpd -g
like everybody else.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Thu Nov 26 03:34:47 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Wed, 25 Nov 2015 19:34:47 -0800
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: Message from "Eric S. Raymond" 
 of "Wed, 25 Nov 2015 21:12:01 EST." <20151126021201.GB3896@thyrsus.com>
Message-ID: <20151126033447.6D3C140605C@ip-64-139-1-69.sjc.megapath.net>

esr at thyrsus.com said:
>> ntpdig is the ntpdate replacement.  It's often used as part of the
>> boot time startup tangle.  CentOS 7 does that.

> Are you sure? I did a check on major distributions in the debate around
> dropping ntpdate.  I'm pretty sure I checked Fedora and they were launching
> with ntpd -g like everybody else. 

Anything ntp related on my systems has been hacked to run whatever I'm 
testing and/or some of my systems have been upgraded from many releases ago 
and who knows what got left behind.  So I could easily be confused.

Did you check CentOS as compared to Fedora?

In any case, we need a how-to-start-ntpd document that covers all the issues. 
 The ones I know about are:
  the initial time may be way off
  wait until the time gets set
  start up as fast as possible
  data bases freak out if time goes backwards

-- 
These are my opinions.  I hate spam.

From dtpoirot at gmail.com  Thu Nov 26 03:40:07 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Wed, 25 Nov 2015 21:40:07 -0600
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re: checking
 in))
In-Reply-To: <20151125231610.DD32840605C@ip-64-139-1-69.sjc.megapath.net>
References: Message from "Daniel Poirot"  of "Wed,
 25 Nov 2015 15:53:08 CST."
 <005701d127cb$ac4bace0$04e306a0$@gmail.com>
 <20151125231610.DD32840605C@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <000601d127fc$253335d0$6f99a170$@gmail.com>

I just wasted the afternoon trying to coerce Outlook into bottom-posting.

What are folks using for am MUA? I am a grey-beard but not enough of one to
use emacs for mail...

Speaking of the grey around the muzzle, after 15 years at NASA, I am in my
second decade of selling and supporting commercial software development
tools into the Linux market. 

All of Linux that matters can be divided into two parts - Red Hat-based
(CentOS, Fedora) and Debian-based (Ubuntu). Capture them and the rest
follow.

The *BSD's are strong in the enterprise server and Internet backbone.
OpenBSD, NetBSD and FreeBSD are like the axis of Cartesian-space - common
origin but orthogonal in every way that matters! 

I would expect a favorable reception in the Solaris camp but there is little
else in the vendor UNIX world still around. The giants - Irix, Ultrix, HP-UX
and AIX - are all but gone.

The testing effort is highly important. Once a full test suite is in place,
Linux distros can be knocked out several per day!

I am very comfortable answering questions in the VM area. My desktop has 16
cores and 72 GB RAM. There are usually several VMs running! One can stand up
a minimal system in just a few minutes. I generally use type-2 hypervisors
like VirtualBox or VMWare rather than the bare metal type.

The wild card in the quest for robust, secure time service may very well be
Android! Mac OS seems well represented already and nobody wants to be
bothered by Windows just yet...

Is DistroWatch still the best measure of the market?

- dan

-----Original Message-----
From: Hal Murray [mailto:hmurray at megapathdsl.net] 
Sent: Wednesday, November 25, 2015 5:16 PM
To: Daniel Poirot
Cc: devel at ntpsec.org; hmurray at megapathdsl.net
Subject: Re: Target OS list? (Was: RE: The libevent issue (Was: Re: checking
in))

dtpoirot at gmail.com said:
> Is there a candidate list of OSes to be targeted?

For Linux, OS isn't good enough.  You also have to specify the distro.  I
think I've seen something similar for one of the *BSDs, but I forget the
details.

For all OSes/distros, you also need to specify the version.  Most
OSes/distros support 2 versions, or 2.5 depending on how you count.  There
is normally something like:
  The current released version
  The previous released version
  The in-progress next version.

There is often a bit of overlap for a short while after a new version is
released.  For example, on Fedora 3 versions are normally supported for a
month or so before support for the now old-old version gets dropped.

> You can put just about anything on a VM these days!  ;-)

I'm not familiar with running VMs.  Is xen on Linux/Fedora reasonably easy
to setup and learn?  (and reasonably solit)  Can I boot from any handy CD?
Or do things work a lot better if the guest kernal knows it will be running
on VM?

--
These are my opinions.  I hate spam.

From esr at thyrsus.com  Thu Nov 26 04:26:35 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 23:26:35 -0500
Subject: The libevent issue (Was: Re: checking in)
In-Reply-To: <20151126033447.6D3C140605C@ip-64-139-1-69.sjc.megapath.net>
References:  <20151126021201.GB3896@thyrsus.com>
 <20151126033447.6D3C140605C@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151126042635.GB5243@thyrsus.com>

Hal Murray :
> > Are you sure? I did a check on major distributions in the debate around
> > dropping ntpdate.  I'm pretty sure I checked Fedora and they were launching
> > with ntpd -g like everybody else. 
> 
> Anything ntp related on my systems has been hacked to run whatever I'm 
> testing and/or some of my systems have been upgraded from many releases ago 
> and who knows what got left behind.  So I could easily be confused.
> 
> Did you check CentOS as compared to Fedora?

I don't think I did. Situation still confused.

> In any case, we need a how-to-start-ntpd document that covers all the issues. 
>  The ones I know about are:
>   the initial time may be way off
>   wait until the time gets set
>   start up as fast as possible
>   data bases freak out if time goes backwards

Agreed. docs/quick.txt might be a start.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Thu Nov 26 04:41:54 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Wed, 25 Nov 2015 23:41:54 -0500
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re: checking
 in))
In-Reply-To: <000601d127fc$253335d0$6f99a170$@gmail.com>
References:  <005701d127cb$ac4bace0$04e306a0$@gmail.com>
 <20151125231610.DD32840605C@ip-64-139-1-69.sjc.megapath.net>
 <000601d127fc$253335d0$6f99a170$@gmail.com>
Message-ID: <20151126044154.GC5243@thyrsus.com>

Daniel Poirot :
> I just wasted the afternoon trying to coerce Outlook into bottom-posting.
> 
> What are folks using for am MUA? I am a grey-beard but not enough of one to
> use emacs for mail...

mutt, here.

> Speaking of the grey around the muzzle, after 15 years at NASA, I am in my
> second decade of selling and supporting commercial software development
> tools into the Linux market. 
> 
> All of Linux that matters can be divided into two parts - Red Hat-based
> (CentOS, Fedora) and Debian-based (Ubuntu). Capture them and the rest
> follow.
> 
> The *BSD's are strong in the enterprise server and Internet backbone.
> OpenBSD, NetBSD and FreeBSD are like the axis of Cartesian-space - common
> origin but orthogonal in every way that matters! 

That pretty much matches the assessment we have of how to get early adoption.
Our strategy has been Linux (mostly Debian family) first, FreeBSD second,
others as they fall in our laps, have a Windows port but don't stress out
about getting it quickly.

> I would expect a favorable reception in the Solaris camp but there is little
> else in the vendor UNIX world still around. The giants - Irix, Ultrix, HP-UX
> and AIX - are all but gone.

I had the impression AIX and HP-UX were still ponderable, though declining.

> I am very comfortable answering questions in the VM area. My desktop has 16
> cores and 72 GB RAM. There are usually several VMs running! One can stand up
> a minimal system in just a few minutes. I generally use type-2 hypervisors
> like VirtualBox or VMWare rather than the bare metal type.

Right, you should have a long conversation with Amar.  I'm trying to
stay walled off from those details; I have other things I have to focus on.

> The wild card in the quest for robust, secure time service may very well be
> Android! Mac OS seems well represented already and nobody wants to be
> bothered by Windows just yet...

Interesting.  NTP Classic traditionally has a Windowa port.

> Is DistroWatch still the best measure of the market?

Dunno.  Haven't followed it closely enough in years.
-- 
		Eric S. Raymond

From fallenpegasus at gmail.com  Thu Nov 26 05:28:34 2015
From: fallenpegasus at gmail.com (Mark Atwood)
Date: Wed, 25 Nov 2015 21:28:34 -0800
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re:
 checking in))
In-Reply-To: <20151126044154.GC5243@thyrsus.com>
References:  <005701d127cb$ac4bace0$04e306a0$@gmail.com>
 <20151125231610.DD32840605C@ip-64-139-1-69.sjc.megapath.net>
 <000601d127fc$253335d0$6f99a170$@gmail.com>
 <20151126044154.GC5243@thyrsus.com>
Message-ID: 

On November 25, 2015 at 8:41:58 PM, Eric S. Raymond (esr at thyrsus.com) wrote:
> Daniel Poirot :
> > I would expect a favorable reception in the Solaris camp but there is little
> > else in the vendor UNIX world still around. The giants - Irix, Ultrix, HP-UX
> > and AIX - are all but gone.
>  
> I had the impression AIX and HP-UX were still ponderable, though declining.

HP sells a *lot* of expensive support contracts for HP-UX, for OpenVMS, and for Tandem Nonstop.

I have it as a level B priority to get build lab accounts on those OSes into our buildbot farm, and then lean on my friends at IBM to do the same for AIX and for Z.

The reason this is even thinkable is that they all at least claim to have a POSIX compliant OS API.

..m?

From dtpoirot at gmail.com  Thu Nov 26 11:50:31 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Thu, 26 Nov 2015 05:50:31 -0600
Subject: Target OS list? (Was: RE: The libevent issue (Was: Re: checking
 in))
In-Reply-To: 
References:  <005701d127cb$ac4bace0$04e306a0$@gmail.com>
 <20151125231610.DD32840605C@ip-64-139-1-69.sjc.megapath.net>
 <000601d127fc$253335d0$6f99a170$@gmail.com>
 <20151126044154.GC5243@thyrsus.com>

Message-ID: <002e01d12840$a9313f50$fb93bdf0$@gmail.com>

I am not here to rain on anyone's parade!

The sooner there is a high fidelity test suite and formal acceptance test, the sooner the "level B's" can help themselves!

I have Solaris and OpenVMS in a VM but never found AIX or HP-UX.

- Dan

-----Original Message-----
From: Mark Atwood [mailto:fallenpegasus at gmail.com] 
Sent: Wednesday, November 25, 2015 11:29 PM
To: esr at thyrsus.com; Daniel Poirot
Cc: devel at ntpsec.org
Subject: Re: Target OS list? (Was: RE: The libevent issue (Was: Re: checking in))

On November 25, 2015 at 8:41:58 PM, Eric S. Raymond (esr at thyrsus.com) wrote:
> Daniel Poirot :
> > I would expect a favorable reception in the Solaris camp but there 
> > is little else in the vendor UNIX world still around. The giants - 
> > Irix, Ultrix, HP-UX and AIX - are all but gone.
>  
> I had the impression AIX and HP-UX were still ponderable, though declining.

HP sells a *lot* of expensive support contracts for HP-UX, for OpenVMS, and for Tandem Nonstop.

I have it as a level B priority to get build lab accounts on those OSes into our buildbot farm, and then lean on my friends at IBM to do the same for AIX and for Z.

The reason this is even thinkable is that they all at least claim to have a POSIX compliant OS API.

..m 

From dtpoirot at gmail.com  Thu Nov 26 12:07:30 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Thu, 26 Nov 2015 06:07:30 -0600
Subject: How much for a 1 GHz computer?
Message-ID: <003301d12843$07bc35f0$1734a1d0$@gmail.com>

This broke my brain:
	https://www.raspberrypi.org/blog/raspberry-pi-zero/ 

buildbot for everyone!

From joel at rtems.org  Thu Nov 26 13:01:03 2015
From: joel at rtems.org (Joel Sherrill)
Date: Thu, 26 Nov 2015 07:01:03 -0600
Subject: How much for a 1 GHz computer?
In-Reply-To: <003301d12843$07bc35f0$1734a1d0$@gmail.com>
References: <003301d12843$07bc35f0$1734a1d0$@gmail.com>
Message-ID: 

On Nov 26, 2015 6:07 AM, "Daniel Poirot"  wrote:
>
> This broke my brain:
>         https://www.raspberrypi.org/blog/raspberry-pi-zero/
>
> buildbot for everyone!
>

An RTEMS user built a SPARC cross toolset for RTEMS, built RTEMS, and ran
tests on a CPU simulator on an original Pi. The performance was comparable
to a mid-90s Sun workstation. About a SPARC Station 20. :)

When we moved to a Pentium Pro Linux computer, build times were five times
faster though.

--joel

> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From esr at thyrsus.com  Thu Nov 26 13:37:15 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 26 Nov 2015 08:37:15 -0500
Subject: ntpsec: reference driver 38/hopf 6021
In-Reply-To: <4f92eb835caeaf15c14cf6fcef8d83b1@nocabal.de>
References: <4f92eb835caeaf15c14cf6fcef8d83b1@nocabal.de>
Message-ID: <20151126133715.GA10005@thyrsus.com>

(Copied to the ntpsec devel list.)

Martin Kotzan :
> Hi Eric,
> 
> great to see you are taking ntp into this century!
> 
> 
> I'd like to holler about the planned removal of driver type 38 (Hopf 6021
> serial).
> 
> This driver is still needed by current Hopf clocks connected via serial
> connection, so the "6021" part is probably a misnomer. These clocks are
> used for infrastructure applications on systems not connected to the
> internet. (So my  employer will start using ntpsec in 5 years or so when
> it has made it into an enterprise distribution.)
> 
> Reference:
> http://www.hopf.com/en/ => Download => Technical Descriptions
> http://www.hopf.com/en/manuals_en.html
> 
> e.g. current GPS clock 6844:
> http://www.hopf.com/manuals/english/pdf_6---/e6844(RC)_0601.pdf
> 
> "6.3.3 Data String for NTP (Network Time Protocol)"
> "parameter of transmission:
> - hopf Standard String (6021)"
> 
> 
> Regards,
> Martin

Noted.  You are our very first device-saving holler from outside the
devteam itself.

Should we change the device name to just "Hopf serial"? What models other
than the 6021 does this cover?
-- 
		Eric S. Raymond

From esr at thyrsus.com  Thu Nov 26 14:05:00 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 26 Nov 2015 09:05:00 -0500
Subject: How to I modify the website?
Message-ID: <20151126140500.GA10437@thyrsus.com>

esr at snark:~/software/ntp-rescue/www-asciidoc$ git pull
fatal: '/data/git/www-asciidoc.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

esr at snark:~/software/ntp-rescue$ cat www-asciidoc/.git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = ssh://esr at oldtimer.ntpsec.org/data/git/www-asciidoc.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master

Even pull access to the website repository no longer works.

How am I supposed to modify it now? I need to take Hopf Serial off the
endangered list.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Thu Nov 26 14:21:15 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 26 Nov 2015 09:21:15 -0500
Subject: ntpsec: reference driver 38/hopf 6021
In-Reply-To: 
References: <4f92eb835caeaf15c14cf6fcef8d83b1@nocabal.de>
 <20151126133715.GA10005@thyrsus.com>

Message-ID: <20151126142115.GA10791@thyrsus.com>

Joel Sherrill :
> There was a long list of devices at the manuals URL. I am in favor of
> renaming it. But an email to the manufacturer would clarify what is
> supported by which drivers and what they refer to the family or families
> as. Then we could use their naming convention.
>
> In RTEMS, we tend to put generic in the name or x's in the part number to
> indicate a range. Like gen52xx which should work on any PowerPC MPC52xx
> board given a bit of info. 

Would you be willing to do this research?
-- 
		Eric S. Raymond

From m-ntp at kotzan.net  Thu Nov 26 14:23:25 2015
From: m-ntp at kotzan.net (Martin Kotzan)
Date: Thu, 26 Nov 2015 15:23:25 +0100
Subject: ntpsec: reference driver 38/hopf 6021
Message-ID: 

On 2015-11-26 14:37, Eric S. Raymond wrote:
> (Copied to the ntpsec devel list.)
> 
> Martin Kotzan :
>> I'd like to holler about the planned removal of driver type 38 (Hopf 
>> 6021
>> serial).
>> 
>> This driver is still needed by current Hopf clocks connected via 
>> serial
>> connection, so the "6021" part is probably a misnomer.
> [...]
> 
> Should we change the device name to just "Hopf serial"? What models 
> other
> than the 6021 does this cover?

"Hopf serial" would be more more fitting for NTP, but these clocks can 
output
about a dozen different strings. Since the hopf documentation refers to
the string as  "Standard 6021 (hopf6021, time and date)" I would argue 
the
"6021" should be kept in the description.

All Hopf GPS and DCF77 clocks with a serial interface will emit that 
string:
"All hopf units with a serial interface can provide the serial data 
string for NTP."
http://www.hopf.com/en/article_en.html#ntpseri

Martin

From verm at darkbeer.org  Thu Nov 26 14:24:18 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Thu, 26 Nov 2015 14:24:18 +0000
Subject: How to I modify the website?
In-Reply-To: <20151126140500.GA10437@thyrsus.com>
References: <20151126140500.GA10437@thyrsus.com>
Message-ID: <20151126142418.GA9012@darkbeer.org>

On 2015-11-26 09:05 -0500, Eric S. Raymond wrote:
> esr at snark:~/software/ntp-rescue/www-asciidoc$ git pull
> fatal: '/data/git/www-asciidoc.git' does not appear to be a git repository
> fatal: Could not read from remote repository.

It's www.git on Gitlab.

Amar.

From hmurray at megapathdsl.net  Thu Nov 26 23:04:40 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Thu, 26 Nov 2015 15:04:40 -0800
Subject: How much for a 1 GHz computer
Message-ID: <20151126230440.739E440605C@ip-64-139-1-69.sjc.megapath.net>

Note that the new board doesn't have an Ethernet.

The Raspberry Pi comes in various versions.
  https://en.wikipedia.org/wiki/Raspberry_Pi

The Pi 2 has  a 4 core ARM v7, 900 MHz, 1 GB of ram, micro SD, Ethernet, and 
4 USB slots.  This is the only version I would consider buying today if the 
goal is hacking with ntpsec.  It runs waf configure and build in 2-3 minutes. 
 (That's without asciidoc.)

There is no TOY/RTC clock.

I use it headless so I haven't paid much attention to the graphics.  You 
probably need console access to get started.  I have an HDMI to VGA adapter 
and ??? to USB for the keyboard/mouse.

The main OS is Rasbian, derived from Debian.

I think it also runs NetBSD and FreeBSD, but I haven't tried them yet.

-- 
These are my opinions.  I hate spam.

From chrisj at ntpsec.org  Fri Nov 27 03:51:11 2015
From: chrisj at ntpsec.org (Chris Johns)
Date: Fri, 27 Nov 2015 14:51:11 +1100
Subject: How much for a 1 GHz computer
In-Reply-To: <20151126230440.739E440605C@ip-64-139-1-69.sjc.megapath.net>
References: <20151126230440.739E440605C@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <5657D32F.4030704@ntpsec.org>

On 27/11/2015 10:04 AM, Hal Murray wrote:
> Note that the new board doesn't have an Ethernet.
> 
> The Raspberry Pi comes in various versions.
>   https://en.wikipedia.org/wiki/Raspberry_Pi
> 
> The Pi 2 has  a 4 core ARM v7, 900 MHz, 1 GB of ram, micro SD, Ethernet, and 
> 4 USB slots.  This is the only version I would consider buying today if the 
> goal is hacking with ntpsec.  It runs waf configure and build in 2-3 minutes. 
>  (That's without asciidoc.)

I agree they are great, I have a few. Just a note on the ethernet, it is
via a USB2 connected MAC device and not directly connected to the DDR so
the performance is ok not great, ie they are not for NAS type applications.

> I use it headless so I haven't paid much attention to the graphics.

The graphics is very good using the Broadcom binary blob. Load Kodi on
it to make a media box and you will see the nice HD video performance.

Chris

From hmurray at megapathdsl.net  Mon Nov 30 09:28:09 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 01:28:09 -0800
Subject: libevent2 quirk on NetBSD
Message-ID: <20151130092809.C4CC4406057@ip-64-139-1-69.sjc.megapath.net>

Does anybody understand this one?  Do we need to add some magic to the recipe 
waf is using?

It built without errors, but can't find the library at run time.

-bash-4.3$ bob2/ntpdig/ntpdig
Shared object "libevent-2.0.so.5" not found

It's in /usr/pkg/lib/

ldd says:
bob2/ntpdig/ntpdig:
        -lm.0 => /usr/lib/libm.so.0
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
        -lc.12 => /usr/lib/libc.so.12
        -lpthread.1 => /usr/lib/libpthread.so.1
        -levent-2.0.5 => not found
        -levent_core-2.0.5 => not found
        -levent_pthreads-2.0.5 => not found
        -lrt.1 => /usr/lib/librt.so.1

It works if I add:
-bash-4.3$ LD_LIBRARY_PATH=/usr/pkg/lib/
-bash-4.3$ export LD_LIBRARY_PATH              
-bash-4.3$ bob2/ntpdig/ntpdig
ntpdig: Must supply at least one of -b hostname, -c hostname, or hostname.
-bash-4.3$ 

I assume that buildbot doesn't have a NetBSD box (yet?).  One of the tests 
uses libevent2.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Mon Nov 30 10:38:49 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 02:38:49 -0800
Subject: mdns stuff in intercept_finish is broken
Message-ID: <20151130103849.7E7FC406057@ip-64-139-1-69.sjc.megapath.net>

I think it only gets tested on NetBSD.

The first problem is that it doesn't include the header file.
The second is that mdns is over in ntpd.c

I think the idea of moving that much actual code over there is a bad idea.

If you want to leave everything there, I suggest commenting out the mdns 
stuff and adding a big note.

--------

This looks fishy :
        sig_desc = NULL;
        sig_desc = strsignal(sig);
        if (sig_desc == NULL)
                sig_desc = "";

It's leftover from when strsignal might not exist and didn't get cleaned up 
when you removed that ifdef test.  My man page says some systems (not Linux) 
might return NULL on an invalid signal.  I assume anything passed to that 
routine will be valid.

-------

There are 2 ways into that code.  One is via a kill command, probably from 
something like "service ntpd stop".  The other is from a SIGBUS.  The SIGBUS 
case should probably bypass the cleanup and exit non-zero.

I think the idea of doing more than a simple log message and exit is to free 
up all the mallocs so the tool that monitors malloc usage would see a close 
to clean result at exit and everything left would be lost memory.  I don't 
see the code I expect.  There is a lot of mallocing in the config file parser 
and big chunks of memory for the mrulist.  The config stuff is using an 
atexit-hook.  That's probably a bad idea if we are going to catch SIGBUS and 
expect to do anything.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Mon Nov 30 10:49:55 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 02:49:55 -0800
Subject: An interesting warning
Message-ID: <20151130104955.3A908406057@ip-64-139-1-69.sjc.megapath.net>

It's from Debian wheezy (and decendents like Rasbian)
gcc (Debian 4.7.2-5) 4.7.2

[109/174] Compiling ntpd/ntp_io.c
../ntpd/ntp_io.c: In function ???process_routing_msgs???:
../ntpd/ntp_io.c:4629:7: warning: comparison between signed and unsigned 
integer expressions [-Wsign-compare]

That line of code is:
             NLMSG_OK(nh, cnt);
It's the middle term in a for loop.

nh is a pointer, cnt is an int.

NLMSG_OK comes from /usr/include/linux/netlink.h
#define NLMSG_OK(nlh,len) ((len) >= (int)sizeof(struct nlmsghdr) && \
                           (nlh)->nlmsg_len >= sizeof(struct nlmsghdr) && \
                           (nlh)->nlmsg_len <= (len))
nlmsg_len comes from a struct nlmsghdr:
        __u32           nlmsg_len;      /* Length of message including header 
*/

That part of the header didn't change from wheezy to jessie.

The 3rd term is comparing an unsigned with an int.  So the real question is 
why the compiler on other systems don't complain.

Is there a clean fix for this, or do we just document it as a glitch in this 
environment?

-- 
These are my opinions.  I hate spam.

From esr at thyrsus.com  Mon Nov 30 13:28:07 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 08:28:07 -0500
Subject: An interesting warning
In-Reply-To: <20151130104955.3A908406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151130104955.3A908406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130132807.GA1545@thyrsus.com>

Hal Murray :
> It's from Debian wheezy (and decendents like Rasbian)
> gcc (Debian 4.7.2-5) 4.7.2
> 
> [109/174] Compiling ntpd/ntp_io.c
> ../ntpd/ntp_io.c: In function ???process_routing_msgs???:
> ../ntpd/ntp_io.c:4629:7: warning: comparison between signed and unsigned 
> integer expressions [-Wsign-compare]
> 
> That line of code is:
>              NLMSG_OK(nh, cnt);
> It's the middle term in a for loop.
> 
> nh is a pointer, cnt is an int.
> 
> NLMSG_OK comes from /usr/include/linux/netlink.h
> #define NLMSG_OK(nlh,len) ((len) >= (int)sizeof(struct nlmsghdr) && \
>                            (nlh)->nlmsg_len >= sizeof(struct nlmsghdr) && \
>                            (nlh)->nlmsg_len <= (len))
> nlmsg_len comes from a struct nlmsghdr:
>         __u32           nlmsg_len;      /* Length of message including header 
> */
> 
> That part of the header didn't change from wheezy to jessie.
> 
> The 3rd term is comparing an unsigned with an int.  So the real question is 
> why the compiler on other systems don't complain.
> 
> Is there a clean fix for this, or do we just document it as a glitch in this 
> environment?

Try replacing the call with NLMSG_OK(nh, (uint32_t)cnt). That might do it.

But please test in other environments before you commit.  It seems to me
the most likely reason that this error isn't everywhere is that the
type declaration of nlmsg_len is not stable across NLS versions.  If so,
trying to fix this could induce more warnings than it solves.
-- 
		Eric S. Raymond

From esr at thyrsus.com  Mon Nov 30 13:39:53 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 08:39:53 -0500
Subject: mdns stuff in intercept_finish is broken
In-Reply-To: <20151130103849.7E7FC406057@ip-64-139-1-69.sjc.megapath.net>
References: <20151130103849.7E7FC406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130133953.GB1545@thyrsus.com>

Hal Murray :
> 
> I think it only gets tested on NetBSD.
> 
> The first problem is that it doesn't include the header file.
> The second is that mdns is over in ntpd.c
> 
> I think the idea of moving that much actual code over there is a bad idea.

I reverted that change last night.  You're right, it was a bad idea.

> This looks fishy :
>         sig_desc = NULL;
>         sig_desc = strsignal(sig);
>         if (sig_desc == NULL)
>                 sig_desc = "";
> 
> It's leftover from when strsignal might not exist and didn't get cleaned up 
> when you removed that ifdef test.  My man page says some systems (not Linux) 
> might return NULL on an invalid signal.  I assume anything passed to that 
> routine will be valid.

Yes, I guess the "sig_desc = NULL;" can go

> There are 2 ways into that code.  One is via a kill command, probably from 
> something like "service ntpd stop".  The other is from a SIGBUS.  The SIGBUS 
> case should probably bypass the cleanup and exit non-zero.
> 
> I think the idea of doing more than a simple log message and exit is to free 
> up all the mallocs so the tool that monitors malloc usage would see a close 
> to clean result at exit and everything left would be lost memory.  I don't 
> see the code I expect.  There is a lot of mallocing in the config file parser 
> and big chunks of memory for the mrulist.  The config stuff is using an 
> atexit-hook.  That's probably a bad idea if we are going to catch SIGBUS and 
> expect to do anything.

You're almost certainly right about all this.  Feel free to clean up
as you like; I can't get very excited about it, as I judge the
objective (free up all mallocs before exit) to be unachievable in
practice given NTP's level of allocation complexity and multiple exit
paths.
-- 
		Eric S. Raymond

From joel at rtems.org  Mon Nov 30 14:02:02 2015
From: joel at rtems.org (Joel Sherrill)
Date: Mon, 30 Nov 2015 08:02:02 -0600
Subject: An interesting warning
In-Reply-To: <20151130132807.GA1545@thyrsus.com>
References: <20151130104955.3A908406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130132807.GA1545@thyrsus.com>
Message-ID: 

On Nov 30, 2015 8:28 AM, "Eric S. Raymond"  wrote:
>
> Hal Murray :
> > It's from Debian wheezy (and decendents like Rasbian)
> > gcc (Debian 4.7.2-5) 4.7.2
> >
> > [109/174] Compiling ntpd/ntp_io.c
> > ../ntpd/ntp_io.c: In function ???process_routing_msgs???:
> > ../ntpd/ntp_io.c:4629:7: warning: comparison between signed and unsigned
> > integer expressions [-Wsign-compare]
> >
> > That line of code is:
> >              NLMSG_OK(nh, cnt);
> > It's the middle term in a for loop.
> >
> > nh is a pointer, cnt is an int.
> >
> > NLMSG_OK comes from /usr/include/linux/netlink.h
> > #define NLMSG_OK(nlh,len) ((len) >= (int)sizeof(struct nlmsghdr) && \
> >                            (nlh)->nlmsg_len >= sizeof(struct nlmsghdr)
&& \
> >                            (nlh)->nlmsg_len <= (len))
> > nlmsg_len comes from a struct nlmsghdr:
> >         __u32           nlmsg_len;      /* Length of message including
header
> > */
> >
> > That part of the header didn't change from wheezy to jessie.
> >
> > The 3rd term is comparing an unsigned with an int.  So the real
question is
> > why the compiler on other systems don't complain.
> >
> > Is there a clean fix for this, or do we just document it as a glitch in
this
> > environment?
>
> Try replacing the call with NLMSG_OK(nh, (uint32_t)cnt). That might do it.

Sizes should be size_t.  That's the crux of the problem although it likely
propagates.

Also.. Is there a style rule for macro argument names? I thought it was
considered bad form to use arguments without leading _. The name len seems
common and could lead to unexpected side-effects.

> But please test in other environments before you commit.  It seems to me
> the most likely reason that this error isn't everywhere is that the
> type declaration of nlmsg_len is not stable across NLS versions.  If so,
> trying to fix this could induce more warnings than it solves.

What other types is it?

> --
>                 Eric S. Raymond
> _______________________________________________
> devel mailing list
> devel at ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From esr at thyrsus.com  Mon Nov 30 14:50:30 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 09:50:30 -0500
Subject: An interesting warning
In-Reply-To: 
References: <20151130104955.3A908406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130132807.GA1545@thyrsus.com>

Message-ID: <20151130145030.GA2517@thyrsus.com>

Joel Sherrill :
> > But please test in other environments before you commit.  It seems to me
> > the most likely reason that this error isn't everywhere is that the
> > type declaration of nlmsg_len is not stable across NLS versions.  If so,
> > trying to fix this could induce more warnings than it solves.
> 
> What other types is it?

I don't know.  But I think size_t seems rather likely.
-- 
		Eric S. Raymond

From verm at darkbeer.org  Mon Nov 30 16:56:04 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 16:56:04 +0000
Subject: PARSE clocks.
Message-ID: <20151130165604.GA24378@darkbeer.org>

I have a patch that allows for defining individual PARSE clocks.  There's no 
sense in compiling all 12 in when a user will in most cases only want one of 
them.

The issue is we define how to specify clocks by their ID.. do we give the 
clocks new IDs and do it that way?

Here is the list of clocks in libparse:

CLOCK_COMPUTIME clk_computime.c Supports Diem's Computime Radio Clock
CLOCK_DCF7000   clk_dcf7000.c   ELV DCF7000 module
CLOCK_HOPF6021  clk_hopf6021.c  Radiocode Clocks HOPF Funkuhr 6021 mit
CLOCK_MEINBERG  clk_meinberg.c  Meinberg clock support
CLOCK_RAWDCF    clk_rawdcf.c    Raw DCF77 pulse clock support
CLOCK_RCC8000   clk_rcc8000.c   Radiocode Clocks Ltd RCC 8000 Intelligent 
                                Off-Air Master Clock support
CLOCK_SCHMID    clk_schmid.c    Schmid clock support
CLOCK_SEL240X   clk_sel240x.c   Schweitzer Engineering Laboratories, Inc.
CLOCK_TRIMTAIP  clk_trimtaip.c  Trimble SV6 clock support
CLOCK_TRIMTSIP  clk_trimtsip.c  Trimble TSIP support
CLOCK_VARITEXT  clk_varitext.c  Supports Varitext's Radio Clock
CLOCK_WHARTON_400A  clk_wharton.c   Support for WHARTON 400A Series clock + 
                                    404.2 serial interface.

We can enumerate this list starting at 47 if that works.

In refclock_conf.c we can mark it as &refclock_none and say it comes from 
libparse in the comment.

Thoughts?

Amar.

From hmurray at megapathdsl.net  Mon Nov 30 17:58:41 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 09:58:41 -0800
Subject: PARSE clocks.
In-Reply-To: Message from Amar Takhar 
 of "Mon, 30 Nov 2015 16:56:04 GMT." <20151130165604.GA24378@darkbeer.org>
Message-ID: <20151130175841.CBBBE406057@ip-64-139-1-69.sjc.megapath.net>

verm at darkbeer.org said:
> We can enumerate this list starting at 47 if that works.

That name space is shared with NTP Classic.  We can't just do arbitrary 
things like that.

The Parse clocks already have their own numbering scheme.  I think it's only 
4 bits, the top nibble of the bottom byte of the IP Address.  It's in the 
html page.

I don't see any simple way to merge that into the current configure scheme.  
Probably best to clone the code that reads the numbers, something like 
--parseclock=, and then have it automatically enable the parseclock driver 
and turn on REFCLOCK.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Mon Nov 30 18:03:04 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 18:03:04 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130175841.CBBBE406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130165604.GA24378@darkbeer.org>
 <20151130175841.CBBBE406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130180304.GA25633@darkbeer.org>

On 2015-11-30 09:58 -0800, Hal Murray wrote:
> verm at darkbeer.org said:
> > We can enumerate this list starting at 47 if that works.
> 
> That name space is shared with NTP Classic.  We can't just do arbitrary 
> things like that.

OK, I haven't looked too closely at this before.

> The Parse clocks already have their own numbering scheme.  I think it's only 
> 4 bits, the top nibble of the bottom byte of the IP Address.  It's in the 
> html page.

OK.

> I don't see any simple way to merge that into the current configure scheme.  
> Probably best to clone the code that reads the numbers, something like 
> --parseclock=, and then have it automatically enable the parseclock driver 
> and turn on REFCLOCK.

Well, I can still make it work under the same --refclock= command instead of 
1-47 for the builtins I can do p1-12 for parse clocks.  The numbers would only 
have meaning in the build system.

This way I can have them show up in --list and used in --refclock.

Does that sound reasonable?

Amar.

From hmurray at megapathdsl.net  Mon Nov 30 18:33:40 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 10:33:40 -0800
Subject: PARSE clocks.
In-Reply-To: Message from Amar Takhar 
 of "Mon, 30 Nov 2015 18:03:04 GMT." <20151130180304.GA25633@darkbeer.org>
Message-ID: <20151130183340.6CAC8406057@ip-64-139-1-69.sjc.megapath.net>

verm at darkbeer.org said:
> Well, I can still make it work under the same --refclock= command instead of
>  1-47 for the builtins I can do p1-12 for parse clocks.  The numbers would
> only  have meaning in the build system.

I assume that means something like --refclock=8p3

I just looked.  The subclock uses mode on the server line in ntp.conf, so 
maybe m rather than p.

> This way I can have them show up in --list and used in --refclock.
> Does that sound reasonable? 

Sounds reasonable to me, but I'm put it as pretty low priority.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Mon Nov 30 18:36:36 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 18:36:36 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130183340.6CAC8406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130180304.GA25633@darkbeer.org>
 <20151130183340.6CAC8406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130183636.GA26945@darkbeer.org>

On 2015-11-30 10:33 -0800, Hal Murray wrote:
> I assume that means something like --refclock=8p3

Yes.

> I just looked.  The subclock uses mode on the server line in ntp.conf, so 
> maybe m rather than p.

OK 'm' works.

> Sounds reasonable to me, but I'm put it as pretty low priority.

I need to separate them individually for testing.  Having them all in at once 
will confuse things.

Amar.

From hmurray at megapathdsl.net  Mon Nov 30 18:52:22 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 10:52:22 -0800
Subject: PARSE clocks.
In-Reply-To: Message from Amar Takhar 
 of "Mon, 30 Nov 2015 18:36:36 GMT." <20151130183636.GA26945@darkbeer.org>
Message-ID: <20151130185222.21D68406057@ip-64-139-1-69.sjc.megapath.net>

verm at darkbeer.org said:
> I need to separate them individually for testing.  Having them all in at
> once  will confuse things. 

Could you please say more.  What are you trying to test?

I haven't looked at the code.  Is there any interaction between sub-drivers, 
or library modules that are only needed by some drivers?  Even if there is, 
is it interesting, or important?

There is already a blizzard of possible options.  Adding more just 
complicates things.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Mon Nov 30 19:01:54 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 19:01:54 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130185222.21D68406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130183636.GA26945@darkbeer.org>
 <20151130185222.21D68406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130190154.GA27366@darkbeer.org>

On 2015-11-30 10:52 -0800, Hal Murray wrote:
> 
> verm at darkbeer.org said:
> > I need to separate them individually for testing.  Having them all in at
> > once  will confuse things. 
> 
> Could you please say more.  What are you trying to test?

I'm not actually trying to test the code but do precise coverage reports so I 
can give meaningful displays of how much each area of the code is used and by 
what.

> I haven't looked at the code.  Is there any interaction between sub-drivers, 
> or library modules that are only needed by some drivers?  Even if there is, 
> is it interesting, or important?

I don't know yet but the reports I'm working on should show this.

> There is already a blizzard of possible options.  Adding more just 
> complicates things.

I'm talking about expanding a list that's already there -- I don't think that's 
anymore complicated than what we have.

We've gone to great lengths to only include what's required to operate having 
all the drives placed in at once goes against what we already have.  It won't be 
any different from what's there instead of using '8' you will use '8m[1-12]'.

Amar.

From esr at thyrsus.com  Mon Nov 30 19:13:57 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 14:13:57 -0500
Subject: PARSE clocks.
In-Reply-To: <20151130165604.GA24378@darkbeer.org>
References: <20151130165604.GA24378@darkbeer.org>
Message-ID: <20151130191357.GA4418@thyrsus.com>

Amar Takhar :
> I have a patch that allows for defining individual PARSE clocks.  There's no 
> sense in compiling all 12 in when a user will in most cases only want one of 
> them.

Cute, but I'm not sure the space savings are worth the additional
complexity cost in the build system. The whole point of the parse
driver architecture is to share code among the subtypes; thus, the
memory overhead of any N-1 is pretty small.

I just ran the numbers with size(1) and in the best case (building
with the single smallest driver) you condition out about 28K of text
space - that's 3% of the 787K build size with all refclocks, 7% of
the build size with none.

This is going to be potentially useful only on very constrained SBCs.
It's highly doubtful such systems will ever talk to a parse clock;
every Stratum 1 SBC design I've ever seen uses a GPS.

How much code does this add to the build system?
-- 
		Eric S. Raymond

From verm at darkbeer.org  Mon Nov 30 19:16:52 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 19:16:52 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130191357.GA4418@thyrsus.com>
References: <20151130165604.GA24378@darkbeer.org>
 <20151130191357.GA4418@thyrsus.com>
Message-ID: <20151130191652.GA27682@darkbeer.org>

On 2015-11-30 14:13 -0500, Eric S. Raymond wrote:
> Cute, but I'm not sure the space savings are worth the additional
> complexity cost in the build system. The whole point of the parse
> driver architecture is to share code among the subtypes; thus, the
> memory overhead of any N-1 is pretty small.

I've already added the code to do this for PPS clocks.

> I just ran the numbers with size(1) and in the best case (building
> with the single smallest driver) you condition out about 28K of text
> space - that's 3% of the 787K build size with all refclocks, 7% of
> the build size with none.

I'm trying to remove the additional symbols so coverage reports are more useful 
on an individual basis.

> This is going to be potentially useful only on very constrained SBCs.
> It's highly doubtful such systems will ever talk to a parse clock;
> every Stratum 1 SBC design I've ever seen uses a GPS.
> 
> How much code does this add to the build system?

Very little as it uses existing interfaces.

Amar.

From esr at thyrsus.com  Mon Nov 30 19:21:41 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 14:21:41 -0500
Subject: PARSE clocks.
In-Reply-To: <20151130190154.GA27366@darkbeer.org>
References:  <20151130183636.GA26945@darkbeer.org>
 <20151130185222.21D68406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130190154.GA27366@darkbeer.org>
Message-ID: <20151130192141.GB4418@thyrsus.com>

Amar Takhar :
> > I haven't looked at the code.  Is there any interaction between sub-drivers, 
> > or library modules that are only needed by some drivers?  Even if there is, 
> > is it interesting, or important?
> 
> I don't know yet but the reports I'm working on should show this.

I do know; I've read the code and understand the architecture.  The
individual clk_*.c subdriver mains do not interact with each other;
they're essentially method tables that plug into generic parse-driver
logic.  Some of them share auxiliary code for data format conversions.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Mon Nov 30 19:32:39 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 11:32:39 -0800
Subject: PARSE clocks.
In-Reply-To: Message from Amar Takhar 
 of "Mon, 30 Nov 2015 19:01:54 GMT." <20151130190154.GA27366@darkbeer.org>
Message-ID: <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>

verm at darkbeer.org said:
> I'm not actually trying to test the code but do precise coverage reports so
> I  can give meaningful displays of how much each area of the code is used
> and by  what. 

That's interesting, but wouldn't be very high on my list.

Does buildbot have a NetBSD system?  How about 32 bit vs 64?

How about a list of what OS/distros are in buildbot and what configurations 
get tested?

Can you run our code on the buildbot systems?  That would be a giant step 
forward.  I don't know much about VM.  There are plenty of ways to screwup 
timekeeping, but it works well on some setups.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Mon Nov 30 19:36:27 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 19:36:27 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130193627.GA28036@darkbeer.org>

On 2015-11-30 11:32 -0800, Hal Murray wrote:
> 
> verm at darkbeer.org said:
> > I'm not actually trying to test the code but do precise coverage reports so
> > I  can give meaningful displays of how much each area of the code is used
> > and by  what. 
> 
> That's interesting, but wouldn't be very high on my list.

OK.

> Does buildbot have a NetBSD system?  How about 32 bit vs 64?

I'm waiting for more access to setup additional VMs for this purpose.

> How about a list of what OS/distros are in buildbot and what configurations 
> get tested?

This is already on the Buildbot website.

> Can you run our code on the buildbot systems?  That would be a giant step 
> forward.  I don't know much about VM.  There are plenty of ways to screwup 
> timekeeping, but it works well on some setups.

Yes I am working on that -- operational testing to actually launch each utility 
and ensure they do something (to start).  Eventually it will get into more 
precise tests.  I'm going to do it on a machine or two I have here.

Amar.

From dtpoirot at gmail.com  Mon Nov 30 20:21:48 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Mon, 30 Nov 2015 14:21:48 -0600
Subject: PARSE clocks.
In-Reply-To: <20151130193627.GA28036@darkbeer.org>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130193627.GA28036@darkbeer.org>
Message-ID: <00ab01d12bac$bdaee4e0$390caea0$@gmail.com>

Where does buildbot run?

Would there be any value in me running on 32 and 64, [Open|Free|Net]BSD
ahead of you?

- dan

-----Original Message-----
From: devel [mailto:devel-bounces at ntpsec.org] On Behalf Of Amar Takhar
Sent: Monday, November 30, 2015 1:36 PM
To: devel at ntpsec.org
Subject: Re: PARSE clocks.

On 2015-11-30 11:32 -0800, Hal Murray wrote:
> 
> verm at darkbeer.org said:
> > I'm not actually trying to test the code but do precise coverage 
> > reports so I  can give meaningful displays of how much each area of 
> > the code is used and by  what.
> 
> That's interesting, but wouldn't be very high on my list.

OK.

> Does buildbot have a NetBSD system?  How about 32 bit vs 64?

I'm waiting for more access to setup additional VMs for this purpose.

> How about a list of what OS/distros are in buildbot and what 
> configurations get tested?

This is already on the Buildbot website.

> Can you run our code on the buildbot systems?  That would be a giant 
> step forward.  I don't know much about VM.  There are plenty of ways 
> to screwup timekeeping, but it works well on some setups.

Yes I am working on that -- operational testing to actually launch each
utility and ensure they do something (to start).  Eventually it will get
into more precise tests.  I'm going to do it on a machine or two I have
here.

Amar.
_______________________________________________
devel mailing list
devel at ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

From verm at darkbeer.org  Mon Nov 30 20:26:02 2015
From: verm at darkbeer.org ('Amar Takhar')
Date: Mon, 30 Nov 2015 20:26:02 +0000
Subject: PARSE clocks.
In-Reply-To: <00ab01d12bac$bdaee4e0$390caea0$@gmail.com>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130193627.GA28036@darkbeer.org>
 <00ab01d12bac$bdaee4e0$390caea0$@gmail.com>
Message-ID: <20151130202602.GA28788@darkbeer.org>

On 2015-11-30 14:21 -0600, Daniel Poirot wrote:
> Where does buildbot run?

At the moment on the HP Cloud.

> Would there be any value in me running on 32 and 64, [Open|Free|Net]BSD
> ahead of you?

It's not worth the effort.  Mark is working on getting us access to another 
cloud system once that's done I have everything setup to do it.  We need to be 
off by January so it shouldn't be too long.

Thank you though!

Amar.

From esr at thyrsus.com  Mon Nov 30 20:26:45 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 15:26:45 -0500
Subject: PARSE clocks.
In-Reply-To: <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130202645.GA7787@thyrsus.com>

Hal Murray :
> How about a list of what OS/distros are in buildbot and what configurations 
> get tested?

Amar, this should be your next priority after getting the unit tests done.
It's the other thing we really want for an 0.9.1 release.  Are the unit
tests done yet?

What's the URL for the buildbot status page?  I want tp put it on the
contacts page.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Mon Nov 30 20:27:50 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 12:27:50 -0800
Subject: PARSE clocks.
In-Reply-To: Message from Amar Takhar 
 of "Mon, 30 Nov 2015 19:36:27 GMT." <20151130193627.GA28036@darkbeer.org>
Message-ID: <20151130202750.AB327406057@ip-64-139-1-69.sjc.megapath.net>

> This is already on the Buildbot website.

Which is still useless for me.

My straw man is that our web sites should work with lnyx.  Yes, it won't show 
the cute graphics.  There are occasional useful/important graphs or flow 
charts.

-- 
These are my opinions.  I hate spam.

From verm at darkbeer.org  Mon Nov 30 20:29:02 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 20:29:02 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130202645.GA7787@thyrsus.com>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130202645.GA7787@thyrsus.com>
Message-ID: <20151130202902.GA28891@darkbeer.org>

On 2015-11-30 15:26 -0500, Eric S. Raymond wrote:
> Hal Murray :
> > How about a list of what OS/distros are in buildbot and what configurations 
> > get tested?
> 
> Amar, this should be your next priority after getting the unit tests done.
> It's the other thing we really want for an 0.9.1 release.  Are the unit
> tests done yet?

The list can be gathered from the Buildbot page all the information is there.  
Maintaining a separate list makes no sense.

The unit tests are close.. there are some things I'm trying to iron out as far 
as how to do the C++ -> C conversion some of the last tests rely heavily on 
method overloading.

I also want to do it in a way that lets us test our utility functions.

> What's the URL for the buildbot status page?  I want tp put it on the
> contacts page.

  https://buildbot.ntpsec.org/

From verm at darkbeer.org  Mon Nov 30 20:32:24 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 20:32:24 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130202750.AB327406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130193627.GA28036@darkbeer.org>
 <20151130202750.AB327406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130203224.GB28891@darkbeer.org>

On 2015-11-30 12:27 -0800, Hal Murray wrote:
> 
> My straw man is that our web sites should work with lnyx.  Yes, it won't show 
> the cute graphics.  There are occasional useful/important graphs or flow 
> charts.

I'm sorry but this is not currently possible, I do not believe it will ever be as 
it does not do any server-side HTML rendering.

The page doesn't have any cute graphics it uses javascript + websockets to show 
realtime data as the builds run.  If you keep https://buildbot.ntpsec.org/ open 
it will update automatically when a build runs.

Amar.

From esr at thyrsus.com  Mon Nov 30 20:45:19 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 15:45:19 -0500
Subject: PARSE clocks.
In-Reply-To: <20151130202902.GA28891@darkbeer.org>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130202645.GA7787@thyrsus.com>
 <20151130202902.GA28891@darkbeer.org>
Message-ID: <20151130204519.GA8343@thyrsus.com>

Amar Takhar :
> On 2015-11-30 15:26 -0500, Eric S. Raymond wrote:
> > Hal Murray :
> > > How about a list of what OS/distros are in buildbot and what configurations 
> > > get tested?
> > 
> > Amar, this should be your next priority after getting the unit tests done.
> > It's the other thing we really want for an 0.9.1 release.  Are the unit
> > tests done yet?
> 
> The list can be gathered from the Buildbot page all the information is there.  
> Maintaining a separate list makes no sense.

Agreed.  We want to be able to extract the list for the 0.9.1 release
announcement, though.

> The unit tests are close.. there are some things I'm trying to iron out as far 
> as how to do the C++ -> C conversion some of the last tests rely heavily on 
> method overloading.
> 
> I also want to do it in a way that lets us test our utility functions.

Getting all these working is still the top priority for 0.9.1

> > What's the URL for the buildbot status page?  I want tp put it on the
> > contacts page.
> 
>   https://buildbot.ntpsec.org/

Thanks. The console view seems like the most useful one.  Is there
some way we can see all of the platform labels?  Because they're truncated,
I can't tell what's supposed to be different about the two columns
that begin "Fedora 21 GCC 4".
-- 
		Eric S. Raymond

From verm at darkbeer.org  Mon Nov 30 21:00:11 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 21:00:11 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130204519.GA8343@thyrsus.com>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130202645.GA7787@thyrsus.com>
 <20151130202902.GA28891@darkbeer.org>
 <20151130204519.GA8343@thyrsus.com>
Message-ID: <20151130210010.GA29442@darkbeer.org>

On 2015-11-30 15:45 -0500, Eric S. Raymond wrote:
> > The list can be gathered from the Buildbot page all the information is there.  
> > Maintaining a separate list makes no sense.
> 
> Agreed.  We want to be able to extract the list for the 0.9.1 release
> announcement, though.

OK, I'll write a small Python script to generate this and build it into the 
build system. as 'waf builders' or some similar command.

I should be able to do this to give a status update too which should go some way 
to satisfy the needs for Hal.

> > I also want to do it in a way that lets us test our utility functions.
> 
> Getting all these working is still the top priority for 0.9.1

Agreed, working on it.

> > > What's the URL for the buildbot status page?  I want tp put it on the
> > > contacts page.
> > 
> >   https://buildbot.ntpsec.org/
> 
> Thanks. The console view seems like the most useful one.  Is there
> some way we can see all of the platform labels?  Because they're truncated,
> I can't tell what's supposed to be different about the two columns
> that begin "Fedora 21 GCC 4".

Builds -> Builders in the menu.

This will update realtime when a commit comes in.  It's the view I sit on the 
most.

Amar.

From verm at darkbeer.org  Mon Nov 30 21:01:09 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 21:01:09 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130202750.AB327406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130193627.GA28036@darkbeer.org>
 <20151130202750.AB327406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130210109.GC29442@darkbeer.org>

On 2015-11-30 12:27 -0800, Hal Murray wrote:
> > This is already on the Buildbot website.
> 
> Which is still useless for me.

I'll see if I can write a command to view this from waf... It'll be a rainy day 
or lunchtime project.  If someone has already done similar I'll just include 
that.

Amar.

From esr at thyrsus.com  Mon Nov 30 21:12:17 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 16:12:17 -0500
Subject: PARSE clocks.
In-Reply-To: <20151130210010.GA29442@darkbeer.org>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130202645.GA7787@thyrsus.com>
 <20151130202902.GA28891@darkbeer.org>
 <20151130204519.GA8343@thyrsus.com>
 <20151130210010.GA29442@darkbeer.org>
Message-ID: <20151130211217.GA9056@thyrsus.com>

Amar Takhar :
> On 2015-11-30 15:45 -0500, Eric S. Raymond wrote:
> > > The list can be gathered from the Buildbot page all the information is there.  
> > > Maintaining a separate list makes no sense.
> > 
> > Agreed.  We want to be able to extract the list for the 0.9.1 release
> > announcement, though.
> 
> OK, I'll write a small Python script to generate this and build it into the 
> build system. as 'waf builders' or some similar command.

Wait. Maybe we're talking about different things.  How could 'waf builders' get
a list of non-broken builds without querying the buildbot state real-time?

> > Thanks. The console view seems like the most useful one.  Is there
> > some way we can see all of the platform labels?  Because they're truncated,
> > I can't tell what's supposed to be different about the two columns
> > that begin "Fedora 21 GCC 4".
> 
> Builds -> Builders in the menu.
> 
> This will update realtime when a commit comes in.  It's the view I sit on the 
> most.

You're right.  I'll update the contacts page to point to that.

I assume red cartouches are broken builds.  What are the pink ones, builds
with warnings?
-- 
		Eric S. Raymond

From verm at darkbeer.org  Mon Nov 30 21:23:19 2015
From: verm at darkbeer.org (Amar Takhar)
Date: Mon, 30 Nov 2015 21:23:19 +0000
Subject: PARSE clocks.
In-Reply-To: <20151130211217.GA9056@thyrsus.com>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130202645.GA7787@thyrsus.com>
 <20151130202902.GA28891@darkbeer.org>
 <20151130204519.GA8343@thyrsus.com>
 <20151130210010.GA29442@darkbeer.org>
 <20151130211217.GA9056@thyrsus.com>
Message-ID: <20151130212319.GA29903@darkbeer.org>

On 2015-11-30 16:12 -0500, Eric S. Raymond wrote:
> 
> Wait. Maybe we're talking about different things.  How could 'waf builders' get
> a list of non-broken builds without querying the buildbot state real-time?

It's a JSON API.  It would connect out to buildbot.ntpsec.org and get the JSON 
list and pretty-print it to the console.

All the HTML rendering is done client-side which is why it requires Javascript 
-- it uses WebSockets to be realtime.

> 
> You're right.  I'll update the contacts page to point to that.
> 
> I assume red cartouches are broken builds.  What are the pink ones, builds
> with warnings?

Yep.  Pink ones are ones I interrupted since I was working on it earlier today.  
Yellow ones are ones that have warnings.

Amar.

From esr at thyrsus.com  Mon Nov 30 21:58:05 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 16:58:05 -0500
Subject: Proposed 0.9.1 release objectives, and state of TESTFRAME
Message-ID: <20151130215805.GA8256@thyrsus.com>

I continue to be extremely pleased with the kinds of bug reports we're
getting - enough to establish that people are testing, but nothing
serious.  Notably, we've had no reports of either crashes or timeskeeping
anomalies.

The following is a proposal for review by our project manager: I think
we should target next Monday for 0.9.1, and our two release objectives
should be (a) All unit tests running, and (b) A feature-test-status-vs.
-platform matrix published on the website, so potential users can know
what they're getting into.

I made significant progress on the capture side of TESTFRAME over the
weekend; the intercept layer can now dump packet sends and reads of
both leapsecond and authkey files.  This brings it to intercepting
everything except packet receives and getaddrinfo calls.

Unfortunately, those last two are the hardest.  There's a great deal
of hair in the code around polling up packets from multiple UDP
interfaces; I'm going to have to thoroughly grok all that before I
know where to intervene.  The getaddrinfo stuff is tricky, too,
because it's threaded-asynchronous.

I feel like I need a pith helmet and a bunch of askaris...
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Mon Nov 30 22:22:45 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 14:22:45 -0800
Subject: PARSE clocks.
In-Reply-To: Message from Amar Takhar 
 of "Mon, 30 Nov 2015 20:32:24 GMT." <20151130203224.GB28891@darkbeer.org>
Message-ID: <20151130222245.0F15A406057@ip-64-139-1-69.sjc.megapath.net>

verm at darkbeer.org said:
> The page doesn't have any cute graphics

My comment about graphics was in reference to using lynx.

> it uses javascript + websockets to show realtime data as the builds run.
> If you keep https://buildbot.ntpsec.org/ open  it will update automatically
> when a build runs. 

I haven't gotten past the blank screen stage.

I don't know anything about websockets.  Wikipedia says:
A secure version of the WebSocket protocol is implemented in Firefox 6...
n older, less secure version of the protocol...  Because of vulnerabilities, 
it was disabled in Firefox 4 and 5,...

I think I have an up to date Fedora 23 system.  My firefox comes from:
firefox-42.0-2.fc23.i686
and firefox --version says "Mozilla Firefox 42.0"

The about window says:
 identifier: Mozilla/5.0 (X11; Fedora; Linux i686; rv:42.0) Gecko/20100101 
Firefox/42.0
The Mosilla/5 part is suspicious, but 42 is a long way past 5 and wiki says 
the RFC version came out in Dec 2011 so I'd expect it to be implemented by 
now.

I found a Mozilla web page that mentions Firefox 6 and 7.  It's possible that 
Fedora is way behind for some reason, but they usually err in the other 
direction.

I can't find anything in the GUI stuff that mentions websockets.  I'm 
somewhat expecting a way to turn them on or off.

I found a mozilla bugzilla page that suggests it defaults to off
  https://bugzilla.mozilla.org/show_bug.cgi?id=1091016
  That's from late 2014, and mentions Mozilla 5.0
  Given that network.websocket.enabled is a hidden pref...

Yes, it would be nice to have it update in real time, but I think it's more 
important that basic stuff works without depending on fancy features.  I 
shouldn't have to be a wizard to get something to work.

-- 
These are my opinions.  I hate spam.

From hmurray at megapathdsl.net  Mon Nov 30 22:50:00 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 14:50:00 -0800
Subject: Proposed 0.9.1 release objectives, and state of TESTFRAME
In-Reply-To: Message from "Eric S. Raymond" 
 of "Mon, 30 Nov 2015 16:58:05 EST." <20151130215805.GA8256@thyrsus.com>
Message-ID: <20151130225000.1F135406057@ip-64-139-1-69.sjc.megapath.net>

> The getaddrinfo stuff is tricky, too, because it's threaded-asynchronous. 

You need to intercept the place where the main thread processes the answer coming back from the helper thread rather than intercepting the call to getaddrinfo.  (I'll say more if you want.)

I don't know where that code is.  Chris might know.  If not, I'll find it.  There may be two places, one for servers and another for pools.

Since Chris is working on rewriting that area, I suggest punting for a while.  Just limit your testing to using numeric IP Addresses rather than names.  (pool won't work)  If you are really desperate for server names, hack the lookup code to avoid threading.  There is a call early on that uses getaddrinfo to decode numeric addresses.  It passes in the no-DNS flag.  The only disadvantage of doing the lookup then/there is that startup may take a long time.

------

It might be worth putting a wrong-thread trap on the msyslog intercept.

-- 
These are my opinions.  I hate spam.

From esr at thyrsus.com  Mon Nov 30 22:57:15 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 17:57:15 -0500
Subject: PARSE clocks.
In-Reply-To: <20151130212319.GA29903@darkbeer.org>
References:  <20151130190154.GA27366@darkbeer.org>
 <20151130193239.6E0A6406057@ip-64-139-1-69.sjc.megapath.net>
 <20151130202645.GA7787@thyrsus.com>
 <20151130202902.GA28891@darkbeer.org>
 <20151130204519.GA8343@thyrsus.com>
 <20151130210010.GA29442@darkbeer.org>
 <20151130211217.GA9056@thyrsus.com>
 <20151130212319.GA29903@darkbeer.org>
Message-ID: <20151130225715.GA10091@thyrsus.com>

Amar Takhar :
> On 2015-11-30 16:12 -0500, Eric S. Raymond wrote:
> > 
> > Wait. Maybe we're talking about different things.  How could 'waf builders' get
> > a list of non-broken builds without querying the buildbot state real-time?
> 
> It's a JSON API.  It would connect out to buildbot.ntpsec.org and get the JSON 
> list and pretty-print it to the console.

Right, that sort of thing is easy and fun to write in Python.

It doesn't belong glommed together with the build system, though - completely
different set of concerns. Call it 'buildbot' and throw it in devel/?  We
might want to add more query functions for it later.

Unit tests need to come first, though.
-- 
		Eric S. Raymond

From hmurray at megapathdsl.net  Mon Nov 30 23:26:07 2015
From: hmurray at megapathdsl.net (Hal Murray)
Date: Mon, 30 Nov 2015 15:26:07 -0800
Subject: Testing..
In-Reply-To: Message from "Daniel Poirot" 
 of "Mon, 30 Nov 2015 14:21:48 CST." <00ab01d12bac$bdaee4e0$390caea0$@gmail.com>
Message-ID: <20151130232607.D6FC7406057@ip-64-139-1-69.sjc.megapath.net>

>From a buildbot discussion that started as a discussion about PARSE clocks.

dtpoirot at gmail.com said:
> Would there be any value in me running on 32 and 64, [Open|Free|Net]BSD
> ahead of you? 

There are several things that would be helpful.

Just setting things up and building once would be a good check that our 
directions are adequate.  There are two areas that probably need improvement. 
 One is just the checklist of packages that need installing.  The other is 
how to actually install them.  (Code to copy/paste is probably the best 
approach and I didn't get that far.)

Until the buildbot gets NetBSD working, occasionally building the latest code 
on NetBSD systems would be helpful.

The other area that would be helpful is actually running the code and keeping 
an eye on things to make sure that it works as well as the version that gets 
distributed with the OS.

Checking that ./waf install puts things in a sensible place would help too.  
The man pages used to get installed in some strange place.  I don't know if 
that has been fixed.

For FreeBSD, getting it running is as simple as adding
  ntpd_program="/usr/local/sbin/ntpd"
to /etc/rc.conf, probably right after the place where you put in
  ntpd_enable="YES"

On NetBSD, I edited /etc/rc.d/ntpd:
  # command="/usr/sbin/${name}"
  command="/usr/local/sbin/${name}"

On Fedora, I edited /usr/lib/systemd/system/ntpd.service:
  # ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS
  ExecStart=/usr/local/sbin/ntpd -u ntp:ntp $OPTIONS

I'll work on some notes about what to look for.

------

I don't think anybody has tried it on OpenBSD yet.

-- 
These are my opinions.  I hate spam.

From dtpoirot at gmail.com  Mon Nov 30 23:41:37 2015
From: dtpoirot at gmail.com (Daniel Poirot)
Date: Mon, 30 Nov 2015 17:41:37 -0600
Subject: Testing..
In-Reply-To: <20151130232607.D6FC7406057@ip-64-139-1-69.sjc.megapath.net>
References: Message from "Daniel Poirot"  of "Mon,
 30 Nov 2015 14:21:48 CST."
 <00ab01d12bac$bdaee4e0$390caea0$@gmail.com>
 <20151130232607.D6FC7406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <001801d12bc8$a8233bf0$f869b3d0$@gmail.com>

What are these "directions" of which you speak?  ;-)

I don't mind keeping an eye on the outlier operating systems... I can get
just about anything to compile.

I have "kernel and compiler" VMs for all six BSD's already setup. 

'waf configure' on OpenBSD poops out with libevent2

I have wasted many an hour trying to get gcc running on the Android QEMU
emulator VM. It is NOT going my way! 

I may have to switch to 'real' hardware instead. GCC on my phone, who would
have ever thought...

- dan

-----Original Message-----
From: Hal Murray [mailto:hmurray at megapathdsl.net] 
Sent: Monday, November 30, 2015 5:26 PM
To: Daniel Poirot
Cc: devel at ntpsec.org; hmurray at megapathdsl.net
Subject: Testing..

>From a buildbot discussion that started as a discussion about PARSE clocks.

dtpoirot at gmail.com said:
> Would there be any value in me running on 32 and 64, 
> [Open|Free|Net]BSD ahead of you?

There are several things that would be helpful.

Just setting things up and building once would be a good check that our
directions are adequate.  There are two areas that probably need
improvement. 
 One is just the checklist of packages that need installing.  The other is
how to actually install them.  (Code to copy/paste is probably the best
approach and I didn't get that far.)

Until the buildbot gets NetBSD working, occasionally building the latest
code on NetBSD systems would be helpful.

The other area that would be helpful is actually running the code and
keeping an eye on things to make sure that it works as well as the version
that gets distributed with the OS.

Checking that ./waf install puts things in a sensible place would help too.

The man pages used to get installed in some strange place.  I don't know if
that has been fixed.

For FreeBSD, getting it running is as simple as adding
  ntpd_program="/usr/local/sbin/ntpd"
to /etc/rc.conf, probably right after the place where you put in
  ntpd_enable="YES"

On NetBSD, I edited /etc/rc.d/ntpd:
  # command="/usr/sbin/${name}"
  command="/usr/local/sbin/${name}"

On Fedora, I edited /usr/lib/systemd/system/ntpd.service:
  # ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS
  ExecStart=/usr/local/sbin/ntpd -u ntp:ntp $OPTIONS

I'll work on some notes about what to look for.

------

I don't think anybody has tried it on OpenBSD yet.

--
These are my opinions.  I hate spam.

From esr at thyrsus.com  Mon Nov 30 23:48:54 2015
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 30 Nov 2015 18:48:54 -0500
Subject: Proposed 0.9.1 release objectives, and state of TESTFRAME
In-Reply-To: <20151130225000.1F135406057@ip-64-139-1-69.sjc.megapath.net>
References:  <20151130215805.GA8256@thyrsus.com>
 <20151130225000.1F135406057@ip-64-139-1-69.sjc.megapath.net>
Message-ID: <20151130234854.GB10091@thyrsus.com>

Hal Murray :
> 
> > The getaddrinfo stuff is tricky, too, because it's threaded-asynchronous. 
> 
> You need to intercept the place where the main thread processes the answer coming back from the helper thread rather than intercepting the call to getaddrinfo.  (I'll say more if you want.)

Yeah, I'd figured that much out.

> I don't know where that code is.  Chris might know.  If not, I'll find it.  There may be two places, one for servers and another for pools.

Agreed, I think there are two distinct places.  The second one, in the middle of
ntp_proto, complicates life a lot.

> Since Chris is working on rewriting that area, I suggest punting for a while.  Just limit your testing to using numeric IP Addresses rather than names.  (pool won't work)  If you are really desperate for server names, hack the lookup code to avoid threading.  There is a call early on that uses getaddrinfo to decode numeric addresses.  It passes in the no-DNS flag.  The only disadvantage of doing the lookup then/there is that startup may take a long time.

Your advice is good, I think.  I had already considered limiting early testing
to numeric IPs to avoid this problem.

I think I'll try timing a version buit to do all its DNS lookups at
start.  Network latencies have been dropping.  It is at least possible
that the asynchronous lookup is a performance hack that we no longer need.

> It might be worth putting a wrong-thread trap on the msyslog intercept.

Good idea.
-- 
		Eric S. Raymond