My pre-1.0 wishlist

[Hal Murray]

[resend to fix typo in cc list]

dfoxfranke at gmail.com said:
> I want to test things at the *user-visible* level. Automate the process of
> supplying configuration files that exercise a variety of functionality,
> running them on real hardware and real networks, and monitoring the results
> with ntpq. That is, take the sort of testing that we're already doing by
> hand and make it systematic and automatic.

I'd probably look at the log files rather than use ntpq.

I'll break that approach into 2 sections.  The first is "Does it work as 
expected under normal conditions?"  Mostly, that needs a test farm to run all 
the combinations we want to test.

The second part is "How many abnormal conditions can we test in an automated 
way?"  That will require generating the abnormal conditions, which probably 
requires a larger test farm.

Some of that will be straightforward.  For example, we can use util/bumpclock 
to bump the clock and watch it recover.  There are several cases. ...

Some of that will be complicated.  For example, how do we fake a GPS receiver 
fading out or long queuing delays on internet links?

---------

> We're using two different senses of "stable". You're using it to mean "free
> of crashes". I'm using it way Debian uses it: a version is stable when it
> ceases to require frequent patching, and if a particular, serious issue
> comes up (usually but not always security-related) then the maintainers will
> provide an update that fixes that issue and leaves everything else
> untouched. A corollary to something being stable in this sense is that we
> can afford high overhead for making changes, because we don't have to make
> very many of them. 

What's the time scale?  How long will distros continue running old "stable" 
versions?




-- 
These are my opinions.  I hate spam.