How the KERNEL_PLL switch kills TESTFRAME

[Eric S. Raymond]

Try not to send mail in HTML; mutt can't read it.

John Bell <jdb at systemsartisans.com>:
>At the risk of sounding stupid (or at least uninformed), couldn't you
>capture TESTFRAME logs from the same hardware *booted on each kind of
>kernel* (with the kernels being otherwise identically configured),
>and just have the WITH and WITHOUT sets of tests to run?

In theory, possible.  It's not the kernel that needs to be configured
differently, it's ntpd.  But you have the right general idea.

>Yes, doubled workload; is that a show-stopper?

The build-time workload wouldn't necessarily even double at all, you
could just run only the set of tests appropriate for your platform.

>Can we scrounge up enough testing environments without having to
>reboot any individual box?

Easily, because no reboots would actually be required.  In any case
that sort of thing is what Raspberry Pis are for.

No, the real problem with this workaround is that all our choices
about how to implement that split would introduce enough complexity
and friction into our test regime to be a real problem (I've been
running these scenarios in my head for a year).  The original appeal
of TESTFRAME was the simplicity of one test runs everywhere and "waf
check" runs all tests; that's gone now, no matter how clever we get.

Remember, you can't even *build* the KERNEL_PLL version on a platform
without adjtimex(2).  So the simplest option, a runtime switch, is out.
You can't even do intercepts on a non-PLL system without the "real" code,
because it won't have the right header files for the data structures
you need to mock.  (Unless you carry copies in the distribution, a
really bad idea because they *will* desynchronize with the real ones
and then you will be hosed in arbitrily nasty and subtle ways.)
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>