Python ntpq lands - what to do next?

[Eric S. Raymond]

Hal Murray <hmurray at megapathdsl.net>:
> 
> esr at thyrsus.com said:
> > OK, I've just modified the autorevision command in pylib to make and use a
> > cache file.  The cache file location is declared as a target, which should
> > mean it gets picked up by waf dist. 
> 
> I think that's working with my hack and with a real tarball.  Thanks.

Good, that's one down.

> There is still a can of worms in this area.  If I do a git pull and build to 
> setup that cache, then do an edit but no commit, version.py doesn't show any 
> difference between the before/after than edit.

Right, and it should change at least the value of WC_MODIFIED.  The only way to
make sure that happens would be to force autorevision to be run on
every build.  I've left a question about this on #waf.  The maintainer is
pretty good about answering those.

> >> waf clean doesn't get pylib/ntp_control.py or pylib/ntp_magic.py or 
> >> pylib/version.py
> > A minor bug.  What's special about those files is that they're generated
> > into the pylib *source* directory by the normal build; waf doesn't try to
> > clean that directory. 
> 
> The c file from bison goes into xxx/host/ntpd/ntp_parser.tab.c
> so waf knows how to work with source files that aren't in the source tree.
> But I don't know if python can find them.

This is yet another face of the mess involving pylib and the build directory.

There are Python libraries needed by ntpq and (soon) by ntpmon under
pylib.  One wrinkle is that they need to be installed not as 'pylib'
under the Python dist-files directory but rather as 'ntp'. That means
we need a custom installation production, which I think I don't have
quite right yet.  Some Python files are being installed twice, once
in the right place under /usr/local/lib/python2.7/dist-packages/
and once under /usr/local/lib.

Another is that we need (or at least I think we need) to set things
up so that, e.g. ntpq can be tested from under ntpq/ without installing
pylib to rootspace.  This constraint is which these files exist:

lrwxrwxrwx 1 esr esr 8 Nov  6 00:14 ntpq/ntp -> ../pylib
lrwxrwxrwx 1 esr esr 9 Oct 20 23:47 ntpstats/ntp -> ../pylib/
lrwxrwxrwx 1 esr esr 9 Oct 20 23:47 ntpsweep/ntp -> ../pylib/
lrwxrwxrwx 1 esr esr 8 Oct 20 23:47 ntptrace/ntp -> ../pylib
lrwxrwxrwx 1 esr esr 8 Oct 20 23:47 ntpwait/ntp -> ../pylib

When you run Python in any of these directories and import (say)
ntp.packet, the symlink forwards that import to pylib.

That would be sufficient if everything you needed to load were one
of the Python sourcefiles that is at home there.  Unfortunately, this
isn't the case.  There are two kinds of generated things in that
directory.

One is three files, version.py, ntp_control.py, and ntp_magic.py, that
are generated during a build. As you now know, version.py is generated
by autorevision.sh and it is not yet clear how to get it to run often
enough. The ntp_control.py and ntp_magic.py files are script-generated
translations of C header files that make a bunch of macro definitions
visible to Pyton.

The other special case is ntpc.so.  This is the Python extension module.
It is generated into the build directory.  In order to make it visible
when running ntpq/ntpq before installation, there is this symlink.

lrwxrwxrwx 1 esr esr    28 Nov  5 17:00 pylib/ntpc.so -> ../build/main/libntp/ntpc.so

This is the part that is coming unstuck for you, because that's a static
link that doesn't track where --out is putting the build dirctory.

Now I'm trying to figure out a way to swap the static "ntp" directory links
for waf-generated links that forward to pylib/ under the *build* directory,
whetever that is (and even if you're using --out).

ntpq/ntpq would no longer run before a waf build, but I could simplify
things by having the generated Python be put there rather than hanging
out as special caseds in the source directory.

I'm sure these is a way to do this, but the opacity of the waf documentation
makes it unreasonably difficult to figure that out.

I may have to unbend and change pylib/ to ntp/.  I dislike giving away
that piece of the tree's namespace, but the name change confuses waf
and requires ugly, only semi-working klugery.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>