Python Library Cleanups

[Fred Wright]

On Mon, 25 Sep 2017, Gary E. Miller via devel wrote:
> On Mon, 25 Sep 2017 20:26:44 -0700 (PDT)
> Fred Wright via devel <devel at ntpsec.org> wrote:
>
> > 1) Waf misuses get_python_lib() in a way that often gets the wrong
> > result, with the effect of installing the libraries in a location
> > where Python doesn't look for them.  This led to various
> > inappropriate recommendations to set PYTHONPATH.  PYTHONPATH is a
> > special developer kludge, analogous to LD_LIBRARY_PATH, which should
> > never be set in normal operation.  Code that depends on setting
> > PYTHONPATH isn't production-ready.
>
> The default gentoo PYTHONPATH onlly looks for system installed
> python libs.  We don't want to install NTPsec python files in the
> system reserved directories.  Thus PYTHONPATH is the only solution.

The location returned by get_python_lib() *without the prefix argument* is
exactly what the GPSD install procedure uses, and why GPSD has never
needed PYTHONPATH.  In fact, the way I tracked down the problem is by
asking why GPSD doesn't have the same problem.

If the directory choice on gentoo is inappropriate, take that up with
whomever packaged Python for it.

> > 2) The in-tree testing setup wasn't fully functional.  In the process
> > of fixing that, I noticed that it was set up to apply the hacks to the
> > *source* tree, which is inconsistent with the project's organizational
> > philosophy of keeping build products out of the source tree.  I've
> > reworked that, and it now works for all Python versions and handles
> > the tests as well as the clients, but the switch to the build-tree
> > orientation has operational impact (on developers, not users):
>
> What was not functional?  I'm all for keeping the source tree clean
> and building in the build directories.

It didn't work with Python 3, and it didn't work for the tests.

> Can you split apart your MR?  It addresses different issues and
> at least #1 is probably not gonna happen.

It's already split into multiple commits, and there seems to be some
difference of opinion regarding the latter.

> Plus these are most likely too late for 1.0.  We are in the final
> testing phases now.

I think requiring users to set PYTHONPATH to run the tools should be
considered a show stopper for a 1.0 release.

These changes are almost entirely just in the build scripts, with the only
actual code changes (Python only) being almost entirely in the error
reporting for not finding the libraries.

Fred Wright