Fix for Python library path problem

[Fred Wright]

On Wed, 27 Sep 2017, Eric S. Raymond via devel wrote:
> Gary E. Miller via devel <devel at ntpsec.org>:
> > What we see that pip does, is edit the sys.path to include the
> > location an egg is installed.  That looks to me like a method
> > to go forward with.
>
> It looks to me like a fscking disaster, introducing yet another
> complication that will cause us endless headaches.
>
> What, now whether an NTPsec install works is going to depend on whether
> popi has been run on some random package with an egg? No, no, and no.
> That's horrible.

Let's step back and look at what the actual requirements are.

First of all, in order for an import to work in Python, the directory
containing the module needs to be in sys.path at the time of the import.
This requirement can be met in one of two ways:

1) Placing the module in a directory which is already in sys.path by
default.

2) Arranging to augment sys.path by some means prior to the import.

One of the ways to do #1 is to use the path returned by get_python_lib()
without the prefix option.  This is what GPSD has done for years.  There
may be other possibilities in this area, but it doesn't look promising in
the general case, if the goal is to come up with a "local" path.

For approach #2, there are a few possibilities:

2.1) Require users to set up PYTHONPATH appropriately.  There seems to be
general agreement that this is a bad idea.

2.2) Make use of some sort of hook to augment sys.path, which is what pip
appears to do.  So far, this approach doesn't look promising, partly
because the only such mechanisms seem to require that the hook itself be
in a "non-local" location in order to be seen by Python before being
applied.  A chicken-and-egg problem.

2.3) Add code to the programs to augment sys.path prior to the import.
This would need to be in all the programs; common code in a library would
suffer from a chicken-and-egg problem.  Since the target location is
site-specific, the programs would need to be patched at build time with
the correct path.  This actually shouldn't be too hard, since they're
already being copied by the substituter, as long as there are no gotchas
with the substituter.  It would work, but be ugly and nonstandard, though
it could be limited to the cases where it's actually needed.

Other suggestions?

Fred Wright