Fix for Python library path problem

[Eric S. Raymond]

Fred Wright via devel <devel at ntpsec.org>:
> Let's step back and look at what the actual requirements are.
> 
> First of all, in order for an import to work in Python, the directory
> containing the module needs to be in sys.path at the time of the import.
> This requirement can be met in one of two ways:
> 
> 1) Placing the module in a directory which is already in sys.path by
> default.
> 
> 2) Arranging to augment sys.path by some means prior to the import.
> 
> One of the ways to do #1 is to use the path returned by get_python_lib()
> without the prefix option.  This is what GPSD has done for years.  There
> may be other possibilities in this area, but it doesn't look promising in
> the general case, if the goal is to come up with a "local" path.
> 
> For approach #2, there are a few possibilities:
> 
> 2.1) Require users to set up PYTHONPATH appropriately.  There seems to be
> general agreement that this is a bad idea.
> 
> 2.2) Make use of some sort of hook to augment sys.path, which is what pip
> appears to do.  So far, this approach doesn't look promising, partly
> because the only such mechanisms seem to require that the hook itself be
> in a "non-local" location in order to be seen by Python before being
> applied.  A chicken-and-egg problem.
> 
> 2.3) Add code to the programs to augment sys.path prior to the import.
> This would need to be in all the programs; common code in a library would
> suffer from a chicken-and-egg problem.  Since the target location is
> site-specific, the programs would need to be patched at build time with
> the correct path.  This actually shouldn't be too hard, since they're
> already being copied by the substituter, as long as there are no gotchas
> with the substituter.  It would work, but be ugly and nonstandard, though
> it could be limited to the cases where it's actually needed.
> 
> Other suggestions?

After researching and trying these out for the last couple days I have
concluded that 2, attempts to adjust the sys.path, are doomed.

site.py won't do it.  It adjusts the load path at Python startup time, but
only in accordance with sys.prefix at the time Python was built.  That's
going to be /usr for Python installed from the package system.  We only get
help from this if the Python instance was built from source with a /usr/local
prefix.

The 2.x alternatives are all complexity traps.  I rejected 2.2 immediately
and experimented with ways to implement 2.3.  Every present and future
client would have templated - you can't do it in the support library
for reasons Fred has already observed.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.