Fix for Python library path problem

[Eric S. Raymond]

Fred Wright via devel <devel at ntpsec.org>:
> 
> On Wed, 27 Sep 2017, Eric S. Raymond via devel wrote:
> 
> > I've pushed a fix for Fred Wright's FixConfig class that seems to
> > solve the problem of incorrect Python library locations.
> >
> > I tested it with no --prefix option and with --prefix=/usr,
> > using install --destdir=/tmp/ntp.
> >
> > Gary, please verify that this addresses your FHS concerns.
> >
> > Fred, please tell me if you think this is broken in some obscure way.
> 
> I'm not sure about "obscure", but if the result isn't in sys.path, then
> it's back to the same old problem.

That's right.  What we can do, though, is win under the following assumption:
if /usr/lib/X/Y/ is in sys.path, so is /usr/local/lib/X/Y/.  Look at
this from my system:

>>> [x for x in sys.path if x.find('/usr/lib') != -1]
['/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
>>> [x for x in sys.path if x.find('/usr/lib') != -1 and x.replace('/usr/lib', '/usr/local/lib') == -1]
[]
>>> [x for x in sys.path if x.find('/usr/lib') != -1 and x.replace('/usr/lib', '/usr/local/lib') != -1]
['/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']

Ubuntu follows that rule.  I think it's a safe bet that everything else does
too - in part because of FHS, in part because of ancient autoconf conventions.

A slightly more sophisticated version would pop off the first path component
and replace it with PREFIX.  We might need to do that if we ever hit a platform
that really wants to install NTP under /opt.  I think this is unlikely, I've
never heard of the /opt convention being used for system daemons.

> Looking at the waf change that introduced the trouble, it looks like it
> was mainly motivated by wanting to allow --prefix to influence the results
> (even though one can always supply --pythondir and --pythonarchdir), and
> they simply caused the no --prefix case to pas the default prefix instead
> of nothing, perhaps without realizing how this screws up the result.

Would you be willing to file an issue about this on the waf tracker?

> AFAICT, Python simply doesn't follow FHS on Linux.

Doesn't it?  Look at my example again.  It looks a lot like somebody, either
Python or Ubuntu's Python packagers, has gone to the effort to ensure that
FHS-compliant library directories under /usr/local/lib exist in parallel with
every system library directory under /usr/lib.

I just checked Raspbian on one of my Pis.  General rule works there too.

Gentoo and our other platforms almost certainly have the same regularity,
otherwise it would be an almighty coincidence that what Gary
considers the right (FHS-compliant) locations were good before your
MR.

I think we're done here.  I'll add an explanatory comment to the massage() logic.

Just to be sure, though, people with access to other platforms - like Red Hat
and FreeBSD - should run these checks in Python

>>> [x for x in sys.path if x.find('/usr/lib') != -1]

>>> [x for x in sys.path if x.find('/usr/lib') != -1 and x.replace('/usr/lib', '/usr/local/lib') == -1]

If the second one ever comes up non-empty we could have a problem.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.