Fix for Python library path problem
Fred Wright
fw at fwright.net
Wed Sep 27 20:45:30 UTC 2017
On Wed, 27 Sep 2017, Eric S. Raymond wrote:
> Fred Wright via devel <devel at ntpsec.org>:
> >
> > On Wed, 27 Sep 2017, Eric S. Raymond via devel wrote:
> >
> > > I've pushed a fix for Fred Wright's FixConfig class that seems to
> > > solve the problem of incorrect Python library locations.
> > >
> > > I tested it with no --prefix option and with --prefix=/usr,
> > > using install --destdir=/tmp/ntp.
> > >
> > > Gary, please verify that this addresses your FHS concerns.
> > >
> > > Fred, please tell me if you think this is broken in some obscure way.
> >
> > I'm not sure about "obscure", but if the result isn't in sys.path, then
> > it's back to the same old problem.
>
> That's right. What we can do, though, is win under the following assumption:
> if /usr/lib/X/Y/ is in sys.path, so is /usr/local/lib/X/Y/. Look at
> this from my system:
>
> >>> [x for x in sys.path if x.find('/usr/lib') != -1]
> ['/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
> >>> [x for x in sys.path if x.find('/usr/lib') != -1 and x.replace('/usr/lib', '/usr/local/lib') == -1]
> []
> >>> [x for x in sys.path if x.find('/usr/lib') != -1 and x.replace('/usr/lib', '/usr/local/lib') != -1]
> ['/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
>
> Ubuntu follows that rule. I think it's a safe bet that everything else does
> too - in part because of FHS, in part because of ancient autoconf conventions.
>
> A slightly more sophisticated version would pop off the first path component
> and replace it with PREFIX. We might need to do that if we ever hit a platform
> that really wants to install NTP under /opt. I think this is unlikely, I've
> never heard of the /opt convention being used for system daemons.
>
> > Looking at the waf change that introduced the trouble, it looks like it
> > was mainly motivated by wanting to allow --prefix to influence the results
> > (even though one can always supply --pythondir and --pythonarchdir), and
> > they simply caused the no --prefix case to pas the default prefix instead
> > of nothing, perhaps without realizing how this screws up the result.
>
> Would you be willing to file an issue about this on the waf tracker?
>
> > AFAICT, Python simply doesn't follow FHS on Linux.
>
> Doesn't it? Look at my example again. It looks a lot like somebody, either
> Python or Ubuntu's Python packagers, has gone to the effort to ensure that
> FHS-compliant library directories under /usr/local/lib exist in parallel with
> every system library directory under /usr/lib.
Whether the directories exists isn't the point. No directories under
/usr/local/lib are in the default sys.path. Hence directories of that
form don't work for imports without special help. E.g.:
fw at ubuntu:~$ python -c 'import sys; print(sys.path)'
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PILcompat',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
> I just checked Raspbian on one of my Pis. General rule works there too.
>
> Gentoo and our other platforms almost certainly have the same regularity,
> otherwise it would be an almighty coincidence that what Gary
> considers the right (FHS-compliant) locations were good before your
> MR.
Well, as long as "good" doesn't have to mean "working without PYTHONPATH
kludgery", then yes. :-)
As far as using the non-FHS location being "evil" goes, note:
fw at ubuntu:~$ python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/usr/lib/python2.7/dist-packages
fw at ubuntu:~$ ls /usr/lib/python2.7/dist-packages
adium_theme_ubuntu-0.3.4.egg-info hpmudext.so
python_apt-0.9.3.5ubuntu2.egg-info
ANSI.py httplib2
python_debian-0.1.21_nmu2ubuntu2.egg-info
ANSI.pyc httplib2-0.8.egg-info
pyxdg-0.25.egg-info
apt ibus
README
aptdaemon indicator_keyboard
reportlab
apt_inst.so ldb.so
reportlab-3.0.egg-info
apt_pkg.so _ldb_text.py
requests
aptsources _ldb_text.pyc
requests-2.2.1.egg-info
apt_xapian_index-0.45.egg-info libxml2mod.so
samba
axi libxml2.py
scanext.la
cairo libxml2.pyc
scanext.so
chardet lockfile-0.8.egg-info
screen.py
chardet-2.0.1.egg-info lockfile.py
screen.pyc
CommandNotFound lockfile.pyc
serial
command_not_found-0.3.egg-info lsb_release.py
sessioninstaller
Crypto lsb_release.pyc
sessioninstaller-0.0.0.egg-info
cupsext.la lxml
sipconfig_nd.py
cupsext.so lxml-3.3.3.egg-info
sipconfig_nd.pyc
cupshelpers ntdb.so
sipconfig.py
cups.so oauthlib
sipconfig.pyc
curl oauthlib-0.6.1.egg-info
sip.so
dblatex-0.3.4_3.egg-info oneconf
six-1.5.2.egg-info
dbtexmf oneconf-0.3.7.14.04.1.egg-info
six.py
dbus OpenSSL
six.pyc
_dbus_bindings.so PAM-0.4.2.egg-info smbc
_dbus_glib_bindings.so PAM.x86_64-linux-gnu.so
_smbc.so
deb822.py pcardext.la
softwarecenter_aptd_plugins
deb822.pyc pcardext.so
software_center_aptd_plugins-0.0.0.egg-info
debconf.py pexpect
ssh_import_id-3.21.egg-info
debconf.pyc pexpect-3.1.egg-info
system_service-0.1.6.egg-info
debian PIL
talloc.so
debian_bundle PILcompat
tdb.so
debtagshw PILcompat.pth
_tdb_text.py
debtagshw-0.1.egg-info Pillow-2.3.0.egg-info
_tdb_text.pyc
defer piston_mini_client
twisted
defer-1.0.6.egg-info piston_mini_client-0.7.5.egg-info
Twisted_Core-13.2.0.egg-info
dirspec pkg_resources.py
Twisted_Web-13.2.0.egg-info
dirspec-13.10.egg-info pkg_resources.pyc
ubuntu-sso-client
drv_libxml2.py pxssh.py
ubuntu-sso-client.pth
drv_libxml2.pyc pxssh.pyc
UbuntuSystemService
duplicity pycrypto-2.6.1.egg-info
unity_lens_photos-1.0.egg-info
duplicity-0.6.23.egg-info pycups-1.9.66.egg-info
urllib3
fdpexpect.py pycurl-7.19.3.egg-info
urllib3-1.7.1.egg-info
fdpexpect.pyc pycurl.so
xapian
FSM.py pygobject-3.12.0.egg-info xdg
FSM.pyc pygtkcompat
xdiagnose
gi pygtk.pth
xdiagnose-3.6.3build2.egg-info
glib pygtk.py
zeitgeist
gobject pygtk.pyc zope
gps-3.9.egg-info pyOpenSSL-0.13.egg-info
zope.interface-4.0.5.egg-info
gtk-2.0 PyQt4
zope.interface-4.0.5-nspkg.pth
gtk-2.0-pysupport-compat.pth pyserial-2.6.egg-info
hpmudext.la pysmbc-1.0.14.1.egg-info
So most of the world elects to follow Python, not FHS.
Fred Wright
More information about the devel
mailing list