Attn: Install path debaters

[Richard Laager]

On 01/03/2018 04:44 PM, Gary E. Miller via devel wrote:
>>> Rather than having me misread your code, can you put a plain
>>> summary here?
PYTHONDIR and PYTHONARCHDIR (whatever they are) are embedded into the
executables and loaded into sys.path at runtime. This is *in addition*
and *in priority to* the default entries in sys.path, and used *only*
for importing the "ntp" module.

PYTHONDIR and PYTHONARCHDIR are calculated without any "fixes". I didn't
write the code to calculate them. I just removed fix_python_config.py.
My understanding is they are calculated as follows:

from distutils import sysconfig
PYTHONDIR=sysconfig.get_python_lib(plat_specific=0, prefix=PREFIX)
PYTHONARCHDIR=sysconfig.get_python_lib(plat_specific=1, prefix=PREFIX)

> Then maybe rlaager can provide a summary of the 4 different cases:
> 
>         A. distro install: /etc, /bin, etc.
>         B. system install: /usr/local/etc, /usr/local/etc, etc.
>         C. user install: ~/etc, ~/etc, etc. or similar
>         D. packager install: /tmp/etc, /tmp/bin, going into a package
It supports any and all arbitrary paths. The module code is already
being installed to PYTHONDIR and PYTHONARCHDIR. This ensures that the
utilities know what those locations are.

Let's review the common scenarios, assuming Python 2.7 installed to
/usr. I will use the upstream "site-packages" consistently rather than
the Debian dist-packages. If you really want to know, on Debian, "A",
"B", and "D" use dist-packages, which "C" uses site-packages.


A:
./waf configure --prefix=/usr --sysconfdir=/etc
PREFIX: /usr
PYTHONDIR: /usr/lib/python2.7/site-packages
PYTHONARCHDIR: /usr/lib/python2.7/site-packages

sudo ./waf install

# The utilities get this embedded:
PYTHONDIR='/usr/lib/python2.7/site-packages'
PYTHONARCHDIR='/usr/lib/python2.7/site-packages'

This duplicates default locations from sys.path. That is harmless.


B:
# This defaults to --prefix=/usr/local
./waf configure
PREFIX: /usr/local
PYTHONDIR: /usr/local/lib/python2.7/site-packages
PYTHONARCHDIR: /usr/local/lib/python2.7/site-packages

sudo ./waf install

# The utilities get this embedded:
PYTHONDIR='/usr/local/lib/python2.7/site-packages'
PYTHONARCHDIR='/usr/local/lib/python2.7/site-packages'

If /usr/local was in sys.path (i.e. distro patching), then this
duplicates a default location, but moves it up in priority. By default,
it was *after* /usr/lib/python2.7/site-packages but now it is *before*.

If /usr/local was not in sys.path (i.e. no distro patching), then this
fixes the original problem. It ensures that we can find the ntp module.

In either case, this ensures that the ntp module from this "B" install
is used. Otherwise, the module from an "A" install would be used, if it
exists. This is an advantage of this method over the .pth approach. This
approach allows an "A" and "B" install to co-exist.


C:
./waf configure --prefix=$HOME/.local
PREFIX: /home/rlaager/.local
PYTHONDIR: /home/rlaager/.local/lib/python2.7/site-packages
PYTHONARCHDIR: /home/rlaager/.local/lib/python2.7/site-packages

./waf install

# The utilities get this embedded:
PYTHONDIR='/home/rlaager/.local/lib/python2.7/site-packages'
PYTHONARCHDIR='/home/rlaager/.local/lib/python2.7/site-packages'

This path was probably already in sys.path. But again, it is moved up,
so the "C" install is preferred over "A". The .pth approach would not
address that.


Arbitrary $HOME (i.e. not $HOME/.local) install:

./waf configure --prefix=$HOME/bin/.ntpsec
PREFIX: /home/rlaager/bin/.ntpsec
PYTHONDIR: /home/rlaager/bin/.ntpsec/lib/python2.7/site-packages
PYTHONARCHDIR: /home/rlaager/bin/.ntpsec/lib/python2.7/site-packages

./waf install

# The utilities get this embedded:
PYTHONDIR='/home/rlaager/bin/.ntpsec/lib/python2.7/site-packages'
PYTHONARCHDIR='/home/rlaager/bin/.ntpsec/lib/python2.7/site-packages'

This arbitrary path would never be in the default sys.path. Adding it is
required to make this work. Here again, adding it causes the ntp module
from this install to be preferred over the version from "A" or "B".


D:
./waf configure --prefix=/usr --sysconfdir=/etc
PREFIX: /usr
PYTHONDIR: /usr/lib/python2.7/site-packages
PYTHONARCHDIR: /usr/lib/python2.7/site-packages

sudo ./waf install --destdir=debian/tmp

# The utilities get this embedded:
PYTHONDIR='/usr/lib/python2.7/site-packages'
PYTHONARCHDIR='/usr/lib/python2.7/site-packages'

Then the package build system bundles everything up into a .deb, which
when eventually unpacked, puts the files in /usr, resulting in the same
result as "A".

This duplicates default locations from sys.path. That is harmless.


> And, most important, in some systems, all FOUR must be able to
> coexist.

Then you need this, not .pth. For example, if you install a .pth file
for the /usr/local case ("B"), /usr is still higher in sys.path, so the
executables from "B" will load the module from "A".

-- 
Richard

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ntpsec.org/pipermail/devel/attachments/20180103/f3447ada/attachment.bin>