Python libs on Debian/Raspbian

[Gary E. Miller]

Yo Richard!

On Tue, 19 Dec 2017 21:00:48 -0600
Richard Laager <rlaager at wiktel.com> wrote:

> On 12/19/2017 07:50 PM, Gary E. Miller via devel wrote:
> 
> I think we're on the same page.
> 
> > I) waf could install a file in /usr/local/etc to tell ntpsec Python
> > programs where to look.  
> 
> How does the utility know to look in /usr/local/etc? If we have to put
> the PREFIX into the utility, this is a more complicated version of
> "H".

As I said: "For completeness, other options, not recommendations"

The solution has been raised already by others, not advocating it,
just trying to be thorugh on the options.

But, as a WAG, if, for example, ntploggps is run from /usr/local/bin
the it might make sense to look in /usr/local/etc.


> To shorten:
> E) PYTHONPATH in the environment
> F) .pth file in /usr/.../pythonX.Y/....-packages

Uh, no.  I looked at this some more.  That first ... can only be lib,
lib32 or lib64.  Waf can not write there, Python only looks there.

> G) utilities look at argv[0] path and modify sys.path

Hmm.  I just tried that, no good.  Here is what I get for argv:

# cat test.py
import sys
print( sys.argv)

# python test.py
['test.py']

But there are other ways to find where the script was run from:

    os.path.dirname(os.path.realpath(__file__))

So, if the modules are always in ../lib/pythonX.Y/ we are good.  Except
/lib may instead by /lib32 or /lib64.

> H) waf puts PREFIX (or PYTHONDIR) into utilities
> 
> I (and some others from what I see) feel F > E, because environment
> variables are hard to get inherited everywhere (cron, systemd, sudo,
> etc.). However, this either requires user-action or violating PREFIX.

Except no one has come up with a .pth plan that actually works.

Also, remember that we have the contraint where the user may have all four
of these installed:

/usr, /usr/lib, ~user, gitclone.  If you have a .pth, then you can no
longer use waf to run regression tests on the gitclone dir.  The
distro ntpd, the local installed ntpd, the user installed ntpd, and the
waf test copy of ntpd, will ALL be forced to use the one and same
modules pointed at by the .pth file.

Sure, there are some possible mitigations, but complexity starts to
balloon.  Any solution needs to work for ALL the cases in a simple
and obvious manner.

Furthermore, the user may not even have write perms to /usr/lib.

And funny about the cron, sudo, openrc, etc. environments.  I seem to
have no difficulty at all setting environment variables in there.  Plus
the bonus of I can simul;taneously have installes, and run to my taste,
the files in /usr, /usr/lib, ~user and gitclone.

F can not do that.

Luckily the options are not mutually exclusive.  Waf can make
more than one recommendation to the user, the doc can mention more
than one option.  If the user wants to seriously limit his options by
going with .pth files then he is free to do so.  After he is warned
of the many problems therein.  Like waf tests will now use the
wrong modules.

> What's wrong with G or H? Either would make this work without user
> action, which is superior to both E & F. For that reason, they seem
> like attractive ways to get the right value into sys.path. Of the
> two, H seems a little more "normal" than G.

Possibly.  As i said, I only included them as soluttions already
proposed, so out options would be complete.

H has problems if the user moves the binaries.  An important case of
the binaries being moved is distro packagers installing to one place,
then scopping up the bianries into a package that gets installed
elsewhere.

I'd agree G is a bit non-standard, but certainly not unprecedented.
I have seen projects that are happy with any tree that looks liek:
	XX/bin
	XX/sbin
	XX/lib
	XX/log

So, to recap.

1) no solution makes everyone happy
2) some solutions need fleshing out
3) any solution needs to support simultaneous, multiple, parallel installs

In unix tradition, if you can't fix it, feature it.  We'll give the
installer multiple options.

In the end, what mostly matters is we get the distro packagers, and
the developers happy.  The rest usually know they have work to do
installing from source.

RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
	gem at rellim.com  Tel:+1 541 382 8588

	    Veritas liberabit vos. -- Quid est veritas?
    "If you can’t measure it, you can’t improve it." - Lord Kelvin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ntpsec.org/pipermail/devel/attachments/20171219/d59e2c79/attachment.bin>