Re: [PLUG] Fixing a Python library mismatch

[brent timothy saner via plug <plug@lists.phillylinux.org>]

On 5/16/20 07:59, Walt Mankowski via plug wrote:
> On Fri, May 15, 2020 at 09:26:30PM -0400, brent timothy saner via plug wrote:
>> three ways to fix:
>>
>> ONE: the proper, stable way
>>
>> use numpy and python 3 from the ubuntu repositories. this ensures the
>> modules and system libraries are always in sync (unless a packager
>> somewhere made an oopsie).
> 
> Sure, but of course I want to have the latest and greatest packages,
> and also follow new Python versions. :)

hrm. a compromise on this might be using a PPA such as
https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa for python and pip
(if you prefer venvs).

> 
>> TWO A: the "i don't like the proper, stable way" way
>> a.k.a. the "i know better than the distro packagers" way
>>
>> just recreate your env. i'd recommend using venv rather than pyenv,
>> though, since it's stdlib in python3.3+
>> (https://docs.python.org/3/library/venv.html). since it looks like
>> you're using python3.8, and assuming it's installed system-wide,
>>
>> python3.8 -m venv some_venv_dir
> 
> Thanks, I'll take a look at venv. I have a hunch this is a more of a
> problem with pip though. See below.

yep! i bring it up because the pyvenv script was deprecated in 3.6:
https://docs.python.org/dev/whatsnew/3.6.html#deprecated-python-modules-functions-and-methods

as noted in your second reply, though, interestingly noted is two
deprecations below in that very link is the --with-system-ffi build flag
deprecation as 3.6 onwards it's the default if you're building python
from source, and they force using the system libffi for 3.7 onwards.
technically a coincidence, but worth noting since it's marginally relevant.

> 
>> TWO B: the "fuck that" way
>>
>> uninstall numpy from your pyenv and reinstall it. as long as it doesn't
>> keep a build cache, it should build from whatever libffi is available.
> 
> I've already tried that numerous times. Also tried
> 
> $ pip cache purge
> $ pip install numpy --no-cache-dir
> 
> but I'm still having the same problem.

yeah, as found in your later reply it looks like it was within python
itself. i figured it would have been numpy since it's a sizable module,
but alas.

> 
>> THREE: the honeybadger way
>> a.k.a. the yolo way
>>
>> this is a hack, but it works 9 times out of 10.
>>
>> just symlink the lib.
>>
>> cd /usr/lib64[0]
>> ln -s libffi.so[1] libffi.so.6
> 
> Noted, but this seems like the sort of thing that is likely to happen
> again and I'd like to know how to properly fix it.
> 
> The reason I suspect pip is that then I install big packages in Perl
> it takes several minutes to compile the code and run all the
> tests. But when I install packages with pip they all install in a few
> seconds. This leads me to believe it's downloading a pre-compiled
> version of numpy.
> 
yeah, pip has something called wheels in newer python releases:

https://pip.pypa.io/en/stable/reference/pip_wheel/
https://pythonwheels.com/

not all packages have (or can have) wheels (for instance, the
python-ldap module can't use wheels because it needs to use the system
openldap libs/headers), but they definitely speed up the install process
since they don't require compilation.

you can tell if a module does a wheel install or not by going to its
pypi "Downlod files" page and looking for a wheel there.
e.g. https://pypi.org/project/python-ldap/#files has no wheel files,
https://pypi.org/project/numpy/#files does ("*.whl").

you can explicitly disable the use of wheels in pip via the following:

pip < 10.x:

<pip install> --no-use-wheel <package>

pip >= 10.x:

<pip install> --no-binary <package>

Attachment: signature.asc
Description: OpenPGP digital signature

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug