BLD: Update to macos-11 for x86_64 wheels. by charris · Pull Request #22534 · numpy/numpy

charris · 2022-11-04T19:42:40Z

The tests were already updated, but the wheels remained at macos-10.15. This updates the wheel builds as well and should avoid the warning that 10.15 will be removed in December 2022.

charris · 2022-11-04T19:44:26Z

@mattip @h-vetinari Ping. Note that the openblas libraries are still labeled 10.9.

charris · 2022-11-04T21:30:50Z

There are Fortran problems. I assume something like

LIBRARY_PATH: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib

is needed here. @mattip ?

mattip · 2022-11-05T16:45:09Z

It seems something like that might be needed for the gfortran compiler.

rgommers · 2022-11-05T17:57:34Z

See https://github.com/scipy/scipy/blob/1d80a2f254380d2b45733258ca42eb6b55c8755b/.github/workflows/wheels.yml#L82-L87 for more details on gfortran. Cc @andyfaff, who discovered the same issue when updating SciPy wheel builds.

charris · 2022-11-05T18:13:26Z

Thanks Ralf. Makes me curious as to why macos-11 works on azure. In any case, I think we should put some effort into getting this working. Do you think macos-12 would be a better target? That seems to be what SciPy is using for the arm64 builds.

rgommers · 2022-11-05T18:19:14Z

Do you think macos-12 would be a better target? That seems to be what SciPy is using for the arm64 builds.

macOS 11 has a bug that leads to a kernel panic when used with some functionality in sparse.linalg, that's why we're avoiding shipping wheels for macOS 11. For NumPy it's better to stay on 11.

mattip · 2022-11-05T19:18:04Z

Over at #22525 I am trying to get OpenBLAS 0.3.21 working. In PR MacPython/openblas-libs#89 I built the openblas libraries and updated the macOS image to mac-11 without updating the gfortran toolchain on macOS.

andyfaff · 2022-11-06T05:53:10Z

Here are my notes from fiddling around today when trying to build macosx_x86_64 wheels on my scipy fork:

I changed the macOS image from macos-10.15 to macos-11. I also changed OpenBLAS to v0.3.21.
When I do that the gfortran-4.9.0-Mavericks compiler can no longer be used to compile programs - it just doesn't work on macos-11.
I then installed gfortran-11.2 for BigSur from fxcoudert instead of the ancient version of gfortran.
When I use this compiler I get lots of

  ld: warning: dylib (/usr/local/gfortran/lib/libgfortran.dylib) was built for newer macOS version (11.5) than being linked (10.9)
  ld: warning: dylib (/usr/local/gfortran/lib/libquadmath.dylib) was built for newer macOS version (11.5) than being linked (10.9)

If I try to set MACOSX_DEPLOYMENT_TARGET=10.15 I (obviously) still get the warnings. I therefore tried older compilers to see if I could go further back than 11.5.
I then tried using gfortran-8.2 and set MACOSX_DEPLOYMENT_TARGET=10.15. This time when I build I no longer get the ld warnings from step 5.
The build 'completes'. However, the delocate step fails:

/usr/local/gfortran/lib/libgfortran.3.dylib not found:
    Needed by: /usr/local/lib/libopenblas.0.dylib
ERROR:delocate.libsana:
  /usr/local/gfortran/lib/libgfortran.3.dylib not found:
    Needed by: /usr/local/lib/libopenblas.0.dylib

This is because OpenBLAS on anaconda is built against libgfortran.3.dylib, but gfortran-8 supplies libgfortran.5.dylib.
8. In order to get libgfortran.3.dylib you have to use gfortran <= 6.x. I didn't try gfortran-6.3 because it's quite old.

From discussions elsewhere it seems that there's a desire (at least in scipy) to update to at least gfortran-8. From my understanding of the above behaviour this would at least allow one to target 10.15. A deployment target of 10.9 doesn't seem sensible to me.

Perhaps we need to:

Make gfortran-8 the minimum compiler (libgfortran.5.dylib)
Update the deployment target to 10.15 across the board.
Update the OpenBLAS multibuild on macos to reflect 1+2.
On scipy can we bump to OpenBLAS v0.3.21?

Note: this only applies to the x86_64 builds, the arm64 gfortran compilers are much more modern.

@mattip, what gfortran compiler did you get to work on macos-11, that uses libgfortran.3.dylib (needed by the OpenBLAS on anaconda), that can also deploy against 10.9/10.14/10.15?

mattip · 2022-11-06T06:34:49Z

what gfortran compiler did you get to work

In #22525 I did not update the macos image at all.

mattip · 2022-11-06T06:37:18Z

Perhaps we need to: ...

These steps make sense to me. Would you like to submit a PR to MacPython/openblas-libs?

rgommers · 2022-11-06T06:51:35Z

+1 to bumping to at least gfortran 8.

Regarding minimum macOS version: 10.15 is a bit too aggressive; the problem is that users on <=10.14 will then get builds from source, rather than an older wheel. 10.14 is about 4 years old, and I expect it's still around enough to be a problem in practice. I also had a quick browse around, currently Matplotlib uses 10.12 and PyArrow 10.14, those are the newest versions I found. I'd expect that 10.12 or 10.13 would be okay, 10.14 borderline, and 10.15 too new.

andyfaff · 2022-11-06T07:34:51Z

These steps make sense to me. Would you like to submit a PR to MacPython/openblas-libs?

Normally yes, but it's not quite clear to me where to even start.

mattip · 2022-11-06T09:08:40Z

Ok, let me try.

I then tried using gfortran-8.2 ...

I think this should replace the one in https://github.com/MacPython/gfortran-install/tree/master/archives, and the README updated appropriately

mattip · 2022-11-06T09:37:08Z

xref MacPython/openblas-libs#90

charris · 2022-11-06T17:55:08Z

Will gfortran 8.2 for Mojave (macOS 10.14) actually install on macos 11?

mattip · 2022-11-06T17:58:22Z

There are now macos-10.11 builds of openblas 0.3.21 with the gfortran8 gfortran compiler on https://anaconda.org/multibuild-wheels-staging/openblas-libs/files. If we could easily update to 0.3.21, we could change the openblas suffix to 10_11 instead of 10_9. Unfortunately then using 0.3.21 will hit the windows build problems from #22525.

That compiler was built for macos 10.14, and there are ld warnings

ld: warning: dylib (/usr/local/gfortran/lib/libgfortran.dylib) was built for newer macOS version (10.14) than being linked (10.11)

Will that cause problems when targeting 10.11?

charris · 2022-11-06T18:36:20Z

@mattip Why 10.11?

mattip · 2022-11-06T20:46:08Z

Why 10.11

If we are not going with 10.14, then 10.11 seemed like a good minimum version.

h-vetinari · 2022-11-06T21:22:08Z

I think it would be a good time to retry the "repackage-gfortran-from-conda-forge" idea. Our gfortran is compatible all the way back to 10.9.

At the time it wasn't clear where to do the repackaging: MacPython/gfortran-install#7

CC @isuruf

andyfaff · 2022-11-06T21:25:06Z

Will gfortran 8.2 for Mojave (macOS 10.14) actually install on macos 11?

The installer package I linked to above is installable on the GH Actions macos-11 image (already tried that).

That compiler was built for macos 10.14, and there are ld warnings
ld: warning: dylib (/usr/local/gfortran/lib/libgfortran.dylib) was built for newer macOS version (10.14) than being linked (10.11)
Will that cause problems when targeting 10.11?

From what the warning says I would've thought so, perhaps the minimum macOS version we can target with that compiler becomes 10.14. I tried to search for the warning but get no clear guidance on whether binaries built with that would work on <10.14.

There may be compilers for gfortran-8 that were built for older macOS, but I can't find any immediately. This page has a gfortran-6 package for 10.11 and up.

isuruf · 2022-11-06T21:33:43Z

I can repackage one from conda-forge for you until we figure out MacPython/gfortran-install#7

Which gfortran do you need? 9.5.0 or 10.4.0 or 11.3.0? They'll be compatible with macos 10.9 and later

andyfaff · 2022-11-06T21:58:01Z

Which gfortran do you need?

I can't speak for the exact gfortran version we need from a scipy POV (@rgommers will know) but the gfortran needs:

be usable/installable on macos-11 (e.g. GH Actions image)
compatibility with 10.9 is great.
for native x86_64 compilation

scipy then needs an OpenBLAS build with the above (either v0.3.18 or v0.3.21).

(Sorry if the above is stating the obvious).

isuruf · 2022-11-06T22:52:24Z

@andyfaff, see MacPython/gfortran-install#11

charris · 2022-11-07T00:24:57Z

I note that gfortran-8.2-Mojave.dmg has the wrong file name inside, we want gfortran, it has gfortran-8.2-Mojave. The original 4.9 version we were using was modified so that the internal file was gfortran. With that change, I suspect we could solve some of our problems.

andyfaff · 2022-11-07T00:33:52Z

I note that gfortran-8.2-Mojave.dmg has the wrong file name inside

The file inside the DMG has a funny name, but is macOS installer PKG. Once the installer has run it places a gfortran executable in the right place with the right name.

This will not solve the openblas problems, but perhaps the f2py failures will go away.

charris · 2022-11-07T01:14:12Z

but is macOS installer PKG

It doesn't have the .pkg extension, does that matter?

andyfaff · 2022-11-07T01:18:56Z

Sorry, the installer is within that folder, /Volumes/gfortran-8.2-Mojave/gfortran-8.2-Mojave/gfortran.pkg

andyfaff · 2022-11-07T01:19:45Z

I'm going to look into the solution posted by @isuruf this morning for scipy.

charris · 2022-11-07T01:56:35Z

I'm going to look into the solution posted by @isuruf this morning for scipy.

The wheels build now, but fail during delocate because of OpenBLAS. @mattip What needs to be done to use the new OpenBLAS builds? Are they against gfortran 8.2?

EDIT: In particular, do they use libgfortran.5.dylib

andyfaff · 2022-11-07T02:31:28Z

There are a few intertwined factors here. We need to update to a newer version of gfortran because gfortran-4.9 doesn't work on macos-11. Whilst the gfortran-8.2 link gets through the compile stage ultimately this solution is not for us because:

gfortran-8.2 uses libgfortran.5.dylib and the currently available OpenBLAS on anaconda needs libgfortran.3.dylib. This is causing the delocate step to fail.
That gfortran-8.2 is designed to deploy on 10.14 and above. So even if OpenBLAS is rebuilt using that gfortran-8.2 we couldn't target macos-10.13 and below.

The good solution is probably to:

Install the gfortran-11.3 compiler suggested by @isuruf in https://github.com/MacPython/gfortran-install/pull/11/files. The compiler can deploy against 10.9.
Rebuild the OpenBLAS binaries using this gfortran-11.3.

isuruf · 2022-11-07T04:04:18Z

@charris, you can fix that by changing https://github.com/numpy/numpy/blob/main/tools/openblas_support.py#L80 to
suffix = 'macosx_10_11_x86_64-gf_dd3e148.tar.gz'

charris · 2022-11-07T16:39:56Z

Mac is working. This is all testing what needs changing right now. All we need is to

pick a compiler
compile OpenBLAS with it. Must work with windows
fix this up to use the new stuff.

andyfaff · 2022-11-08T04:06:55Z

tools/openblas_support.py

        typ = 'tar.gz'
    elif plat == 'macosx-x86_64':
-        suffix = 'macosx_10_9_x86_64-gf_1becaaa.tar.gz'
+        suffix = 'macosx_10_11_x86_64-gf_dd3e148.tar.gz'


I think this OpenBLAS change means that you have to set MACOSX_DEPLOYMENT_TARGET=10.11. One needs to be making numpy-1.24.0.dev0-cp39-cp39-macosx_10_11_x86_64.whl instead of numpy-1.24.0.dev0-cp39-cp39-macosx_10_9_x86_64.whl.
Using https://github.com/MacPython/gfortran-install/pull/11/files may still allow macosx_10_9_x86_64-gf_1becaaa.tar.gz to be used, but that would require experimentation.

charris · 2022-11-17T14:08:02Z

Closing in favor of #22525.

mattip · 2022-11-17T16:00:07Z

.github/workflows/wheels.yml

        buildplat:
          - [ubuntu-20.04, manylinux_x86_64]
-          - [macos-10.15, macosx_*]
+          - [macos-11, macosx_*]


This still needs to be done in #22525. Should I add it there?

Heh, too late. I'll just make a PR for that.

BLD: Update to macos-11 for x86_64 wheels.

84fd9cd

The tests were already updated, but the wheels remained at macos-10.15. This updates the wheel builds as well and should avoid the warning that 10.15 will be removed in December 2022.

charris added the 36 - Build Build related PR label Nov 4, 2022

charris added this to the 1.24.0 release milestone Nov 4, 2022

WIP: Use gfortran-8.2

8aa8bb6

This will not solve the openblas problems, but perhaps the f2py failures will go away.

charris force-pushed the update-to-macos-11-for-wheels branch from c3f2d63 to 8aa8bb6 Compare November 7, 2022 01:00

wip

a56396f

wip

6f65565

wip

ada0e0e

wip

586aad6

andyfaff reviewed Nov 8, 2022

View reviewed changes

andyfaff mentioned this pull request Nov 8, 2022

WHL: bump macos image, OpenBLAS version, gfortran-11, MACOSX_DEPLOYMENT_TARGET=10.11 scipy/scipy#17370

Closed

charris mentioned this pull request Nov 15, 2022

BLD: update OpenBLAS to 0.3.21 and clean up openblas download test #22525

Merged

charris closed this Nov 17, 2022

charris deleted the update-to-macos-11-for-wheels branch November 17, 2022 14:10

mattip reviewed Nov 17, 2022

View reviewed changes

mattip mentioned this pull request Nov 17, 2022

BLD: update wheel builds on macos to macos-12 image #22612

Merged

Uh oh!

Conversation

charris commented Nov 4, 2022

Uh oh!

charris commented Nov 4, 2022

Uh oh!

charris commented Nov 4, 2022

Uh oh!

mattip commented Nov 5, 2022

Uh oh!

rgommers commented Nov 5, 2022

Uh oh!

charris commented Nov 5, 2022

Uh oh!

rgommers commented Nov 5, 2022

Uh oh!

mattip commented Nov 5, 2022

Uh oh!

andyfaff commented Nov 6, 2022

Uh oh!

mattip commented Nov 6, 2022

Uh oh!

mattip commented Nov 6, 2022

Uh oh!

rgommers commented Nov 6, 2022

Uh oh!

andyfaff commented Nov 6, 2022

Uh oh!

mattip commented Nov 6, 2022

Uh oh!

mattip commented Nov 6, 2022

Uh oh!

charris commented Nov 6, 2022

Uh oh!

mattip commented Nov 6, 2022

Uh oh!

charris commented Nov 6, 2022

Uh oh!

mattip commented Nov 6, 2022

Uh oh!

h-vetinari commented Nov 6, 2022

Uh oh!

andyfaff commented Nov 6, 2022

Uh oh!

isuruf commented Nov 6, 2022

Uh oh!

andyfaff commented Nov 6, 2022

Uh oh!

isuruf commented Nov 6, 2022

Uh oh!

charris commented Nov 7, 2022

Uh oh!

andyfaff commented Nov 7, 2022

Uh oh!

charris commented Nov 7, 2022

Uh oh!

andyfaff commented Nov 7, 2022

Uh oh!

andyfaff commented Nov 7, 2022

Uh oh!

charris commented Nov 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andyfaff commented Nov 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isuruf commented Nov 7, 2022

Uh oh!

charris commented Nov 7, 2022

Uh oh!

andyfaff Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

charris commented Nov 17, 2022

Uh oh!

mattip Nov 17, 2022

Choose a reason for hiding this comment

Uh oh!

charris Nov 17, 2022

Choose a reason for hiding this comment

charris commented Nov 7, 2022 •

edited

Loading

andyfaff commented Nov 7, 2022 •

edited

Loading