Keep 32-bit build support on Mojave

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Keep 32-bit build support on Mojave

Ryan Schmidt-24
The release of macOS Mojave is imminent and I want to make sure we do the right thing for the first release of MacPorts on Mojave. Let me know if you have any thoughts about the below.

Ideally I would like to keep the ability to build 32-bit ports in MacPorts on Mojave, mainly because I don't want my wine ports to stop working. The checks that currently set universal_archs to only x86_64 on 10.14 would be bumped to 10.15.

The 10.13 (or earlier) SDK would have to be used when building 32-bit. I have a port for installing SDKs which I will finish up and submit as a PR. Here is the preliminary port:

https://github.com/ryandesign/macports-ports/blob/MacOSX.sdk/devel/MacOSX.sdk/Portfile

I would then like to modify base so that if configure.sdk_version is set to a version that is not provided by Xcode or the command line tools, it will use one provided by a subport of the MacOSX.sdk port instead of generating an error as it currently does. And also, if we are building for 32-bit on Mojave, default to the 10.13 SDK. Here is a preliminary version of those changes:

https://github.com/ryandesign/macports-base/commits/MacOSX.sdk

This should allow those few 32-bit-only ports, and wine which requires a universal build, to continue to be usable on Mojave. But I have not tested this on Mojave yet.

This should also allow us to fix build failures of a variety of ports on older systems that need newer SDKs, such as textmate2.

This will give us the year until 10.15 is released to figure out the solution to disabling the universal variant when it won't work (https://trac.macports.org/ticket/57133).

Either now for 10.14 if the above is unworkable, or at the latest for 10.15, we still have the problem that upgrading users will have macports.conf files that say "universal_archs x86_64 i386" but new installations will say "universal_archs x86_64". Most of the settings that we write into macports.conf are commented out so that defaults take effect, but for some reason universal_archs is one of the few settings where we don't comment it out. If we do nothing in base to handle this, we'll have to write a ProblemHotlist entry and probably constantly guide users through making this change.

There is also the problem that the --with-universal-archs MacPorts configure flag affects not only the universal_archs macports.conf setting but also the archs used to build the darwintrace part of MacPorts base. On Mojave I think we want to be able to build MacPorts base x86_64 only, but have universal_archs in macports.conf be x86_64 i386. Building base universal x86_64 i386 is possible on Mojave using the 10.13 SDK which we could manage when building the binary installers, but it's not going to work for users when they run selfupdate and the 10.13 SDK isn't there.

Reply | Threaded
Open this post in threaded view
|

Re: Keep 32-bit build support on Mojave

mf2k
Hi Ryan,


> On Sep 24, 2018, at 2:22 AM, Ryan Schmidt <[hidden email]> wrote:
>
> The release of macOS Mojave is imminent and I want to make sure we do the right thing for the first release of MacPorts on Mojave. Let me know if you have any thoughts about the below.
>
> Ideally I would like to keep the ability to build 32-bit ports in MacPorts on Mojave, mainly because I don't want my wine ports to stop working. The checks that currently set universal_archs to only x86_64 on 10.14 would be bumped to 10.15.
>
> The 10.13 (or earlier) SDK would have to be used when building 32-bit. I have a port for installing SDKs which I will finish up and submit as a PR. Here is the preliminary port:
>
> https://github.com/ryandesign/macports-ports/blob/MacOSX.sdk/devel/MacOSX.sdk/Portfile
>
> I would then like to modify base so that if configure.sdk_version is set to a version that is not provided by Xcode or the command line tools, it will use one provided by a subport of the MacOSX.sdk port instead of generating an error as it currently does. And also, if we are building for 32-bit on Mojave, default to the 10.13 SDK. Here is a preliminary version of those changes:
>
> https://github.com/ryandesign/macports-base/commits/MacOSX.sdk
>
> This should allow those few 32-bit-only ports, and wine which requires a universal build, to continue to be usable on Mojave. But I have not tested this on Mojave yet.
>
> This should also allow us to fix build failures of a variety of ports on older systems that need newer SDKs, such as textmate2.
>
> This will give us the year until 10.15 is released to figure out the solution to disabling the universal variant when it won't work (https://trac.macports.org/ticket/57133).
>
> Either now for 10.14 if the above is unworkable, or at the latest for 10.15, we still have the problem that upgrading users will have macports.conf files that say "universal_archs x86_64 i386" but new installations will say "universal_archs x86_64". Most of the settings that we write into macports.conf are commented out so that defaults take effect, but for some reason universal_archs is one of the few settings where we don't comment it out. If we do nothing in base to handle this, we'll have to write a ProblemHotlist entry and probably constantly guide users through making this change.
>
> There is also the problem that the --with-universal-archs MacPorts configure flag affects not only the universal_archs macports.conf setting but also the archs used to build the darwintrace part of MacPorts base. On Mojave I think we want to be able to build MacPorts base x86_64 only, but have universal_archs in macports.conf be x86_64 i386. Building base universal x86_64 i386 is possible on Mojave using the 10.13 SDK which we could manage when building the binary installers, but it's not going to work for users when they run selfupdate and the 10.13 SDK isn't there.

This looks great! Thank you for working on this issue. I look forward to trying it on the release of Mojave soon.

I am curious about the solution that the Crossover project has in mind for wine on 64-bit.


Cheers!
Frank

Reply | Threaded
Open this post in threaded view
|

Re: Keep 32-bit build support on Mojave

Joshua Root-8
In reply to this post by Ryan Schmidt-24
On 2018-9-24 18:22 , Ryan Schmidt wrote:
> https://github.com/ryandesign/macports-base/commits/MacOSX.sdk

Looks mostly reasonable, but there are a couple of issues.

First, configure_get_sdkroot is the wrong place to register a callback,
since it can easily be called multiple times. The callback should be
registered once, unconditionally, and should add the dependency only if
the SDK doesn't exist (or if it's an SDK that normally wouldn't exist?
There is still a dependency on the SDK port even if it's already installed).

Second, I'm not sure about changing the SDK only some of the time, or
not changing the deployment target. We've always recommended changing
the deployment target for an entire installation globally if it's going
to be changed, and Apple only supports using a deployment target <= the
SDK being used. If it was only the i386 slices that were built against a
different SDK, that would be different, but the x86_64 slices of
universal builds will also be built against a different SDK than
non-universal x86_64 ports. A non-universal port built against the 10.14
SDK could thus end up linked with a universal port built against the
10.13 SDK.

I'm not sure how much of a problem all that is. It's unknown territory.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Keep 32-bit build support on Mojave

Ken Cunningham

On 2018-10-01, at 3:50 PM, Joshua Root wrote:

> On 2018-9-24 18:22 , Ryan Schmidt wrote:
>> https://github.com/ryandesign/macports-base/commits/MacOSX.sdk
>

> Second, I'm not sure about changing the SDK only some of the time, or
> not changing the deployment target.

> I'm not sure how much of a problem all that is. It's unknown territory.
>
> - Josh

I would think if we decide to try this we should also set the deployment target to 10.13 when we're building against the 10.13 SDK, and build the whole port (32 and 64bit) with the same settings for SDK and deployment target.

It seems to me that should give us a good chance at a reliable situation. Apple does support building on an older system and deploying on a newer system, and in effect, that's what we would be mimicking.

My only worry is some of the SDK in "/" sneaking in somehow to complicate things, so to date, I haven't installed the command-line tools on Mojave. So far, no ports have complained about being forced to use the SDK only.

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Keep 32-bit build support on Mojave

Joshua Root-8
In reply to this post by Joshua Root-8
On 2018-10-2 08:50 , Joshua Root wrote:

> Second, I'm not sure about changing the SDK only some of the time, or
> not changing the deployment target. We've always recommended changing
> the deployment target for an entire installation globally if it's going
> to be changed, and Apple only supports using a deployment target <= the
> SDK being used. If it was only the i386 slices that were built against a
> different SDK, that would be different, but the x86_64 slices of
> universal builds will also be built against a different SDK than
> non-universal x86_64 ports. A non-universal port built against the 10.14
> SDK could thus end up linked with a universal port built against the
> 10.13 SDK.

What if instead of using the 10.13 SDK, we create a port for a universal
10.14 SDK? The only thing preventing the 10.14 SDK from being used for
universal builds is the .tbd files, which list only x86_64 for the
architectures of all the libs. They are YAML I believe, or certainly
some form of plain text, and so can easily be edited.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Keep 32-bit build support on Mojave

Rainer Müller-4
On 2018-10-24 23:53, Joshua Root wrote:

> On 2018-10-2 08:50 , Joshua Root wrote:
>> Second, I'm not sure about changing the SDK only some of the time, or
>> not changing the deployment target. We've always recommended changing
>> the deployment target for an entire installation globally if it's going
>> to be changed, and Apple only supports using a deployment target <= the
>> SDK being used. If it was only the i386 slices that were built against a
>> different SDK, that would be different, but the x86_64 slices of
>> universal builds will also be built against a different SDK than
>> non-universal x86_64 ports. A non-universal port built against the 10.14
>> SDK could thus end up linked with a universal port built against the
>> 10.13 SDK.
>
> What if instead of using the 10.13 SDK, we create a port for a universal
> 10.14 SDK? The only thing preventing the 10.14 SDK from being used for
> universal builds is the .tbd files, which list only x86_64 for the
> architectures of all the libs. They are YAML I believe, or certainly
> some form of plain text, and so can easily be edited.

If I remember correctly, the way it works on older macOS versions is
that ld is invoked with a -syslibroot argument pointing to the SDK,
which causes ld to search in that path, where it finds the .tbd, before
even looking in the regular paths for .dylib files.

Is this still the case on 10.14 or is this now hardcoded in ld itself?

Something like this to get things started:

  echo '#!/bin/sh'$'\n''echo "$@"'$'\n''exit 1' > ld
  chmod +x ld
  clang -fuse-ld=$PWD/ld -o foo foo.c

Maybe we could use a wrapper for ld that ignores that argument or
replace all -lfoo references with absolute paths to libfoo.dylib?

Rainer