Some questions regarding MacPorts legacy support package

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Some questions regarding MacPorts legacy support package

Jason Liu
I've created a wrapper file for AppKit (see attached file), which is eventually destined to be added to the MacPorts legacy support package. I wanted to have the dev mailing list take a look at it and maybe provide me with some initial feedback before I start actually working on adding it to GitHub. In addition, I have some questions regarding the MacPorts legacy support package.

The list of #defines in the wrapper should be fairly comprehensive, at least for the changes that Apple made when they released 10.12. I have tried my best to match the style of the other wrapper files in the legacy support package. Also, I put a description inside the file. It's pretty lengthy, but I didn't know where else to put it. It didn't seem appropriate for either the MacPorts guide or the legacy support package's README file; but I also didn't want to keep all that info sitting in my own personal notes.

Question 1:

I've noticed that in MacportsLegacySupport.h and other wrapper files, __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ is used for macOS version detection. I also noticed this line in MacportsLegacySupport.h:

/* Not needed -- #include "AvailabilityMacros.h" */

Would it be considered kosher to use any of the other version detection constants from AvailabilityMacros.h, such as MAC_OS_X_VERSION_MAX_ALLOWED? Or would that be considered risky/dangerous/undesirable for some reason?

Question 2:

A related question is that in MacportsLegacySupport.h, you guys use version numbers such as 101300, 1070, etc. instead of the constants that are defined in AvailabilityMacros.h, such as MAC_OS_X_VERSION_10_13, MAC_OS_X_VERSION_10_7, etc. Is there any particular reason for not using the constants and going with the actual integer numbers? It looks like not even Apple's own source code is consistent with this. In AvailabilityMacros.h, they use the version number constants MAC_OS_X_VERSION_10_*, but in other header files like assert.h, pthread.h, etc., they use the raw integers, i.e. 1070.

Question 3:

As you can see from the attached file, I am currently creating a wrapper for AppKit.h. However, in the projects that I'm trying to package for MacPorts, their code usually uses '#include <Cocoa/Cocoa.h>'; and the system Cocoa.h, in turn, has a '#import <AppKit/AppKit.h>'. Is this going to be a problem? Is the MacPorts legacy support package able to intervene and insert its wrapper files even if a project's source code doesn't directly #include/#import that specific header file, but instead, the header to be patched is nested somewhere inside a tree/chain of header #includes?

-- 
Jason Liu

AppKit.h.zip (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Some questions regarding MacPorts legacy support package

Ken Cunningham

Some questions regarding MacPorts legacy support package

Jason Liu [hidden email]
Sat Jul 4 20:44:59 UTC 2020


I've created a wrapper file for AppKit (see attached file), which is
eventually destined to be added to the MacPorts legacy support package. I
wanted to have the dev mailing list take a look at it and maybe provide me
with some initial feedback before I start actually working on adding it to
GitHub. In addition, I have some questions regarding the MacPorts legacy
support package.

The list of #defines in the wrapper should be fairly comprehensive, at
least for the changes that Apple made when they released 10.12. I have
tried my best to match the style of the other wrapper files in the legacy
support package. Also, I put a description inside the file. It's pretty
lengthy, but I didn't know where else to put it. It didn't seem appropriate
for either the MacPorts guide or the legacy support package's README file;
but I also didn't want to keep all that info sitting in my own personal
notes.

Question 1:

I've noticed that in MacportsLegacySupport.h and other wrapper files,
__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ is used for macOS version
detection. I also noticed this line in MacportsLegacySupport.h:

/* Not needed -- #include "AvailabilityMacros.h" */

Would it be considered kosher to use any of the other version detection
constants from AvailabilityMacros.h, such as MAC_OS_X_VERSION_MAX_ALLOWED?
Or would that be considered risky/dangerous/undesirable for some reason?


We have tried to make this (so far) such that this was not needed, and just go with the compiler default “__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__” which is set to a number by every compiler on Apple/Darwin systems that matches the deployment target the compiler sees.




Question 2:

A related question is that in MacportsLegacySupport.h, you guys use version
numbers such as 101300, 1070, etc. instead of the constants that are
defined in AvailabilityMacros.h, such as MAC_OS_X_VERSION_10_13,
MAC_OS_X_VERSION_10_7, etc. Is there any particular reason for not using
the constants and going with the actual integer numbers? It looks like not
even Apple's own source code is consistent with this. In
AvailabilityMacros.h, they use the version number constants
MAC_OS_X_VERSION_10_*, but in other header files like assert.h, pthread.h,
etc., they use the raw integers, i.e. 1070.


We are sticking with the numbers. See this <https://trac.macports.org/wiki/LeopardSDKFixes#Incorrect__MAC_OS_X_VERSION_MAX_ALLOWED> for the initial inspiration for that, but it just avoids a lot of confusion about which constants are available in which files and when.

So — numbers.



Question 3:

As you can see from the attached file, I am currently creating a wrapper
for AppKit.h. However, in the projects that I'm trying to package for
MacPorts, their code usually uses '#include <Cocoa/Cocoa.h>'; and the
system Cocoa.h, in turn, has a '#import <AppKit/AppKit.h>'. Is this going
to be a problem? Is the MacPorts legacy support package able to intervene
and insert its wrapper files even if a project's source code doesn't
directly #include/#import that specific header file, but instead, the
header to be patched is nested somewhere inside a tree/chain of header
#includes?


This will be — something new. Nobody actually knows how well this will work.

It probably might best be something optionally used rather than a default in legacysupport, as the opportunity for unexpected wreckage seems high. On the other hand, after year or so, if it helps but doesn’t break things, it might be defaultable.

If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

There are still many opinions that it should not be used, and the issues / fixes sent upstream instead, but I think we’re all coming to realize that something like legacysupprt is the only way forward if we’re going to support older systems.

GCC uses “fixincludes” to do much the same thing for their compiler.

Thank you for your interest and help!

Ken





Reply | Threaded
Open this post in threaded view
|

Re: several messages

Fred Wright

On Wed, 3 Jun 2020, Jason Liu wrote:

[...]
> A solution I found which some projects (e.g. Qemu) have implemented
> basically replaces the new AppKit constants with the old AppKit ones using
> *#define* directives if the OS version is below 10.12. I've created a
> separate header file that gathers together a list of the constants I've
> been able to find, which is modeled on information from this message:
[...]

It's interesting that you cite Qemu as an example of this, when the Qemu
port is currently failing to build on 10.9 for reasons that look very
similar to this issue.

On Sat, 4 Jul 2020, Ken Cunningham wrote:
>> Some questions regarding MacPorts legacy support package
>>
>> Jason Liu jasonliu at umich.edu  <mailto:macports-dev%40lists.macports.org?Subject=Re%3A%20Some%20questions%20regarding%20MacPorts%20legacy%20support%20package&In-Reply-To=%3CCAHUrRf4a%2BvaF8YLE5eAh2Fo5g2ZaZTK%2BW6ecg%3DTj7tXtgx5vvg%40mail.gmail.com%3E>
>> Sat Jul 4 20:44:59 UTC 2020
[...]

>> Question 1:
>>
>> I've noticed that in MacportsLegacySupport.h and other wrapper files,
>> __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ is used for macOS version
>> detection. I also noticed this line in MacportsLegacySupport.h:
>>
>> /* Not needed -- #include "AvailabilityMacros.h" */
>>
>> Would it be considered kosher to use any of the other version detection
>> constants from AvailabilityMacros.h, such as MAC_OS_X_VERSION_MAX_ALLOWED?
>> Or would that be considered risky/dangerous/undesirable for some reason?
>
> We have tried to make this (so far) such that this was not needed, and
> just go with the compiler default
> “__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__” which is set to a number
> by every compiler on Apple/Darwin systems that matches the deployment
> target the compiler sees.
I'm not sure where this "__ENVIRONMENT" prefix is coming from.  Is that a
version of the definitions created by legacy-support?

Normally, there are four macros of this form available:

__MAC_OS_X_VERSION_MIN_REQUIRED
__MAC_OS_X_VERSION_MAX_ALLOWED
MAC_OS_X_VERSION_MIN_REQUIRED
MAC_OS_X_VERSION_MAX_ALLOWED

The general idea is that MIN_REQUIRED is for features that weren't
available before a certain OS version, and MAX_ALLOWED is for features
that were *removed* after a certain OS version.  Since OS versions mostly
try to be backward compatible with older versions, the first one is useful
far more often than the second.  As a general rule, if you're not sure
which one you want, then you almost certainly want MIN_REQUIRED.

Since having those defined differently would only be expected in cases
where the build target is a range of OS versions, and since MacPorts has
no concept of "OS version universality" and hence always targets one
specific OS version, once would expect the two definitions to be the same
in the MacPorts environment.  But aside from it being cleaner to use the
correct macro for the context, there's at least one case where they're
*not* the same.  From the default compiler on 10.5.8:

MacMini:OSX fw$ ./conftest
__APPLE__ = 1
__MAC_OS_X_VERSION_MIN_REQUIRED = 1058
__MAC_OS_X_VERSION_MAX_ALLOWED = 1060
_POSIX_VERSION = 200112

Thus, using MAX_ALLOWED to distinguish 10.5 from 10.6 doesn't work.  There
are several ports that don't build on 10.5 for precisely this reason.

As far as the double-undersore versus non-double-underscore aspect is
concerned, the double-underscore versions are usually defined by the
compiler itself (with no includes), but sometimes not, in which case it's
necessary to include Availability.h (but *not* necessarily
AvailabilityMacros.h).  The non-double-underscore versions are defined in
AvailabilityMacros.h.  Hence, any use of those demands including
AvailabilityMacros.h.

If anyone can explain why there's ever a reason to use the
non-double-underscore versions, I'd like to hear it.  Otherwise, doing so
is simply adding a dependency on an include that might not otherwise be
needed, and the double-underscore versions are preferable.

This is also something which some ports get wrong and have broken builds
or missing features as a consequence.

>> Question 2:
>>
>> A related question is that in MacportsLegacySupport.h, you guys use version
>> numbers such as 101300, 1070, etc. instead of the constants that are
>> defined in AvailabilityMacros.h, such as MAC_OS_X_VERSION_10_13,
>> MAC_OS_X_VERSION_10_7, etc. Is there any particular reason for not using
>> the constants and going with the actual integer numbers? It looks like not
>> even Apple's own source code is consistent with this. In
>> AvailabilityMacros.h, they use the version number constants
>> MAC_OS_X_VERSION_10_*, but in other header files like assert.h, pthread.h,
>> etc., they use the raw integers, i.e. 1070.
>>
>
> We are sticking with the numbers. See this
> <https://trac.macports.org/wiki/LeopardSDKFixes#Incorrect__MAC_OS_X_VERSION_MAX_ALLOWED>
> for the initial inspiration for that, but it just avoids a lot of
> confusion about which constants are available in which files and when.
Yes, the basic problem is that older includes don't have the newer
definitions.  So in general, if you want to support a wide range of OS
versions, it's best to stick to the numeric constants.

>> Question 3:
>>
>> As you can see from the attached file, I am currently creating a wrapper
>> for AppKit.h. However, in the projects that I'm trying to package for
>> MacPorts, their code usually uses '#include <Cocoa/Cocoa.h>'; and the
>> system Cocoa.h, in turn, has a '#import <AppKit/AppKit.h>'. Is this going
>> to be a problem? Is the MacPorts legacy support package able to intervene
>> and insert its wrapper files even if a project's source code doesn't
>> directly #include/#import that specific header file, but instead, the
>> header to be patched is nested somewhere inside a tree/chain of header
>> #includes?
>>
>
> This will be — something new. Nobody actually knows how well this will work.
It's also worth noting that this particular kind of fix probably only
needs include additions and no library additions, which is something the
PortGroup may not be set up to handle.  Of course some ports may need the
added library for *other* reasons, and that needs to be handled correctly.

> It probably might best be something optionally used rather than a
> default in legacysupport, as the opportunity for unexpected wreckage
> seems high. On the other hand, after year or so, if it helps but doesn’t
> break things, it might be defaultable.
>
> If a year or so sounds long, don’t worry. I wrote up the first version
> of legacysupport as “SnowLeopardFixes” in 2016, and it did not get
> really adopted until several years later, after a great deal of
> discussion.
>
> There are still many opinions that it should not be used, and the issues
> / fixes sent upstream instead, but I think we’re all coming to realize
> that something like legacysupprt is the only way forward if we’re going
> to support older systems.
In general, direct upstream support seems preferable when possible, but
some upstream developers refuse to support old OS versions, sometimes
using "security" as a lame justification.

Fred Wright
Reply | Threaded
Open this post in threaded view
|

Re: several messages

Ken Cunningham

several messages

Fred Wright [hidden email]
Sun Jul 5 04:08:09 UTC 2020



I'm not sure where this "__ENVIRONMENT" prefix is coming from.  Is that a 
version of the definitions created by legacy-support?

every compiler on darwin forever defines it

% clang  -dM -E - < /dev/null | grep ENV
#define __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ 101500
Reply | Threaded
Open this post in threaded view
|

Re: several messages

Ken Cunningham
In reply to this post by Fred Wright
oh, btw Fred, your impression of MAC_OS_X_VERSION_MAX_ALLOWED is not quite right.

It doesn't indicate removed features. It indicates the SDK version you are currently building with, so you know what API you can use.

This is always confusing, which is why we use what we do in legacysupport, in part.

Ask if you have any questions about it still.

K
Reply | Threaded
Open this post in threaded view
|

Re: several messages

Joshua Root-8
In reply to this post by Fred Wright
On 2020-7-5 14:08 , Fred Wright wrote:

>
> Since having those defined differently would only be expected in cases
> where the build target is a range of OS versions, and since MacPorts has
> no concept of "OS version universality" and hence always targets one
> specific OS version, once would expect the two definitions to be the
> same in the MacPorts environment.  But aside from it being cleaner to
> use the correct macro for the context, there's at least one case where
> they're *not* the same.  From the default compiler on 10.5.8:
>
> MacMini:OSX fw$ ./conftest
> __APPLE__ = 1
> __MAC_OS_X_VERSION_MIN_REQUIRED = 1058
> __MAC_OS_X_VERSION_MAX_ALLOWED = 1060
> _POSIX_VERSION = 200112
>
> Thus, using MAX_ALLOWED to distinguish 10.5 from 10.6 doesn't work. 
> There are several ports that don't build on 10.5 for precisely this reason.

The reason for this, by the way, is that this toolchain is aware that
10.6 exists and it is always assumed by default that you want access to
the latest features. Any symbols that are present in the max_allowed OS
but not in the min_required are weak-linked, and programs are expected
to check that those symbols are non-null before using them. That is the
Apple way of being compatible with multiple OS versions in a single
binary, but naturally it almost never happens in the open source world.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Some questions regarding MacPorts legacy support package

ryandesign2
Administrator
In reply to this post by Jason Liu


On Jul 4, 2020, at 15:44, Jason Liu wrote:

> Question 2:
>
> A related question is that in MacportsLegacySupport.h, you guys use version numbers such as 101300, 1070, etc. instead of the constants that are defined in AvailabilityMacros.h, such as MAC_OS_X_VERSION_10_13, MAC_OS_X_VERSION_10_7, etc. Is there any particular reason for not using the constants and going with the actual integer numbers? It looks like not even Apple's own source code is consistent with this. In AvailabilityMacros.h, they use the version number constants MAC_OS_X_VERSION_10_*, but in other header files like assert.h, pthread.h, etc., they use the raw integers, i.e. 1070.

Using the numbers is preferred. Once upon a time there was an Apple document stating that. Numbers will always work. The constants will only work if they are defined. MAC_OS_X_VERSION_10_13 for example is only defined in the 10.13 SDK and later, and if you reference an undefined constant, the preprocessor won't necessarily complain but will evaluate it to 0, which is probably not what you want.

Some projects use the constants and then have a central header where they check each constant to make sure it's defined and otherwise they define it themselves. This seems like a lot of work for no particular benefit to me. Just use the numbers. They're documented, they're clear, they're not changing.
Reply | Threaded
Open this post in threaded view
|

Re: Some questions regarding MacPorts legacy support package

Jason Liu
In reply to this post by Ken Cunningham
This will be — something new. Nobody actually knows how well this will work.

Then I guess we're all going to find out the answer in the course of adding this new piece, huh? :P
 
It probably might best be something optionally used rather than a default in legacysupport, as the opportunity for unexpected wreckage seems high. On the other hand, after year or so, if it helps but doesn’t break things, it might be defaultable.

Other than using the __MACPORTS_LEGACY_SUPPORT_APPKIT__ macro, is there some way of making it optional?

If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now? I'd really like to try to get my Blender port into the tree sooner rather than later. My portfile is written for Blender 2.82a, which was the version that was current at the start of the pandemic when I started working on the port, but since then Blender has already moved on to the 2.83 series of releases.

-- 
Jason Liu


On Sat, Jul 4, 2020 at 8:23 PM Ken Cunningham <[hidden email]> wrote:

Some questions regarding MacPorts legacy support package

Jason Liu [hidden email]
Sat Jul 4 20:44:59 UTC 2020


I've created a wrapper file for AppKit (see attached file), which is
eventually destined to be added to the MacPorts legacy support package. I
wanted to have the dev mailing list take a look at it and maybe provide me
with some initial feedback before I start actually working on adding it to
GitHub. In addition, I have some questions regarding the MacPorts legacy
support package.

The list of #defines in the wrapper should be fairly comprehensive, at
least for the changes that Apple made when they released 10.12. I have
tried my best to match the style of the other wrapper files in the legacy
support package. Also, I put a description inside the file. It's pretty
lengthy, but I didn't know where else to put it. It didn't seem appropriate
for either the MacPorts guide or the legacy support package's README file;
but I also didn't want to keep all that info sitting in my own personal
notes.

Question 1:

I've noticed that in MacportsLegacySupport.h and other wrapper files,
__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ is used for macOS version
detection. I also noticed this line in MacportsLegacySupport.h:

/* Not needed -- #include "AvailabilityMacros.h" */

Would it be considered kosher to use any of the other version detection
constants from AvailabilityMacros.h, such as MAC_OS_X_VERSION_MAX_ALLOWED?
Or would that be considered risky/dangerous/undesirable for some reason?


We have tried to make this (so far) such that this was not needed, and just go with the compiler default “__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__” which is set to a number by every compiler on Apple/Darwin systems that matches the deployment target the compiler sees.




Question 2:

A related question is that in MacportsLegacySupport.h, you guys use version
numbers such as 101300, 1070, etc. instead of the constants that are
defined in AvailabilityMacros.h, such as MAC_OS_X_VERSION_10_13,
MAC_OS_X_VERSION_10_7, etc. Is there any particular reason for not using
the constants and going with the actual integer numbers? It looks like not
even Apple's own source code is consistent with this. In
AvailabilityMacros.h, they use the version number constants
MAC_OS_X_VERSION_10_*, but in other header files like assert.h, pthread.h,
etc., they use the raw integers, i.e. 1070.


We are sticking with the numbers. See this <https://trac.macports.org/wiki/LeopardSDKFixes#Incorrect__MAC_OS_X_VERSION_MAX_ALLOWED> for the initial inspiration for that, but it just avoids a lot of confusion about which constants are available in which files and when.

So — numbers.



Question 3:

As you can see from the attached file, I am currently creating a wrapper
for AppKit.h. However, in the projects that I'm trying to package for
MacPorts, their code usually uses '#include <Cocoa/Cocoa.h>'; and the
system Cocoa.h, in turn, has a '#import <AppKit/AppKit.h>'. Is this going
to be a problem? Is the MacPorts legacy support package able to intervene
and insert its wrapper files even if a project's source code doesn't
directly #include/#import that specific header file, but instead, the
header to be patched is nested somewhere inside a tree/chain of header
#includes?


This will be — something new. Nobody actually knows how well this will work.

It probably might best be something optionally used rather than a default in legacysupport, as the opportunity for unexpected wreckage seems high. On the other hand, after year or so, if it helps but doesn’t break things, it might be defaultable.

If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

There are still many opinions that it should not be used, and the issues / fixes sent upstream instead, but I think we’re all coming to realize that something like legacysupprt is the only way forward if we’re going to support older systems.

GCC uses “fixincludes” to do much the same thing for their compiler.

Thank you for your interest and help!

Ken





Reply | Threaded
Open this post in threaded view
|

Re: Some questions regarding MacPorts legacy support package

Ken Cunningham


On Jul 5, 2020, at 3:03 PM, Jason Liu <[hidden email]> wrote:



Other than using the __MACPORTS_LEGACY_SUPPORT_APPKIT__ macro, is there some way of making it optional?

It will have to be specifically disabled by default, and then turned out optionally with some new legacysupport toggle that I/we invent to turn it on (like the one I added to force legacysupport static linking recently, for example).


If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now?


Yes, you should absolutely please do that — right now if you like — as a proof-of-concept. Stick in in files/appkitworkaround, and add something like an "-isystem ${filespath}/appkitworkaround" to your cpp flags, and see how it goes. No harm done there. If there are only one or two ports that need this, this is for sure the way to go at the beginning.

We only came up with SnowLeopardFixes -> LegacySupport after literally hundreds of ports needed the same patches for the same missing functions.

You would be surprised at the way things break so easily with what seems like a simple change in one thing that shouldn’t hurt anything. I recently symlinked one single library to make a broken port build and forgot I did it, and it caused a Tornado of fallout that would have been impossible to believe).

There are probably hundreds of ports that have their own workarounds in them that will respond the wrong way to this. Or maybe there aren’t. It’s impossible to know.

Ken
Reply | Threaded
Open this post in threaded view
|

Re: several messages

Jason Liu
In reply to this post by Fred Wright
On Sun, Jul 5, 2020 at 12:08 AM Fred Wright <[hidden email]> wrote:

On Wed, 3 Jun 2020, Jason Liu wrote:

[...]
> A solution I found which some projects (e.g. Qemu) have implemented
> basically replaces the new AppKit constants with the old AppKit ones using
> *#define* directives if the OS version is below 10.12. I've created a
> separate header file that gathers together a list of the constants I've
> been able to find, which is modeled on information from this message:
[...]

It's interesting that you cite Qemu as an example of this, when the Qemu
port is currently failing to build on 10.9 for reasons that look very
similar to this issue.

The reason why the Qemu port is probably failing is because the code in the link that I referenced was added to Qemu in 2017. It looks like Qemu has since removed those #defines, probably due to what you said at the end:

In general, direct upstream support seems preferable when possible, but some upstream developers refuse to support old OS versions, sometimes using "security" as a lame justification.

My guess, however, that the more likely reason upstream projects don't typically keep these sorts of fixes around is because they don't want it dirtying up their code for any longer than necessary. Even if you strip out the comment lines, the #defines in my file add up to over 100 lines of code that upstream projects probably don't want lingering around just for the sake of legacy support.

-- 
Jason Liu


On Sun, Jul 5, 2020 at 12:08 AM Fred Wright <[hidden email]> wrote:

On Wed, 3 Jun 2020, Jason Liu wrote:

[...]
> A solution I found which some projects (e.g. Qemu) have implemented
> basically replaces the new AppKit constants with the old AppKit ones using
> *#define* directives if the OS version is below 10.12. I've created a
> separate header file that gathers together a list of the constants I've
> been able to find, which is modeled on information from this message:
[...]

It's interesting that you cite Qemu as an example of this, when the Qemu
port is currently failing to build on 10.9 for reasons that look very
similar to this issue.

On Sat, 4 Jul 2020, Ken Cunningham wrote:
>> Some questions regarding MacPorts legacy support package
>>
>> Jason Liu jasonliu at umich.edu  <mailto:[hidden email]?Subject=Re%3A%20Some%20questions%20regarding%20MacPorts%20legacy%20support%20package&In-Reply-To=%3CCAHUrRf4a%2BvaF8YLE5eAh2Fo5g2ZaZTK%2BW6ecg%3DTj7tXtgx5vvg%40mail.gmail.com%3E>
>> Sat Jul 4 20:44:59 UTC 2020
[...]
>> Question 1:
>>
>> I've noticed that in MacportsLegacySupport.h and other wrapper files,
>> __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ is used for macOS version
>> detection. I also noticed this line in MacportsLegacySupport.h:
>>
>> /* Not needed -- #include "AvailabilityMacros.h" */
>>
>> Would it be considered kosher to use any of the other version detection
>> constants from AvailabilityMacros.h, such as MAC_OS_X_VERSION_MAX_ALLOWED?
>> Or would that be considered risky/dangerous/undesirable for some reason?
>
> We have tried to make this (so far) such that this was not needed, and
> just go with the compiler default
> “__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__” which is set to a number
> by every compiler on Apple/Darwin systems that matches the deployment
> target the compiler sees.

I'm not sure where this "__ENVIRONMENT" prefix is coming from.  Is that a
version of the definitions created by legacy-support?

Normally, there are four macros of this form available:

__MAC_OS_X_VERSION_MIN_REQUIRED
__MAC_OS_X_VERSION_MAX_ALLOWED
MAC_OS_X_VERSION_MIN_REQUIRED
MAC_OS_X_VERSION_MAX_ALLOWED

The general idea is that MIN_REQUIRED is for features that weren't
available before a certain OS version, and MAX_ALLOWED is for features
that were *removed* after a certain OS version.  Since OS versions mostly
try to be backward compatible with older versions, the first one is useful
far more often than the second.  As a general rule, if you're not sure
which one you want, then you almost certainly want MIN_REQUIRED.

Since having those defined differently would only be expected in cases
where the build target is a range of OS versions, and since MacPorts has
no concept of "OS version universality" and hence always targets one
specific OS version, once would expect the two definitions to be the same
in the MacPorts environment.  But aside from it being cleaner to use the
correct macro for the context, there's at least one case where they're
*not* the same.  From the default compiler on 10.5.8:

MacMini:OSX fw$ ./conftest
__APPLE__ = 1
__MAC_OS_X_VERSION_MIN_REQUIRED = 1058
__MAC_OS_X_VERSION_MAX_ALLOWED = 1060
_POSIX_VERSION = 200112

Thus, using MAX_ALLOWED to distinguish 10.5 from 10.6 doesn't work.  There
are several ports that don't build on 10.5 for precisely this reason.

As far as the double-undersore versus non-double-underscore aspect is
concerned, the double-underscore versions are usually defined by the
compiler itself (with no includes), but sometimes not, in which case it's
necessary to include Availability.h (but *not* necessarily
AvailabilityMacros.h).  The non-double-underscore versions are defined in
AvailabilityMacros.h.  Hence, any use of those demands including
AvailabilityMacros.h.

If anyone can explain why there's ever a reason to use the
non-double-underscore versions, I'd like to hear it.  Otherwise, doing so
is simply adding a dependency on an include that might not otherwise be
needed, and the double-underscore versions are preferable.

This is also something which some ports get wrong and have broken builds
or missing features as a consequence.

>> Question 2:
>>
>> A related question is that in MacportsLegacySupport.h, you guys use version
>> numbers such as 101300, 1070, etc. instead of the constants that are
>> defined in AvailabilityMacros.h, such as MAC_OS_X_VERSION_10_13,
>> MAC_OS_X_VERSION_10_7, etc. Is there any particular reason for not using
>> the constants and going with the actual integer numbers? It looks like not
>> even Apple's own source code is consistent with this. In
>> AvailabilityMacros.h, they use the version number constants
>> MAC_OS_X_VERSION_10_*, but in other header files like assert.h, pthread.h,
>> etc., they use the raw integers, i.e. 1070.
>>
>
> We are sticking with the numbers. See this
> <https://trac.macports.org/wiki/LeopardSDKFixes#Incorrect__MAC_OS_X_VERSION_MAX_ALLOWED>
> for the initial inspiration for that, but it just avoids a lot of
> confusion about which constants are available in which files and when.

Yes, the basic problem is that older includes don't have the newer
definitions.  So in general, if you want to support a wide range of OS
versions, it's best to stick to the numeric constants.

>> Question 3:
>>
>> As you can see from the attached file, I am currently creating a wrapper
>> for AppKit.h. However, in the projects that I'm trying to package for
>> MacPorts, their code usually uses '#include <Cocoa/Cocoa.h>'; and the
>> system Cocoa.h, in turn, has a '#import <AppKit/AppKit.h>'. Is this going
>> to be a problem? Is the MacPorts legacy support package able to intervene
>> and insert its wrapper files even if a project's source code doesn't
>> directly #include/#import that specific header file, but instead, the
>> header to be patched is nested somewhere inside a tree/chain of header
>> #includes?
>>
>
> This will be — something new. Nobody actually knows how well this will work.

It's also worth noting that this particular kind of fix probably only
needs include additions and no library additions, which is something the
PortGroup may not be set up to handle.  Of course some ports may need the
added library for *other* reasons, and that needs to be handled correctly.

> It probably might best be something optionally used rather than a
> default in legacysupport, as the opportunity for unexpected wreckage
> seems high. On the other hand, after year or so, if it helps but doesn’t
> break things, it might be defaultable.
>
> If a year or so sounds long, don’t worry. I wrote up the first version
> of legacysupport as “SnowLeopardFixes” in 2016, and it did not get
> really adopted until several years later, after a great deal of
> discussion.
>
> There are still many opinions that it should not be used, and the issues
> / fixes sent upstream instead, but I think we’re all coming to realize
> that something like legacysupprt is the only way forward if we’re going
> to support older systems.

In general, direct upstream support seems preferable when possible, but
some upstream developers refuse to support old OS versions, sometimes
using "security" as a lame justification.

Fred Wright
Reply | Threaded
Open this post in threaded view
|

Re: several messages

ryandesign2
Administrator


On Jul 5, 2020, at 17:15, Jason Liu wrote:

> My guess, however, that the more likely reason upstream projects don't typically keep these sorts of fixes around is because they don't want it dirtying up their code for any longer than necessary. Even if you strip out the comment lines, the #defines in my file add up to over 100 lines of code that upstream projects probably don't want lingering around just for the sake of legacy support.

Some developers welcome bug reports and fixes for legacy systems. Others want to require more recent versions of macOS. Some refuse to even look at my bug reports if I mention "macOS" or "MacPorts". So each developer is different and we shouldn't assume they won't accept fixes; we should ask.

The size of the fix probably makes a difference. A hundred line fix for the sake of legacy systems is surely more difficult to accept than a couple line fix.

Reply | Threaded
Open this post in threaded view
|

Re: Some questions regarding MacPorts legacy support package

Jason Liu
In reply to this post by Ken Cunningham
On Sun, Jul 5, 2020 at 6:15 PM Ken Cunningham <[hidden email]> wrote:

On Jul 5, 2020, at 3:03 PM, Jason Liu <[hidden email]> wrote:

If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now?


Yes, you should absolutely please do that — right now if you like — as a proof-of-concept. Stick in in files/appkitworkaround, and add something like an "-isystem ${filespath}/appkitworkaround" to your cpp flags, and see how it goes. No harm done there. If there are only one or two ports that need this, this is for sure the way to go at the beginning.

Okay, then I'll proceed going that route. It sounds like adding the AppKit wrapper to the legacy support package is turning out to be much more of a long-term project than I originally anticipated.

-- 
Jason Liu


On Sun, Jul 5, 2020 at 6:15 PM Ken Cunningham <[hidden email]> wrote:


On Jul 5, 2020, at 3:03 PM, Jason Liu <[hidden email]> wrote:



Other than using the __MACPORTS_LEGACY_SUPPORT_APPKIT__ macro, is there some way of making it optional?

It will have to be specifically disabled by default, and then turned out optionally with some new legacysupport toggle that I/we invent to turn it on (like the one I added to force legacysupport static linking recently, for example).


If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now?


Yes, you should absolutely please do that — right now if you like — as a proof-of-concept. Stick in in files/appkitworkaround, and add something like an "-isystem ${filespath}/appkitworkaround" to your cpp flags, and see how it goes. No harm done there. If there are only one or two ports that need this, this is for sure the way to go at the beginning.

We only came up with SnowLeopardFixes -> LegacySupport after literally hundreds of ports needed the same patches for the same missing functions.

You would be surprised at the way things break so easily with what seems like a simple change in one thing that shouldn’t hurt anything. I recently symlinked one single library to make a broken port build and forgot I did it, and it caused a Tornado of fallout that would have been impossible to believe).

There are probably hundreds of ports that have their own workarounds in them that will respond the wrong way to this. Or maybe there aren’t. It’s impossible to know.

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Some questions regarding MacPorts legacy support package

Jason Liu
On Sun, Jul 5, 2020 at 6:15 PM Ken Cunningham <[hidden email]> wrote:

On Jul 5, 2020, at 3:03 PM, Jason Liu <[hidden email]> wrote:
If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now?


Yes, you should absolutely please do that — right now if you like — as a proof-of-concept. Stick in in files/appkitworkaround, and add something like an "-isystem ${filespath}/appkitworkaround" to your cpp flags, and see how it goes. No harm done there. If there are only one or two ports that need this, this is for sure the way to go at the beginning.

On Sat, Jul 4, 2020 at 8:23 PM Ken Cunningham <[hidden email]> wrote:

Question 3:

As you can see from the attached file, I am currently creating a wrapper
for AppKit.h. However, in the projects that I'm trying to package for
MacPorts, their code usually uses '#include <Cocoa/Cocoa.h>'; and the
system Cocoa.h, in turn, has a '#import <AppKit/AppKit.h>'. Is this going
to be a problem? Is the MacPorts legacy support package able to intervene
and insert its wrapper files even if a project's source code doesn't
directly #include/#import that specific header file, but instead, the
header to be patched is nested somewhere inside a tree/chain of header
#includes?


This will be — something new. Nobody actually knows how well this will work.


I have what is potentially some very encouraging news to report on this front. Adding my AppKit wrapper file locally to my Blender port, and using the -isystem compiler flag, it appears that using this method does indeed have the effect of "intervening" when the header file that is to be wrapped occurs in the middle of a nested chain of #includes. I'm not sure whether this success will be the case for all compilers, but at least for my particular port, the wrapper is visible to the clang++-mp-9.0 compiler.

This hopefully provides further evidence that eventually adding the AppKit wrapper to the MacPorts legacy support package will be the correct way to go in the long term.

-- 
Jason Liu


On Mon, Jul 6, 2020 at 2:44 PM Jason Liu <[hidden email]> wrote:
On Sun, Jul 5, 2020 at 6:15 PM Ken Cunningham <[hidden email]> wrote:

On Jul 5, 2020, at 3:03 PM, Jason Liu <[hidden email]> wrote:

If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now?


Yes, you should absolutely please do that — right now if you like — as a proof-of-concept. Stick in in files/appkitworkaround, and add something like an "-isystem ${filespath}/appkitworkaround" to your cpp flags, and see how it goes. No harm done there. If there are only one or two ports that need this, this is for sure the way to go at the beginning.

Okay, then I'll proceed going that route. It sounds like adding the AppKit wrapper to the legacy support package is turning out to be much more of a long-term project than I originally anticipated.

-- 
Jason Liu


On Sun, Jul 5, 2020 at 6:15 PM Ken Cunningham <[hidden email]> wrote:


On Jul 5, 2020, at 3:03 PM, Jason Liu <[hidden email]> wrote:



Other than using the __MACPORTS_LEGACY_SUPPORT_APPKIT__ macro, is there some way of making it optional?

It will have to be specifically disabled by default, and then turned out optionally with some new legacysupport toggle that I/we invent to turn it on (like the one I added to force legacysupport static linking recently, for example).


If a year or so sounds long, don’t worry. I wrote up the first version of legacysupport as “SnowLeopardFixes” in 2016, and it did not get really adopted until several years later, after a great deal of discussion.

A year?! If it's going to take it that long to get it incorporated into the legacy support package, then maybe I'm better off just directly adding my header to the 'files/' folder in my Blender and MaterialX ports for now?


Yes, you should absolutely please do that — right now if you like — as a proof-of-concept. Stick in in files/appkitworkaround, and add something like an "-isystem ${filespath}/appkitworkaround" to your cpp flags, and see how it goes. No harm done there. If there are only one or two ports that need this, this is for sure the way to go at the beginning.

We only came up with SnowLeopardFixes -> LegacySupport after literally hundreds of ports needed the same patches for the same missing functions.

You would be surprised at the way things break so easily with what seems like a simple change in one thing that shouldn’t hurt anything. I recently symlinked one single library to make a broken port build and forgot I did it, and it caused a Tornado of fallout that would have been impossible to believe).

There are probably hundreds of ports that have their own workarounds in them that will respond the wrong way to this. Or maybe there aren’t. It’s impossible to know.

Ken