Changing default cxx_stdlib to libc++

classic Classic list List threaded Threaded
133 messages Options
1234 ... 7
Reply | Threaded
Open this post in threaded view
|

Changing default cxx_stdlib to libc++

Joshua Root-8
Code to check C++ stdlib linkage in rev-upgrade is now in master. That
takes care of the main obstacle to being able to change the default stdlib.

That leaves a couple of questions:

Which OS versions can we make the change on? At first glance it looks
like probably 10.6 through 10.8. 10.5 appears to have a much more
involved bootstrap process.

Do we need to make changes to the compiler selection logic? The
LibcxxOnOlderSystems instructions override the selection in
macports.conf. Will there be any bootstrap issues for users after the
switch?

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham
Thank God.

10.6.8, agree.

I use clang 3.9 as a good balance between capability and compatability, but might as well sort out how to use clang 5 as that is more consistent w older systems, and I nearly have thread_local working there on 10.6

Ken

Sent from my iPhone

> On Mar 8, 2018, at 9:24 AM, Joshua Root <[hidden email]> wrote:
>
> Code to check C++ stdlib linkage in rev-upgrade is now in master. That
> takes care of the main obstacle to being able to change the default stdlib.
>
> That leaves a couple of questions:
>
> Which OS versions can we make the change on? At first glance it looks
> like probably 10.6 through 10.8. 10.5 appears to have a much more
> involved bootstrap process.
>
> Do we need to make changes to the compiler selection logic? The
> LibcxxOnOlderSystems instructions override the selection in
> macports.conf. Will there be any bootstrap issues for users after the
> switch?
>
> - Josh
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ryan Schmidt-24
In reply to this post by Joshua Root-8

On Mar 8, 2018, at 11:24, Joshua Root wrote:

> Code to check C++ stdlib linkage in rev-upgrade is now in master. That
> takes care of the main obstacle to being able to change the default stdlib.

Currently, we publish archives on packages.macports.org for 10.8 and earlier that are built for libstdc++. Currently, if we were to create archives for libc++ on those systems, the archive names would be the same. Is your plan to leave it that way, or to change the archive names for libc++ to distinguish them from the old libstdc++ archives?

Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Joshua Root-8
On 2018-3-9 07:07 , Ryan Schmidt wrote:
>
> On Mar 8, 2018, at 11:24, Joshua Root wrote:
>
>> Code to check C++ stdlib linkage in rev-upgrade is now in master. That
>> takes care of the main obstacle to being able to change the default stdlib.
>
> Currently, we publish archives on packages.macports.org for 10.8 and earlier that are built for libstdc++. Currently, if we were to create archives for libc++ on those systems, the archive names would be the same. Is your plan to leave it that way, or to change the archive names for libc++ to distinguish them from the old libstdc++ archives?

I was not planning to rename the archives. It would work if we left all
the existing archives, albeit inefficiently when an archive using the
old stdlib is downloaded and immediately gets rebuilt. I figured we
could remove all the archives that use C++ (quick hacked together
example script attached - probably want to clean up old archives before
running it...).

- Josh

cxx_archives.sh (436 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ryan Schmidt-24

On Mar 8, 2018, at 15:26, Joshua Root wrote:

> On 2018-3-9 07:07 , Ryan Schmidt wrote:
>
>> On Mar 8, 2018, at 11:24, Joshua Root wrote:
>>
>>> Code to check C++ stdlib linkage in rev-upgrade is now in master. That
>>> takes care of the main obstacle to being able to change the default stdlib.
>>
>> Currently, we publish archives on packages.macports.org for 10.8 and earlier that are built for libstdc++. Currently, if we were to create archives for libc++ on those systems, the archive names would be the same. Is your plan to leave it that way, or to change the archive names for libc++ to distinguish them from the old libstdc++ archives?
>
> I was not planning to rename the archives. It would work if we left all
> the existing archives, albeit inefficiently when an archive using the
> old stdlib is downloaded and immediately gets rebuilt.

Ok, rev-upgrade would detect a mismatch between the user's selected cxx_stdlib and the one in the archive they received. That does answer one concern of mine.

> I figured we
> could remove all the archives that use C++ (quick hacked together
> example script attached - probably want to clean up old archives before
> running it...).
>
> - Josh
> <cxx_archives.sh>

When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++. When is that? Certainly no sooner than two weeks after release since that's how long it takes MacPorts to prompt a user to selfupdate. Do all users follow that prompt right away? The prompt goes away if you sync rather than selfupdate, right? Due to the length of time between MacPorts releases, users may be in the habit of running sync rather than selfupdate, since sync is faster. It may be some time before all users actually upgrade to 2.5.

How will users upgrade from libstdc++ to libc++? What subset of the instructions on the LibcxxOnOlderSystems page would they have to follow? What if they don't do that and stay on libstdc++?

This strategy means we won't have any libc++ archives available for legacy users when they switch over. It will take us time to build them. If we changed the archive name and/or directory they're in, we could build libc++ archives in advance and have them ready for users when they switch.

Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham

On 2018-03-08, at 4:27 PM, Ryan Schmidt wrote:


> How will users upgrade from libstdc++ to libc++? What subset of the instructions on the LibcxxOnOlderSystems page would they have to follow? What if they don't do that and stay on libstdc++?

We do have to ponder the bootstrap toolchain process carefully. There is a clang-3.4 build that is against libstdc++ as a stepping stone, and some ld64 bootstrapping.

Probably need to walk it through on a VM to make sure it all jives. I can help with that, as I'm sure others can.

Ken



db
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

db
In reply to this post by Ryan Schmidt-24
On 9 Mar 2018, at 01:27, Ryan Schmidt <[hidden email]> wrote:
> When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++.

How about those that build MP from source?

Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Joshua Root-8
In reply to this post by Ryan Schmidt-24
On 2018-3-9 11:27 , Ryan Schmidt wrote:
>
> When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++. When is that? Certainly no sooner than two weeks after release since that's how long it takes MacPorts to prompt a user to selfupdate. Do all users follow that prompt right away? The prompt goes away if you sync rather than selfupdate, right? Due to the length of time between MacPorts releases, users may be in the habit of running sync rather than selfupdate, since sync is faster. It may be some time before all users actually upgrade to 2.5.

Sure. We could release a version that records and checks the stdlib and
save the actual changeover for a later version. Don't want to wait too
long though.

I imagine we'd run the script once right before we release the base
version that changes the cxx_stdlib default value, then do the release,
turn off the legacy builders, turn on the new ones, and run the script
again just to get any ports that got built in between.

> How will users upgrade from libstdc++ to libc++?

The default value of cxx_stdlib will simply change with a base release.
We'll want to announce the changeover well in advance of that release,
and have prominent notices shown (to tell users who followed
LibcxxOnOlderSystems that they can return to 'buildfromsource ifneeded'
among other things).

> What subset of the instructions on the LibcxxOnOlderSystems page would they have to follow?

None of them, hopefully.

We may have to create some bootstrap versions of toolchain ports in
order to achieve this, so that they can be automatically installed as
dependencies with no manual intervention. I'm hoping someone more
familiar with building a C++11 toolchain on these old OS versions will
chime in.

> What if they don't do that and stay on libstdc++?
They would have to specifically set cxx_stdlib in macports.conf in order
to do that. If they do, it will work fine. They may want to set
'buildfromsource always' to prevent useless downloading of C++ ports.

> This strategy means we won't have any libc++ archives available for legacy users when they switch over. It will take us time to build them. If we changed the archive name and/or directory they're in, we could build libc++ archives in advance and have them ready for users when they switch.

Well, to be fair, users who followed LibcxxOnOlderSystems get no
binaries right now (and users on libstdc++ get no C++11 ports, binary or
otherwise). We could turn on the non-legacy builders early but not
deploy the archives, or deploy them somewhere else and then move them?

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Joshua Root-8
In reply to this post by db
On 2018-3-9 20:38 , db wrote:
> On 9 Mar 2018, at 01:27, Ryan Schmidt <[hidden email]> wrote:
>> When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++.
>
> How about those that build MP from source?

There are no additional problems arising from that, whether you mean
building base from source or building all ports from source. In fact
there's one less problem if you're not downloading any binary archives,
as you can't get ones that are built for the wrong cxx_stdlib.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham
In reply to this post by Joshua Root-8


> On Mar 9, 2018, at 06:18, Joshua Root <[hidden email]> wrote:
>
> We may have to create some bootstrap versions of toolchain ports in
> order to achieve this, so that they can be automatically installed as
> dependencies with no manual intervention.

Maybe we could just force clang 3.4 to build against libstdc++. -- perhaps dependant on whether libcxx is installed or not....

Ken
db
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

db
In reply to this post by Joshua Root-8
On 9 Mar 2018, at 15:21, Joshua Root <[hidden email]> wrote:
> On 2018-3-9 20:38 , db wrote:
>> On 9 Mar 2018, at 01:27, Ryan Schmidt <[hidden email]> wrote:
>>> When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++.
>> How about those that build MP from source?
> There are no additional problems arising from that

If I build base and ports from source, will buildfromsource be changed unilaterally by MP or would it need user intervention? I also have certain versions of gcc and llvm locally to avoid rebuilding after minor versions and revbumps.
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Joshua Root-8
On 2018-3-10 08:43 , db wrote:
> On 9 Mar 2018, at 15:21, Joshua Root <[hidden email]> wrote:
>> On 2018-3-9 20:38 , db wrote:
>>> On 9 Mar 2018, at 01:27, Ryan Schmidt <[hidden email]> wrote:
>>>> When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++.
>>> How about those that build MP from source?
>> There are no additional problems arising from that
>
> If I build base and ports from source, will buildfromsource be changed unilaterally by MP or would it need user intervention? I also have certain versions of gcc and llvm locally to avoid rebuilding after minor versions and revbumps.

We're not going to modify your macports.conf.

- Josh
db
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

db
On 10 Mar 2018, at 00:09, Joshua Root <[hidden email]> wrote:
> On 2018-3-10 08:43 , db wrote:
>>
>> If I build base and ports from source, will buildfromsource be changed unilaterally by MP or would it need user intervention? I also have certain versions of gcc and llvm locally to avoid rebuilding after minor versions and revbumps.
> We're not going to modify your macports.conf.

How will I know about the lib change and the availability of binaries? Besides rebuilding from source from time to time, I only do port sync.
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham

On 2018-03-10, at 3:38 AM, db wrote:


> How will I know about the lib change and the availability of binaries? Besides rebuilding from source from time to time, I only do port sync.

I think I"m likely not going to change anything on my LibcxxOnOlderSystems 10.6.8 system for a while, db. It will continue to work just as it does now, and I don't mind building from source -- kinda got used to it. So there will be no difference at all. I'll probably keep building against a current lib curl installed via /opt/bootstrap as well, just 'cause it works.

However, at some point in the next few months, when most of the dust has settled and most or all of the available binaries are all properly built against libc++, I'll change the single line in macports.conf from "buildfromsource always" to "buildfromsource if needed" , and get some prebuilt binaries. This will be nice for the real big ones, like octave and llvm and clang and qt4 etc.

The real benefit to older systems will be supporting them. If it all goes as planned, there should be considerably less work to do to fix odd errors that show up on older compilers. I may be able to roll out the webkit2gtk fixes I have that work on libc++ but are just too much of a PITA to sort out against gcc7's libstdc++.

All ports will build with a version of clang similar to a 10.13 system's clang, and against libc++.

The only hiccups should be deficiencies in the SDK, libc, and some Xcode things -- libc we can fix most of, the rest is  -- well -- every system has to die sometime.

I would like to see clang set up on these systems to default to adding -stdlib=libc++ if it's not otherwise specified on the build line, like newer systems do -- but even if  that isn't allowable, there should be a marked reduction in fixing ancient headers / outdated compiler capabilities, etc

Best,

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham

On 2018-03-10, at 9:06 AM, Kenneth F. Cunningham wrote:


> I'll probably keep building against a current lib curl installed via /opt/bootstrap as well, just 'cause it works.

I meant to say I'll keep building MacPorts against a current libcurl...

> Best,
>
> Ken

db
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

db
In reply to this post by Ken Cunningham
On 10 Mar 2018, at 18:06, "Kenneth F. Cunningham" <[hidden email]> wrote:
> I think I"m likely not going to change anything on my LibcxxOnOlderSystems 10.6.8 system for a while, db. It will continue to work just as it does now, and I don't mind building from source -- kinda got used to it.

Except for building from source for minor versions and revbumps, especially large binaries, and for ports that have open defects. Oh well…
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham

On 2018-03-10, at 12:23 PM, db wrote:

> On 10 Mar 2018, at 18:06, "Kenneth F. Cunningham" <[hidden email]> wrote:
>> I think I"m likely not going to change anything on my LibcxxOnOlderSystems 10.6.8 system for a while, db. It will continue to work just as it does now, and I don't mind building from source -- kinda got used to it.
>
> Except for building from source for minor versions and revbumps, especially large binaries, and for ports that have open defects. Oh well…

Well, everything for 10.6.8 users on MacPorts is about to get supremely better. Hard to be too upset about that!

Ken
db
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

db
On 11 Mar 2018, at 02:47, "Kenneth F. Cunningham" <[hidden email]> wrote:
> On 2018-03-10, at 12:23 PM, db wrote:
>> Except for building from source for minor versions and revbumps, especially large binaries, and for ports that have open defects. Oh well…
> Well, everything for 10.6.8 users on MacPorts is about to get supremely better. Hard to be too upset about that!

I don't doubt that! What I don't understand is, why would port be allowed to build a port from source on a certain system that already failed on the buildbot, and what's worse, be left with a port in a non-working state, while downloading binaries would work, because it couldn't donwload something that's failed to build.
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Ken Cunningham


> On Mar 11, 2018, at 06:06, db <[hidden email]> wrote:
>
>> On 11 Mar 2018, at 02:47, "Kenneth F. Cunningham" <[hidden email]> wrote:
>>> On 2018-03-10, at 12:23 PM, db wrote:
>>> Except for building from source for minor versions and revbumps, especially large binaries, and for ports that have open defects. Oh well…
>> Well, everything for 10.6.8 users on MacPorts is about to get supremely better. Hard to be too upset about that!
>
> I don't doubt that! What I don't understand is, why would port be allowed to build a port from source on a certain system that already failed on the buildbot, and what's worse, be left with a port in a non-working state, while downloading binaries would work, because it couldn't donwload something that's failed to build.

You can sent your buildfromsource value to "never" I believe .... would that do what you're asking?

K
Reply | Threaded
Open this post in threaded view
|

Re: Changing default cxx_stdlib to libc++

Rainer Müller-4
In reply to this post by Joshua Root-8
On 2018-03-09 15:21, Joshua Root wrote:

> On 2018-3-9 20:38 , db wrote:
>> On 9 Mar 2018, at 01:27, Ryan Schmidt <[hidden email]> wrote:
>>> When would we run this script to delete libstdc++ archives? Presumably, after all legacy users have MacPorts 2.5 and have switched to libc++.
>>
>> How about those that build MP from source?
>
> There are no additional problems arising from that, whether you mean
> building base from source or building all ports from source. In fact
> there's one less problem if you're not downloading any binary archives,
> as you can't get ones that are built for the wrong cxx_stdlib.

I think we should use cxx_stdlib as one additional criteria to respect
when selecting an archive site here:

https://github.com/macports/macports-ports/blob/master/_resources/port1.0/fetch/archive_sites.tcl

https://github.com/macports/macports-base/blob/995dde8476c48580db4f6eedfde09e90dc5e8c99/src/package1.0/portarchivefetch.tcl#L66

Then the archive site would no longer be used if anyone changes the
default in macports.conf.

Rainer
1234 ... 7