Preparing for 2.6.0 beta

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Preparing for 2.6.0 beta

Joshua Root-8
I think we're pretty close to being ready to make a new stable branch
and tag a beta. Clemens is going to make one more change related to
Xcode checking; are there any other changes that really need to go in
before the beta?

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

ryandesign2
Administrator


On Aug 23, 2019, at 22:45, Joshua Root wrote:

> I think we're pretty close to being ready to make a new stable branch
> and tag a beta. Clemens is going to make one more change related to
> Xcode checking; are there any other changes that really need to go in
> before the beta?

It would be nice if the problems with archive_site_local could be fixed. We use a custom archive_site_local on Buildbot and Travis and because of these bugs it doesn't work quite right.



https://trac.macports.org/ticket/57718

Unexpected behavior when archive_site_local contains more than 1 URL

I submitted a PR for this. You requested tests verifying the behavior. I don't know how to write tests for MacPorts.



https://trac.macports.org/ticket/57719

archive_site_local URLs are attempted twice



https://trac.macports.org/ticket/57720

preferred_hosts has no effect on archive_site_local



https://trac.macports.org/ticket/57690

Warning when disabling default archive_sites

This isn't really related, but maybe something we should fix: either our behavior is wrong or our documentation about being able to disable this setting is wrong.



I think similar problems probably exist for the local setting for master_sites and patch_sites.

Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Mojca Miklavec-2
In reply to this post by Joshua Root-8
On Sat, 24 Aug 2019 at 05:45, Joshua Root <[hidden email]> wrote:
>
> I think we're pretty close to being ready to make a new stable branch
> and tag a beta. Clemens is going to make one more change related to
> Xcode checking; are there any other changes that really need to go in
> before the beta?

I would be grateful for addressing these:

https://trac.macports.org/ticket/50448
https://trac.macports.org/ticket/56042
https://trac.macports.org/ticket/55471

Mojca
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Joshua Root-8
On 2019-8-24 21:59 , Mojca Miklavec wrote:

> On Sat, 24 Aug 2019 at 05:45, Joshua Root <[hidden email]> wrote:
>>
>> I think we're pretty close to being ready to make a new stable branch
>> and tag a beta. Clemens is going to make one more change related to
>> Xcode checking; are there any other changes that really need to go in
>> before the beta?
>
> I would be grateful for addressing these:
>
> https://trac.macports.org/ticket/50448

Way too late to do anything about that in base. The plan is as described
in comment 16.

> https://trac.macports.org/ticket/56042

I don't think this is really needed with bootstrapping of libcxx working.

> https://trac.macports.org/ticket/55471

Not sure what is even left to do there, but it also seems less important
if we don't support multiple stdlibs support going forward.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

ryandesign2
Administrator


On Aug 24, 2019, at 08:55, Joshua Root wrote:

> On 2019-8-24 21:59 , Mojca Miklavec wrote:
>
>> I would be grateful for addressing these:
>>
>> https://trac.macports.org/ticket/50448
>
> Way too late to do anything about that in base. The plan is as described
> in comment 16.

I didn't realize that was the plan we had settled on, but I haven't been following MacPorts developments that closely this year.

I saw the commits from you to base, changing the default stdlib to libc++ on 10.6-10.8. Looks like your proposal is for users to run rev-upgrade after upgrading.

What is the plan for the buildbot and our prebuilt archives? Obviously we would need to rebuild all of the archives for C++ software on 10.6-10.8. Since you are not planning on changing the archive filenames or their location on the packages server, and since the buildbot uses the presence of the archives on the packages server as an indicator that the ports have already been built, the buildbot will not know that it should rebuild them. I imagine we will have to manually identify all the ports that use C++, remove their archives from the packages server, uninstall them on the 10.6-10.8 buildbot workers, and then trigger new builds for them.

Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Ken Cunningham
In reply to this post by Joshua Root-8
We might expect to see some breakage in ports that use xcodebuild on 10.6 to 10.8 once the default is libc++.

I have tweaked (hacked) my own 10.6 sdk with symlinks to current cctools, clang, & ld64 at times to fix these errors. For general use, I wondered about using xcode's sdk overlay feature to fix it.

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Joshua Root-8
In reply to this post by ryandesign2
On 2019-8-25 00:40 , Ryan Schmidt wrote:
>
> What is the plan for the buildbot and our prebuilt archives? Obviously we would need to rebuild all of the archives for C++ software on 10.6-10.8. Since you are not planning on changing the archive filenames or their location on the packages server, and since the buildbot uses the presence of the archives on the packages server as an indicator that the ports have already been built, the buildbot will not know that it should rebuild them. I imagine we will have to manually identify all the ports that use C++, remove their archives from the packages server, uninstall them on the 10.6-10.8 buildbot workers, and then trigger new builds for them.

Yes, that sounds about right, except for the manually part. With the
cxx_stdlib info now in the archive metadata, it's relatively quick to
identify which archives are linked with libstdc++. Probably still best
to do a delete_old_archives.py run with a low targetSize first. (Can
grep the output for 'darwin_1[012]'.)

You could use the list of archives to identify which ports to uninstall
from the workers, or just 'port uninstall installed' and let them
reinstall from archives if possible as needed.

- Josh

cxx_archives.sh (1016 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Mojca Miklavec-2
In reply to this post by ryandesign2
On Sat, 24 Aug 2019 at 16:40, Ryan Schmidt wrote:

> On Aug 24, 2019, at 08:55, Joshua Root wrote:
> > On 2019-8-24 21:59 , Mojca Miklavec wrote:
> >
> >> I would be grateful for addressing these:
> >>
> >> https://trac.macports.org/ticket/50448
> >
> > Way too late to do anything about that in base. The plan is as described
> > in comment 16.
>
> I didn't realize that was the plan we had settled on, but I haven't been following MacPorts developments that closely this year.
>
> I saw the commits from you to base, changing the default stdlib to libc++ on 10.6-10.8. Looks like your proposal is for users to run rev-upgrade after upgrading.
>
> What is the plan for the buildbot and our prebuilt archives? Obviously we would need to rebuild all of the archives for C++ software on 10.6-10.8. Since you are not planning on changing the archive filenames or their location on the packages server, and since the buildbot uses the presence of the archives on the packages server as an indicator that the ports have already been built, the buildbot will not know that it should rebuild them. I imagine we will have to manually identify all the ports that use C++, remove their archives from the packages server, uninstall them on the 10.6-10.8 buildbot workers, and then trigger new builds for them.

Would it be possible to set up at least one builder with 10.6 running
MacPorts master, building all ports with libc++ (apparently default by
now in master), just not uploading the archives to the public site, or
potentially uploading them to a 10.6 subfolder? That is, instead of
putting the file under
    http://packages.macports.org/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
we would put it under
    http://packages.macports.org/10.6/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
or something else than 10.6 (darwin10.x86_64, 10.6.x86_64, ...).

That way we could at least have all the packages ready by the time
when we do the switch.

A few years back Ryan's biggest opposition to introducing subfolders
with macOS version was the ability to get a quick overview of macOS
versions where the port built successfully. With the new app I believe
this is no longer of any concern.

Independent from the stdlibc++ vs. libc++ question I still believe
that it would be really nice to be able to easily include/exclude the
set of macOS versions. Say that we organize the meeting with very poor
internet connectivity (which incidentally seems to be the case this
October). It comes really handy if we can bring a mirror of all files,
but excluding the platforms we are sure we won't be needing. Sure
that's also possible to do by filtering out certain filenames, but
being able to just skip some folders is much easier.

Mojca
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Fred Wright
In reply to this post by Joshua Root-8


On Sat, 24 Aug 2019, Joshua Root wrote:

> I think we're pretty close to being ready to make a new stable branch
> and tag a beta. Clemens is going to make one more change related to
> Xcode checking; are there any other changes that really need to go in
> before the beta?

Though I wouldn't call a 16-month-old ticket "urgent", it might be nice if
someone could look at:

  https://trac.macports.org/ticket/56437

This might become more significant with the libcxx churn.

Fred Wright
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Joshua Root-8
In reply to this post by Mojca Miklavec-2
On 2019-8-25 03:00 , Mojca Miklavec wrote:

> Would it be possible to set up at least one builder with 10.6 running
> MacPorts master, building all ports with libc++ (apparently default by
> now in master), just not uploading the archives to the public site, or
> potentially uploading them to a 10.6 subfolder? That is, instead of
> putting the file under
>     http://packages.macports.org/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
> we would put it under
>     http://packages.macports.org/10.6/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
> or something else than 10.6 (darwin10.x86_64, 10.6.x86_64, ...).
>
> That way we could at least have all the packages ready by the time
> when we do the switch.

I guess that depends on how much time Ryan has to work on this.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Clemens Lang-2
In reply to this post by Joshua Root-8
Hi Josh,

On Sat, Aug 24, 2019 at 01:45:30PM +1000, Joshua Root wrote:
> I think we're pretty close to being ready to make a new stable branch
> and tag a beta. Clemens is going to make one more change related to
> Xcode checking

Here's the change: https://github.com/macports/macports-base/pull/143

--
Clemens
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Mojca Miklavec-2
In reply to this post by Joshua Root-8
On Sat, 24 Aug 2019 at 05:45, Joshua Root wrote:
>
> I think we're pretty close to being ready to make a new stable branch

+1 :)

> are there any other changes that really need to go in
> before the beta?

It would be kind of cool if we could display a message inviting users
to submit statistics for the new site when they upgrade.

But that's neither urgent nor written yet.
(If anyone has time to write it, it would be cool, but definitely no
need to delay the release for it.)

Mojca
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Joshua Root-8
On 2019-8-27 07:39 , Mojca Miklavec wrote:
> It would be kind of cool if we could display a message inviting users
> to submit statistics for the new site when they upgrade.
>
> But that's neither urgent nor written yet.
> (If anyone has time to write it, it would be cool, but definitely no
> need to delay the release for it.)

Well there's this:
<https://github.com/macports/macports-base/blob/master/Makefile.in#L112>

I think it's only visible when selfupdating if you use -v though.

The GSoC project idea for 'port news' would be handy for this sort of
thing. Best not to the delay the release any more though as you say.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Joshua Root-8
I'll go ahead and branch and tag a beta1 on the weekend if there are no
objections.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Bjarne D Mathiesen
In reply to this post by Joshua Root-8
Joshua Root wrote:
> I think we're pretty close to being ready to make a new stable branch
> and tag a beta. Clemens is going to make one more change related to
> Xcode checking; are there any other changes that really need to go in
> before the beta?

⛔️
Presently, I'm of the opinion that the effort to automate
LibcxxOnOlderSystems ought to be ripped out again - it's definitevely
👎🏻not👎🏻 working - see the LibcxxOnOlderSystems threads in macports-users
⛔️

--
Bjarne D Mathiesen
Korsør ; Danmark ; Europa
----------------------------------------------------------------------
denne besked er skrevet i et (næsten) M$-frit miljø
MacOS X 10.13.6 High Sierra :
   17" 2011 MacBook Pro ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3
   2012 Mac Pro ; 2 x 3.46GHz 6-Core Xeon ; 48GB
MacOS X 10.6.8 Snow Leopard :
   Mac Mini ; 2GHz Core 2 Duo (64 bit) ; 4GB (3GB actual) 667MHz
   Mac Mini ; 1.83GHz Core Duo (32 bit) ; 2GB 667Mhz
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Ken Cunningham
In reply to this post by Joshua Root-8
oh no, bjarne....has to happen.!

whatever issue might remain, we will fix.

this will greatly simplify things for older systems, and keep them supported longer. Trying to build c++11 w clang against the gcc headers is driving me batty and causing way too much wasted effort...

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

ryandesign2
Administrator
In reply to this post by Joshua Root-8


On Aug 24, 2019, at 23:20, Joshua Root wrote:

> On 2019-8-25 03:00 , Mojca Miklavec wrote:
>> Would it be possible to set up at least one builder with 10.6 running
>> MacPorts master, building all ports with libc++ (apparently default by
>> now in master), just not uploading the archives to the public site, or
>> potentially uploading them to a 10.6 subfolder? That is, instead of
>> putting the file under
>>    http://packages.macports.org/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
>> we would put it under
>>    http://packages.macports.org/10.6/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
>> or something else than 10.6 (darwin10.x86_64, 10.6.x86_64, ...).
>>
>> That way we could at least have all the packages ready by the time
>> when we do the switch.
>
> I guess that depends on how much time Ryan has to work on this.

In my opinion, it would have been a good idea to develop a plan for having the archives for libc++ on older systems at different URLs (either subdirectories or different archive filenames). That way, we could have deployed new builders to build the libc++ archives for older systems prior to the release, so that they would be available at release.

However, since Joshua decided to flip the switch for libc++ on older systems without such a plan being in place, we won't have archives for libc++ on older systems available at release of 2.6.0, and users will just have to wait for the builds to finish or build things themselves.

Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

ryandesign2
Administrator
In reply to this post by Mojca Miklavec-2


On Aug 24, 2019, at 12:00, Mojca Miklavec wrote:

> Independent from the stdlibc++ vs. libc++ question I still believe
> that it would be really nice to be able to easily include/exclude the
> set of macOS versions. Say that we organize the meeting with very poor
> internet connectivity (which incidentally seems to be the case this
> October). It comes really handy if we can bring a mirror of all files,
> but excluding the platforms we are sure we won't be needing. Sure
> that's also possible to do by filtering out certain filenames, but
> being able to just skip some folders is much easier.

It is easy to write an rsync command do that that already.


Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

Joshua Root-8
In reply to this post by ryandesign2
On 2019-9-20 11:38 , Ryan Schmidt wrote:

>
>
> On Aug 24, 2019, at 23:20, Joshua Root wrote:
>
>> On 2019-8-25 03:00 , Mojca Miklavec wrote:
>>> Would it be possible to set up at least one builder with 10.6 running
>>> MacPorts master, building all ports with libc++ (apparently default by
>>> now in master), just not uploading the archives to the public site, or
>>> potentially uploading them to a 10.6 subfolder? That is, instead of
>>> putting the file under
>>>    http://packages.macports.org/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
>>> we would put it under
>>>    http://packages.macports.org/10.6/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
>>> or something else than 10.6 (darwin10.x86_64, 10.6.x86_64, ...).
>>>
>>> That way we could at least have all the packages ready by the time
>>> when we do the switch.
>>
>> I guess that depends on how much time Ryan has to work on this.
>
> In my opinion, it would have been a good idea to develop a plan for having the archives for libc++ on older systems at different URLs (either subdirectories or different archive filenames). That way, we could have deployed new builders to build the libc++ archives for older systems prior to the release, so that they would be available at release.
>
> However, since Joshua decided to flip the switch for libc++ on older systems without such a plan being in place, we won't have archives for libc++ on older systems available at release of 2.6.0, and users will just have to wait for the builds to finish or build things themselves.

There's been over a year since the 2.5 release to develop that plan.
Nobody did it. The transition needs to happen.

But if we really want to back out the stdlib change, there's still time.

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: Preparing for 2.6.0 beta

ryandesign2
Administrator


On Sep 19, 2019, at 20:45, Joshua Root wrote:

> On 2019-9-20 11:38 , Ryan Schmidt wrote:
>>
>>
>> On Aug 24, 2019, at 23:20, Joshua Root wrote:
>>
>>> On 2019-8-25 03:00 , Mojca Miklavec wrote:
>>>> Would it be possible to set up at least one builder with 10.6 running
>>>> MacPorts master, building all ports with libc++ (apparently default by
>>>> now in master), just not uploading the archives to the public site, or
>>>> potentially uploading them to a 10.6 subfolder? That is, instead of
>>>> putting the file under
>>>>   http://packages.macports.org/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
>>>> we would put it under
>>>>   http://packages.macports.org/10.6/clang-3.4/clang-3.4-3.4.2_12.darwin_10.x86_64.tbz2
>>>> or something else than 10.6 (darwin10.x86_64, 10.6.x86_64, ...).
>>>>
>>>> That way we could at least have all the packages ready by the time
>>>> when we do the switch.
>>>
>>> I guess that depends on how much time Ryan has to work on this.
>>
>> In my opinion, it would have been a good idea to develop a plan for having the archives for libc++ on older systems at different URLs (either subdirectories or different archive filenames). That way, we could have deployed new builders to build the libc++ archives for older systems prior to the release, so that they would be available at release.
>>
>> However, since Joshua decided to flip the switch for libc++ on older systems without such a plan being in place, we won't have archives for libc++ on older systems available at release of 2.6.0, and users will just have to wait for the builds to finish or build things themselves.
>
> There's been over a year since the 2.5 release to develop that plan.
> Nobody did it. The transition needs to happen.
>
> But if we really want to back out the stdlib change, there's still time.

I wouldn't back it out. It does need to happen. We've been talking about it for many years.

12