Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by neverpanic):

 Replying to [comment:44 ryandesign]:
 > Yes, I know that merely using a bundled curl that's still linked with
 the system's openssl will not solve all of the problems being discussed in
 this ticket. But I think we should still do it because it will solve some
 of the problems (e.g. comment:30).

 I think we all agree that we should not bundle an SSL library with
 MacPorts.

 Now that the mirroring is fixed, I'm wondering whether we should just
 close this ticket as wontfix. You mentioned that you want a bundled
 libcurl in MacPorts for other reasons, but the cosmetic issue you linked
 in #51045 does in my opinion not warrant the additional effort of bundling
 libcurl (especially since it's fixed since two OS releases and we could
 add a simple workaround in base for that).

 Do you have any other reasons for bundling a libcurl? If not, I would
 close this ticket as wontfix.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:46>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by ryandesign):

 Replying to [comment:46 neverpanic]:
 > Replying to [comment:44 ryandesign]:
 > > Yes, I know that merely using a bundled curl that's still linked with
 the system's openssl will not solve all of the problems being discussed in
 this ticket. But I think we should still do it because it will solve some
 of the problems (e.g. comment:30).
 >
 > I think we all agree that we should not bundle an SSL library with
 MacPorts.

 I do not agree yet. Older OSes need a newer ssl library to fetch from some
 https sites. So can't we provide a newer openssl or libressl on older
 systems, and use Apple's Secure Transport on newer ones? Yes there might
 be security vulnerabilities discovered in the ssl library we bundle, but
 isn't it likely that there are fewer vulnerabilities in it than in the old
 openssl version used on those systems?

 Your objection in comment:3 was that we should not distribute a CA bundle.
 Would bundling an ssl library require us to include CA bundle? Doesn't the
 OS ship with one that we could use? Or is that part of the problem—the old
 OS's CA bundle doesn't contain the information needed to trust new sites?


 > Now that the mirroring is fixed, I'm wondering whether we should just
 close this ticket as wontfix. You mentioned that you want a bundled
 libcurl in MacPorts for other reasons, but the cosmetic issue you linked
 in #51045 does in my opinion not warrant the additional effort of bundling
 libcurl (especially since it's fixed since two OS releases and we could
 add a simple workaround in base for that).
 >
 > Do you have any other reasons for bundling a libcurl? If not, I would
 close this ticket as wontfix.

 #51045 was merely the first example of a curl bug that's fixed in newer
 versions that I found to link to this ticket. There's another cosmetic
 issue I found on Leopard that's been fixed. I'm sure there have been more
 bugs fixed in curl over the years.

 The main difficulty or effort of bundling libcurl was originally that we
 didn't bundle any other software with MacPorts, so a method of doing that
 had to be invented. Now that we have created that method and already use
 it to bundle 4 libraries, it shouldn't be that difficult to bundle a few
 more libraries for modern ssl support or xz decompression (#52000).

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:47>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
In reply to this post by MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by yan12125):

 FWIW, I have a branch that bundles LibreSSL + libcurl with macports-base
 at https://github.com/yan12125/macports-base/tree/bundle-curl.

 I choose LibreSSL just because it's based on autotools, while OpenSSL has
 a custom Perl-based configure system that once brought some headache to
 me. It should not be difficult to switch LibreSSL back to OpenSSL for
 macports-base.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:48>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
In reply to this post by MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by noloader):

 Replying to [comment:47 ryandesign]:
 > Replying to [comment:46 neverpanic]:
 > > Replying to [comment:44 ryandesign]:
 > > > Yes, I know that merely using a bundled curl that's still linked
 with the system's openssl will not solve all of the problems being
 discussed in this ticket. But I think we should still do it because it
 will solve some of the problems (e.g. comment:30).
 > >
 > > I think we all agree that we should not bundle an SSL library with
 MacPorts.
 >
 > I do not agree yet. Older OSes need a newer ssl library to fetch from
 some https sites. So can't we provide a newer openssl or libressl on older
 systems, and use Apple's Secure Transport on newer ones? Yes there might
 be security vulnerabilities discovered in the ssl library we bundle, but
 isn't it likely that there are fewer vulnerabilities in it than in the old
 openssl version used on those systems?
 >
 > Your objection in comment:3 was that we should not distribute a CA
 bundle. Would bundling an ssl library require us to include CA bundle?
 Doesn't the OS ship with one that we could use? Or is that part of the
 problem—the old OS's CA bundle doesn't contain the information needed to
 trust new sites?

 Newer libraries with vulnerabilities seems the lesser of the two evils.
 Early 2000's tools with early CA lists don't help with security much.
 People will just disable the certificate checks, which puts them in a
 worse place.

 And newer libraries that "just work" seems like a worthy usability and
 security goals. People won't have to do things like `wget --no-check-
 certificate` and `curl --insecure`.

 Also, those old libraries and programs built on them are going to cause
 more trouble in the future because they can't do TLS 1.2. A TLS 1.2-only
 configuration is just common place nowadays. For example, GitHub recently
 made the engineering change: https://githubengineering.com/crypto-removal-
 notice/ .

 After the GitHub change things like `wget --no-check-certificate --tlsv1`
 and `curl --insecure --tlsv1` simply will not work. It is technically
 impossible for some sites because of the site's configuration.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:49>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
In reply to this post by MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by noloader):

 Replying to [comment:49 noloader]:
 > ...
 > > > > Yes, I know that merely using a bundled curl that's still linked
 with the system's openssl will not solve all of the problems being
 discussed in this ticket. But I think we should still do it because it
 will solve some of the problems (e.g. comment:30).
 > > >
 > > > I think we all agree that we should not bundle an SSL library with
 MacPorts.
 > >
 > > I do not agree yet. Older OSes need a newer ssl library to fetch from
 some https sites. So can't we provide a newer openssl or libressl on older
 systems, and use Apple's Secure Transport on newer ones? Yes there might
 be security vulnerabilities discovered in the ssl library we bundle, but
 isn't it likely that there are fewer vulnerabilities in it than in the old
 openssl version used on those systems?
 > >
 > > Your objection in comment:3 was that we should not distribute a CA
 bundle. Would bundling an ssl library require us to include CA bundle?
 Doesn't the OS ship with one that we could use? Or is that part of the
 problem—the old OS's CA bundle doesn't contain the information needed to
 trust new sites?
 >
 > Newer libraries with vulnerabilities seems the lesser of the two evils.
 Early 2000's tools with early CA lists don't help with security much.
 People will just disable the certificate checks, which puts them in a
 worse place.
 >
 > And newer libraries that "just work" seems like a worthy usability and
 security goals. People won't have to do things like `wget --no-check-
 certificate` and `curl --insecure`.
 >
 > Also, those old libraries and programs built on them are going to cause
 more trouble in the future because they can't do TLS 1.2. A TLS 1.2-only
 configuration is just common place nowadays. For example, GitHub recently
 made the engineering change: https://githubengineering.com/crypto-removal-
 notice/ .
 >
 > After the GitHub change things like `wget --no-check-certificate
 --tlsv1` and `curl --insecure --tlsv1` simply will not work. It is
 technically impossible for some sites because of the site's configuration.

 This is worth mentioning... TLS 1.3 is in Last Call (LC) status on the TLS
 WG mailing list. I think the future trend will be sites supporting TLS 1.2
 and TLS 1.3 as sites are reconfigured for the new standard. Also see
 [https://www.ietf.org/mail-archive/web/tls/current/msg25387.html Last
 Call: <draft-ietf-tls-tls13-24.txt> (The Transport Layer Security (TLS)
 Protocol Version 1.3) to Proposed Standard].

 Without getting too bogged down in what version of `cacert.pem` to use,
 maybe the better engineering question would be, how can MacPorts ensure
 TLS 1.2 and TLS 1.3 "just work" for users. If you answer that question I
 believe the CA list question will follow without much effort.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:50>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
In reply to this post by MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by kencu):

 Replying to [comment:48 yan12125]:
 > FWIW, I have a branch that bundles LibreSSL + libcurl with macports-base
 at https://github.com/yan12125/macports-base/tree/bundle-curl.
 >
 Well that looks pretty fabulous! All that is missing in the CA
 certificates I guess. Well done.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:51>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
In reply to this post by MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by ryandesign):

 Replying to [comment:48 yan12125]:
 > FWIW, I have a branch that bundles LibreSSL + libcurl with macports-base
 at https://github.com/yan12125/macports-base/tree/bundle-curl.
 >
 > I choose LibreSSL just because it's based on autotools, while OpenSSL
 has a custom Perl-based configure system that once brought some headache
 to me. It should not be difficult to switch LibreSSL back to OpenSSL for
 macports-base.

 Yes thanks for this. There has been some spirited debate about the merits
 of openssl vs libressl, including pointing out that as of High Sierra,
 Apple is distributing libressl in macOS instead of openssl. I would be
 fine with using libressl in MacPorts instead of openssl.

 I see that your branch deletes the MacPorts `--with-curlprefix` configure
 option. We probably want to keep that option for users who want to
 override it, but default it to using the bundled copy.

 And as I mentioned above, on systems new enough to have a Secure Transport
 implementation that supports TLS 1.2, I'd like to use that, and only build
 and use the bundled libressl on older systems. /usr/bin/curl didn't start
 using Secure Transport until Mavericks, but it has been around a lot
 longer than that. I'd have to do some testing to figure out how far back
 it supports TLS 1.2. Ideally we would do a configure script test for
 Secure Transport's capabilities. The only check curl's own configure
 script does is for the existence of
 /System/Library/Frameworks/Security.framework, but that has existed since
 Mac OS X v10.0 so we'd need a more specific check than that.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:52>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

MacPorts
In reply to this post by MacPorts
#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by neverpanic):

 Replying to [comment:47 ryandesign]:
 > I do not agree yet. Older OSes need a newer ssl library to fetch from
 some https sites. So can't we provide a newer openssl or libressl on older
 systems, and use Apple's Secure Transport on newer ones? Yes there might
 be security vulnerabilities discovered in the ssl library we bundle, but
 isn't it likely that there are fewer vulnerabilities in it than in the old
 openssl version used on those systems?

 I do not want to ship a library with potentially known and dangerous
 security vulnerabilities to users. If we use Apple's library and it does
 not work, the blame is on Apple, and users on old OSes cannot expect
 support from Apple. I do not want to create the impression that we support
 this setup.

 Yes, there might be fewer problems, but if there are problems they would
 then be *our* problems. At the moment, they are not.


 > Your objection in comment:3 was that we should not distribute a CA
 bundle. Would bundling an ssl library require us to include CA bundle?
 Doesn't the OS ship with one that we could use? Or is that part of the
 problem—the old OS's CA bundle doesn't contain the information needed to
 trust new sites?

 The latter, the CA bundle would eventually be too old for new certificates
 to validate.

 > #51045 was merely the first example of a curl bug that's fixed in newer
 versions that I found to link to this ticket. There's another cosmetic
 issue I found on Leopard that's been fixed. I'm sure there have been more
 bugs fixed in curl over the years.

 Do you have specific tickets? I'm not convinced that we should bundle
 every library we use.


 Replying to [comment:48 yan12125]:
 > FWIW, I have a branch that bundles LibreSSL + libcurl with macports-base
 at https://github.com/yan12125/macports-base/tree/bundle-curl.

 Thank you, that's helpful for users that want to run this at their own
 risk.

 Replying to [comment:49 noloader]:
 > Newer libraries with vulnerabilities seems the lesser of the two evils.
 Early 2000's tools with early CA lists don't help with security much.
 People will just disable the certificate checks, which puts them in a
 worse place.

 Early 2000's tools are pretty much not going to give you any reasonable
 assumption of security at this point. I'd rather want users to realize
 that then to give them a false sense of security.


 > And newer libraries that "just work" seems like a worthy usability and
 security goals. People won't have to do things like `wget --no-check-
 certificate` and `curl --insecure`.
 >
 > Also, those old libraries and programs built on them are going to cause
 more trouble in the future because they can't do TLS 1.2. A TLS 1.2-only
 configuration is just common place nowadays. For example, GitHub recently
 made the engineering change: https://githubengineering.com/crypto-removal-
 notice/ .

 This is not the point of this ticket. Software installed by MacPorts will
 always use modern libraries and support modern crypto. This is about the
 libraries used by MacPorts itself. So either, your `curl` is MacPorts
 curl, which will work, or it is Apple's curl, which we will also not fix
 with this.

--
Ticket URL: <https://trac.macports.org/ticket/51516#comment:53>
MacPorts <https://www.macports.org/>
Ports system for macOS