Introducing a minimum ICU version for WebKit

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Introducing a minimum ICU version for WebKit

Ross.Kirsling

Hi everybody,

 

Just sending out an email blast for visibility regarding https://bugs.webkit.org/show_bug.cgi?id=209694.

 

This patch:

  • Upgrades the Mac ICU headers under Source/WTF/icu from ICU 55 to ICU 62, matching Mojave
  • Introduces a minimum ICU version of 60.2 throughout the codebase, as required by GTK for Ubuntu 18.04 LTS

 

As written in the ChangeLog, the immediate motivations are:

  1. To properly establish a minimum ICU version for WebKit as a whole
    (responding to a pain point identified in https://bugs.webkit.org/show_bug.cgi?id=209579)
  2. To support the development of ECMA-402 Intl API features, which JSC is quite behind on
    (and which often boil down to exposing ICU functionality to JavaScript)

 

The only remaining concern of which I am aware is that AppleWin’s ICU headers, stored in WebKitAuxiliaryLibrary.zip, need to be upgraded from ICU 49 to 62 (to match the lib files stored in WebKitSupportLibrary.zip). We do have a potential workaround for this (i.e. having CMake copy the Mac headers to WebKitLibraries/win) but it is feared that this may break Apple-internal builds and we would certainly like to avoid a revert if possible. Any help on this front would be greatly appreciated.

 

Thanks,

Ross


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Ryosuke Niwa-2

On Fri, Apr 3, 2020 at 4:25 PM Kirsling, Ross <[hidden email]> wrote:

Hi everybody,

 

Just sending out an email blast for visibility regarding https://bugs.webkit.org/show_bug.cgi?id=209694.

 

This patch:

  • Upgrades the Mac ICU headers under Source/WTF/icu from ICU 55 to ICU 62, matching Mojave
  • Introduces a minimum ICU version of 60.2 throughout the codebase, as required by GTK for Ubuntu 18.04 LTS

 

As written in the ChangeLog, the immediate motivations are:

  1. To properly establish a minimum ICU version for WebKit as a whole
    (responding to a pain point identified in https://bugs.webkit.org/show_bug.cgi?id=209579)
  2. To support the development of ECMA-402 Intl API features, which JSC is quite behind on
    (and which often boil down to exposing ICU functionality to JavaScript)

 

The only remaining concern of which I am aware is that AppleWin’s ICU headers, stored in WebKitAuxiliaryLibrary.zip, need to be upgraded from ICU 49 to 62 (to match the lib files stored in WebKitSupportLibrary.zip). We do have a potential workaround for this (i.e. having CMake copy the Mac headers to WebKitLibraries/win) but it is feared that this may break Apple-internal builds and we would certainly like to avoid a revert if possible. Any help on this front would be greatly appreciated.


FWIW, I've been told that Brent / Per might be able to help you there but might need some more time due to other more urgent commitments.

- R. Niwa


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Michael Catanzaro-2
In reply to this post by Ross.Kirsling

Any objections to uploading a bundled ICU 60 under Source/ThirdParty?

Seems easier than forcing downstreams to work out bundling themselves.
Most major distros will just stop providing WebKit security updates if
we don't bundle it for them. E.g. this is sure to kill Ubuntu's current
long streak of providing our updates to Ubuntu 16.04.


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Alexey Proskuryakov-4

The license is not BSD or LGPL, so that's one aspect to consider.

Why exactly are distros unwilling to update ICU?

- Alexey


9 апр. 2020 г., в 10:32 AM, Michael Catanzaro <[hidden email]> написал(а):


Any objections to uploading a bundled ICU 60 under Source/ThirdParty?

Seems easier than forcing downstreams to work out bundling themselves. Most major distros will just stop providing WebKit security updates if we don't bundle it for them. E.g. this is sure to kill Ubuntu's current long streak of providing our updates to Ubuntu 16.04.


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev



_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Darin Adler
Here’s my take:

The source code we check into the WebKit repository is there for the convenience of the people *contributing* to WebKit, and is need not be the sole input when building and packaging WebKit for distribution.

Including ICU sources in a GTK WebKit tarball would not necessarily require checking ICU source code into the WebKit repository.

Everything else being equal, adding more files to the WebKit repository does make things less convenient for contributors. I would like us to do what we can to cut down the files, including things like unused PNG files in the LayoutTests directories, any source code that was needed at one time but is not needed in practice any more, including source code in the ThirdParty directory.

— Darin
_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Don.Olmstead
In reply to this post by Michael Catanzaro-2
Hi Michael,

There are a couple problems with checking in a version of ICU.

* Other libraries used by WebKit have dependencies on ICU. For our ports harfbuzz, libxml2, libxslt, libpsl and CFlite all require ICU.
* ICU doesn't come with a CMake build system and its non-trivial to make one. We've largely used this https://github.com/LibCMaker/ICU_CMake_Files to handle building ICU but its use is also why ICU is the library we aren't able to update on a regular cadence.
* We aren't terribly good with updating things in ThirdParty. Sometimes this is because there aren't releases, see gtest. Others because they don't actually have releases, see ANGLE. On top of that there might be local modifications to make things work within WebKit.

Could you possibly give some overview of how WebKit is packaged by Linux distributions? I would have thought they would use flatpack/jhbuild to get the dependencies that the WPE/GTK ports are using especially if those dependencies have their own set of bug and security fixes. My experience with Linux is minimal so some context here would be appreciated.

For our ports we use vcpkg to build and manage dependencies. We host it at https://github.com/WebKitForWindows/WebKitRequirements and have an internal fork for PlayStation. I'm guessing it's probably similar to flatpack/jhbuild in terms of functionality but in our case we just use GitHub releases to have binaries ready.

Perhaps there are better ways for us to approach the requirements that would be beneficial to all ports?

-----Original Message-----
From: webkit-dev <[hidden email]> On Behalf Of Michael Catanzaro
Sent: Thursday, April 9, 2020 10:32 AM
To: Kirsling, Ross <[hidden email]>
Cc: [hidden email]
Subject: Re: [webkit-dev] Introducing a minimum ICU version for WebKit


Any objections to uploading a bundled ICU 60 under Source/ThirdParty?

Seems easier than forcing downstreams to work out bundling themselves.
Most major distros will just stop providing WebKit security updates if we don't bundle it for them. E.g. this is sure to kill Ubuntu's current long streak of providing our updates to Ubuntu 16.04.


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Michael Catanzaro-2
In reply to this post by Alexey Proskuryakov-4

On Thu, Apr 9, 2020 at 11:49 am, Alexey Proskuryakov <[hidden email]>
wrote:
>
> The license is not BSD or LGPL, so that's one aspect to consider.
>
> Why exactly are distros unwilling to update ICU?
>
> - Alexey

Distros would upgrade to newer minor releases of the library, but
updating system packages to new major versions is not allowed in most
stable distros.

Sometimes distros make exceptions (e.g. for WebKitGTK, which is special
due to very high number of CVEs), but ICU does not warrant an
exception. There are probably hundreds of applications using ICU in
distros, if not more. Who knows what might break!


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Michael Catanzaro-2
In reply to this post by Don.Olmstead


On Thu, Apr 9, 2020 at 7:08 pm, "Olmstead, Don" <[hidden email]>
wrote:
> Hi Michael,
>
> There are a couple problems with checking in a version of ICU.
>
> * Other libraries used by WebKit have dependencies on ICU. For our
> ports harfbuzz, libxml2, libxslt, libpsl and CFlite all require ICU.

You're right, it's a bad idea.

> * ICU doesn't come with a CMake build system and its non-trivial to
> make one. We've largely used this
> https://github.com/LibCMaker/ICU_CMake_Files to handle building ICU
> but its use is also why ICU is the library we aren't able to update
> on a regular cadence.

Another good point. Hadn't thought that through far enough.

> * We aren't terribly good with updating things in ThirdParty.
> Sometimes this is because there aren't releases, see gtest. Others
> because they don't actually have releases, see ANGLE. On top of that
> there might be local modifications to make things work within WebKit.

Of course, bundling things in ThirdParty is an absolute last resort.
But when using system packages is no longer possible, what else to do?
I have to support WebKit on RHEL 7, which has ICU 50. Mike from SUSE is
trying to support SLE 12, which has ICU 52. You can see it doesn't add
up. :)

But I didn't object to the ICU 60 proposal because (a) our dependency
policy allows cutting off distros older than Ubuntu 18.04 and Debian
Buster, and (b) I figured we would just bundle it. Instead, a better
approach is probably to come up with downstream patches to revert all
the changes that require ICU 60 and maintain them as long as we can.
Same goes for CMake, but it's going to get a lot harder as WebKit usees
more and more new CMake features. At some point we'll probably need to
try bundling a custom CMake, since eventually that will be easier than
patching WebKit to use ancient CMake.

> Could you possibly give some overview of how WebKit is packaged by
> Linux distributions? I would have thought they would use
> flatpack/jhbuild to get the dependencies that the WPE/GTK ports are
> using especially if those dependencies have their own set of bug and
> security fixes. My experience with Linux is minimal so some context
> here would be appreciated.

No, distros build against their own system packages.

JHBuild is a developer tool. And flatpak is cool, but not used to build
distro packages. Fedora is experimenting with distributing applications
using flatpak, but you can't distribute system libraries that way.

> For our ports we use vcpkg to build and manage dependencies. We host
> it at https://github.com/WebKitForWindows/WebKitRequirements and have
> an internal fork for PlayStation. I'm guessing it's probably similar
> to flatpack/jhbuild in terms of functionality but in our case we just
> use GitHub releases to have binaries ready.
>
> Perhaps there are better ways for us to approach the requirements
> that would be beneficial to all ports?

I don't think there's really anything to do other than to decide how
many old distros we're willing to cut off of updates. Currently the
dependencies policy protects distros for three years. To continue
supporting Ubuntu 16.04 (which reaches EOL in April 2021), it would
need to be extended to five years. Probably not many developers would
be very happy with that at all. But Ubuntu have a five-year lifecycle,
so it's tough beans for WebKit users if we drop support before that
time. RHEL and SLE have 10 and 12-year lifecycles, respectively...
nobody is going to propose supporting that upstream, we just have to
either figure out how to deal with it or cut off updates eventually
once it becomes too hard to build.

I'd support extending our dependencies policy from three years up to
five years... only problem is, five years makes it really hard to use
new C++ features, and we like those. :( Anyway, with the current
dependencies policy, ICU 60 is allowed. We'll just have to figure out
how to deal with it downstream.

Michael


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Michael Catanzaro-2
In reply to this post by Michael Catanzaro-2
On Thu, Apr 9, 2020 at 2:48 pm, Michael Catanzaro
<[hidden email]> wrote:
> Sometimes distros make exceptions (e.g. for WebKitGTK, which is
> special due to very high number of CVEs), but ICU does not warrant an
> exception. There are probably hundreds of applications using ICU in
> distros, if not more. Who knows what might break!

Also, sadly ICU does not maintain a stable API or ABI. So every
application and library using ICU would need to be rebuilt and updated
at the same time. Then the update would break any custom software that
users have using the system ICU. Such an update would go badly...
probably would wind up in tech news once people notice the breakage.

WebKitGTK can be updated because we maintain stable API/ABI.


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

pulkomandy
>Also, sadly ICU does not maintain a stable API or ABI. So every
>application and library using ICU would need to be rebuilt and updated
>at the same time. Then the update would break any custom software that
>users have using the system ICU. Such an update would go badly...
>probably would wind up in tech news once people notice the breakage.
>

Icu can be built with all its abi inside a different namespace for each version, so that multiple versions can live side by side and even be loaded in the same process without conflicts. Are Linux distros not packaging it that way? It makes updates easier as the transition can be made more gradually, at least.

And it would make it possible to ship a newer icu with webkit, while not risking conflicts with the system provided one.

--
Adrien.  
_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Brent Fulgham-4
In reply to this post by Ryosuke Niwa-2
Please note that Per Arne updated the Apple ZIP files to have the correctly aligned ICU libraries, so the Windows bots should have what they need.

I apologize for taking so long to complete that.

Thanks,

-Brent

On Apr 3, 2020, at 4:57 PM, Ryosuke Niwa <[hidden email]> wrote:


On Fri, Apr 3, 2020 at 4:25 PM Kirsling, Ross <[hidden email]> wrote:

Hi everybody,

 

Just sending out an email blast for visibility regarding https://bugs.webkit.org/show_bug.cgi?id=209694.

 

This patch:

  • Upgrades the Mac ICU headers under Source/WTF/icu from ICU 55 to ICU 62, matching Mojave
  • Introduces a minimum ICU version of 60.2 throughout the codebase, as required by GTK for Ubuntu 18.04 LTS

 

As written in the ChangeLog, the immediate motivations are:

  1. To properly establish a minimum ICU version for WebKit as a whole
    (responding to a pain point identified in https://bugs.webkit.org/show_bug.cgi?id=209579)
  2. To support the development of ECMA-402 Intl API features, which JSC is quite behind on
    (and which often boil down to exposing ICU functionality to JavaScript)

 

The only remaining concern of which I am aware is that AppleWin’s ICU headers, stored in WebKitAuxiliaryLibrary.zip, need to be upgraded from ICU 49 to 62 (to match the lib files stored in WebKitSupportLibrary.zip). We do have a potential workaround for this (i.e. having CMake copy the Mac headers to WebKitLibraries/win) but it is feared that this may break Apple-internal builds and we would certainly like to avoid a revert if possible. Any help on this front would be greatly appreciated.


FWIW, I've been told that Brent / Per might be able to help you there but might need some more time due to other more urgent commitments.

- R. Niwa

_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a minimum ICU version for WebKit

Ross.Kirsling

No need for apology—I hugely appreciate the support on this, given the shrouded process of getting those ZIP files updated! :D

 

Ross

 

From: <[hidden email]> on behalf of Brent Fulgham <[hidden email]>
Date: Thursday, April 9, 2020 at 3:16 PM
To: Ryosuke Niwa <[hidden email]>
Cc: "Kirsling, Ross" <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: [webkit-dev] Introducing a minimum ICU version for WebKit

 

Please note that Per Arne updated the Apple ZIP files to have the correctly aligned ICU libraries, so the Windows bots should have what they need.

 

I apologize for taking so long to complete that.

 

Thanks,

 

-Brent



On Apr 3, 2020, at 4:57 PM, Ryosuke Niwa <[hidden email]> wrote:

 


On Fri, Apr 3, 2020 at 4:25 PM Kirsling, Ross <[hidden email]> wrote:

Hi everybody,

 

Just sending out an email blast for visibility regarding https://bugs.webkit.org/show_bug.cgi?id=209694.

 

This patch:

  • Upgrades the Mac ICU headers under Source/WTF/icu from ICU 55 to ICU 62, matching Mojave
  • Introduces a minimum ICU version of 60.2 throughout the codebase, as required by GTK for Ubuntu 18.04 LTS

 

As written in the ChangeLog, the immediate motivations are:

  1. To properly establish a minimum ICU version for WebKit as a whole
    (responding to a pain point identified in https://bugs.webkit.org/show_bug.cgi?id=209579)
  2. To support the development of ECMA-402 Intl API features, which JSC is quite behind on
    (and which often boil down to exposing ICU functionality to JavaScript)

 

The only remaining concern of which I am aware is that AppleWin’s ICU headers, stored in WebKitAuxiliaryLibrary.zip, need to be upgraded from ICU 49 to 62 (to match the lib files stored in WebKitSupportLibrary.zip). We do have a potential workaround for this (i.e. having CMake copy the Mac headers to WebKitLibraries/win) but it is feared that this may break Apple-internal builds and we would certainly like to avoid a revert if possible. Any help on this front would be greatly appreciated.

 

FWIW, I've been told that Brent / Per might be able to help you there but might need some more time due to other more urgent commitments.

 

- R. Niwa

 

_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev

 


_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev