WPT tests and runtime-enabled features

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

WPT tests and runtime-enabled features

Ali Juma
Hi all,

While a feature is in development behind a runtime-enabled flag, is there a way to write tests in web-platform-tests that enable the feature when they're run as part of LayoutTests? Adding "<!--- webkit-test-runner [enableFeatureName=true] -->" tags would technically work but seems a bit questionable.

The point of this would be to be able to use web platform tests as regression tests while developing a feature, allowing the tests to be upstreamed sooner and saving having to write them outside of web-platform-tests first and then move them over later.

Thanks,
Ali

_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: WPT tests and runtime-enabled features

youenn fablet
Hi Ali,

Yes this is something we should have a solution for. Right now, we have two ways:
- Enabling the runtime-enabled flag only in DumpRenderTree/WebKitTestRunner
- Updating WebKit testharnessreport.js to call internal APIs for some/all WPT tests

Ideally, we should be able to set additional configuration information to WPT tests without modifying them.
A configuration file alongside the WPT test might be feasible and could supersede the <!--- webkit-test-runner [enableFeatureName=true] --> technique.
Folder-based hierarchical configuration might be nice but I am unsure whether that would meet the complexity tradeoff bar.
Nothing yet started there so any help would be greatly appreciated!
    y

Le ven. 2 juin 2017 à 13:37, Ali Juma <[hidden email]> a écrit :
Hi all,

While a feature is in development behind a runtime-enabled flag, is there a way to write tests in web-platform-tests that enable the feature when they're run as part of LayoutTests? Adding "<!--- webkit-test-runner [enableFeatureName=true] -->" tags would technically work but seems a bit questionable.

The point of this would be to be able to use web platform tests as regression tests while developing a feature, allowing the tests to be upstreamed sooner and saving having to write them outside of web-platform-tests first and then move them over later.

Thanks,
Ali

_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: WPT tests and runtime-enabled features

Philip Jägenstedt
FWIW, in Chromium, if we had an off-by-default feature that we wanted to test using wpt, we would probably use the https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/VirtualTestSuites mechanism.

On Sat, Jun 3, 2017 at 1:03 AM, youenn fablet <[hidden email]> wrote:
Hi Ali,

Yes this is something we should have a solution for. Right now, we have two ways:
- Enabling the runtime-enabled flag only in DumpRenderTree/WebKitTestRunner
- Updating WebKit testharnessreport.js to call internal APIs for some/all WPT tests

Ideally, we should be able to set additional configuration information to WPT tests without modifying them.
A configuration file alongside the WPT test might be feasible and could supersede the <!--- webkit-test-runner [enableFeatureName=true] --> technique.
Folder-based hierarchical configuration might be nice but I am unsure whether that would meet the complexity tradeoff bar.
Nothing yet started there so any help would be greatly appreciated!
    y

Le ven. 2 juin 2017 à 13:37, Ali Juma <[hidden email]> a écrit :
Hi all,

While a feature is in development behind a runtime-enabled flag, is there a way to write tests in web-platform-tests that enable the feature when they're run as part of LayoutTests? Adding "<!--- webkit-test-runner [enableFeatureName=true] -->" tags would technically work but seems a bit questionable.

The point of this would be to be able to use web platform tests as regression tests while developing a feature, allowing the tests to be upstreamed sooner and saving having to write them outside of web-platform-tests first and then move them over later.

Thanks,
Ali

_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev



_______________________________________________
webkit-dev mailing list
[hidden email]
https://lists.webkit.org/mailman/listinfo/webkit-dev
Loading...