Hello,

Since Prince supports a lot of CSS2.1 and CSS3 modules (even CSS4 modules), I think it would be great to gather and report Prince test results of several CSS test suites into implementation reports.

I have done so in the past for the CSS3 Writing modes implementation report:

[css-writing-modes-3] Prince 10 rev 4 implementation report
http://lists.w3.org/Archives/Public/public-css-testsuite/2015Sep/0007.html

and it is my will and intent to continue to do so when latest stable release of Prince improves its support of CSS3 writing modes specification.

My suggestion here is about other CSS test suites like CSS3 background and borders (603 tests), CSS2.1 test suite (10329 tests), etc..

All the test suites are listed here:

CSS Test Harness
http://test.csswg.org/harness/

Consider this as a suggestion and not at all as a task-to-do. I know that taking all the tests of a test suite takes a lot of time.

Gérard Talbot

Thanks for the suggestion!

6 ½ years later.

When latest stable release of Prince improves its support of CSS3 writing modes specification, I would like to be notified about it.

It would be nice if we could gather and report Prince's test results of the CSS3 Writing modes test suite.

Gérard Talbot

We do hope to revisit writing modes once we have support for CSS grid.

Okay, Mike. No problem.

All the best to you and yours for 2024

> It would be nice if we could gather and report Prince's test results of the CSS3 Writing modes test suite.

We currently make use of parts of the WPT suite, but the automation doesn't work for us much of the time particularly when it comes to interactive features. Unfortunately, many of the tests rely on dynamic changes even when they aren't related to the feature being tested (e.g., to check a range of property values), and we don't have a way to effectively run these tests except by converting the last version of the document into a ref test. There's other issues like this, too. Ultimately, we have to apply our own test criteria based on the intent of the test author (to the extent possible), as well as some pragmatic considerations.

Note that the documented test assumptions include that the device is interactive, so the above should not be surprising - the WPT suite doesn't actually apply to Prince.

The WPT documentation also has this:
"Tests should not rely on unrelated features if doing so causes failures in the latest stable release of Apple Safari, Google Chrome, or Mozilla Firefox."

This wouldn't make for a fair comparison between those browsers and other UAs, but again I think the creators of WPT didn't intend it as a benchmark suite, only a test suite. The percentage of tests passed isn't a particularly meaningful measure.

So a report could be informative, but it would need a caveat about being non-interactive, and other assumptions, and probably should avoid the percentages.

Mark.

> many of the tests rely on dynamic changes even when they aren't related to the feature being tested (e.g., to check a range of property values)

Mark,

That (check a range of property values, dynamic changes) is true but with recent tests (after 2018 or so) and in various CSS specs. My post was (6½ years ago) focusing and still is focusing on writing-modes test suite. Back in 2014, 2015 and 2016, I wrote about half of those tests in that writing-modes test suite. Most of them are also in this page:

http://www.gtalbot.org/BrowserBugsSection/CSS3WritingModes/

> the device is interactive, so the above should not be surprising - the WPT suite doesn't actually apply to Prince.

The tests of the CSS test suites (more or less 98% of them) are now automated (screen shots compared of test with its reference file) but this was _not_ the case back in 2014.

The whole WPT suite includes a lot of specs, not just CSS. Some CSS test suites can still apply to non-interactive medium and to web-aware softwares like Prince.

I posted this

https://lists.w3.org/Archives/Public/public-css-testsuite/2015Sep/0007.html

and I created

http://www.gtalbot.org/BrowserBugsSection/CSS3WritingModes/implementation-report-Prince-10-rev-4-20150904.txt

Gérard Talbot

> The whole WPT suite includes a lot of specs, not just CSS. Some CSS test suites can still apply to non-interactive medium and to web-aware softwares like Prince.

Perhaps, then, the WPT authors can be persuaded to remove that assumption, at least for some specs? It may not have been present 6 ½ years ago, but it is now (including in the writing modes suite, apparently, under "uncommon assumptions").

If they don't want to do that, they presumably have a reason for targeting interactive agents only. I couldn't say what that reason is - it's really up to them, not us, to decide to broaden their scope to include PDF formatters like Prince. It would be nice if they did, though