Skip to content

Specify Options in spec tests using bitflags from_str - #1122

Open
ollpu wants to merge 6 commits into
pulldown-cmark:mainfrom
ollpu:spec-test-options
Open

ollpu wants to merge 6 commits into
pulldown-cmark:mainfrom
ollpu:spec-test-options

Conversation

@ollpu

@ollpu ollpu commented Jul 27, 2026 •

Copy link
Copy Markdown
Collaborator

I feel that the current system where we define ad-hoc test types example_deflists etc. and pass those around as booleans is starting reach its limits. Instead, we could use bitflags::parser::from_str to define the Options directly.

For convenience, I added two ways to set options. The effective options are the union of both.

Global options can be added to a file by writing on the first line:

Parser options: `ENABLE_ABC | ENABLE_XYZ`

Further options can be written in the lang string:

```````````````````````````````` example ENABLE_YUV

I also made it so that the test generator struct Spec keeps track of the current line in the file and names the test like blockquotes_tags_test_0005_l44, so it's easier to locate.

@ollpu
ollpu force-pushed the spec-test-options branch from 1e8c511 to 88113bd Compare July 27, 2026 11:38
@Martin1887

Copy link
Copy Markdown
Collaborator

Nice!

ollpu added 4 commits August 16, 2026 18:40
Global options can be added to a file by writing on the first line:

Parser options: `ENABLE_ABC | ENABLE_XYZ`

Further options can be written in the lang string:

```````````````````````````````` example ENABLE_YUV
This should make it easier to find a failing test.
@ollpu
ollpu force-pushed the spec-test-options branch from a120576 to 4fcaa4e Compare August 16, 2026 16:11
Comment thread pulldown-cmark/specs/cjk_friendly_emphasis.txt Outdated
@ollpu ollpu changed the title RFC: Specify Options in spec tests using bitflags from_str Specify Options in spec tests using bitflags from_str Aug 16, 2026
@ollpu
ollpu requested a review from notriddle August 16, 2026 16:18

@notriddle notriddle left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a fix to the heading_attrs.txt problem here: notriddle@26306b4

You can add it to your branch by running these commands:

$ git fetch https://github.com/notriddle/pulldown-cmark spec-test-options
$ git merge FETCH_HEAD

Comment thread pulldown-cmark/specs/heading_attrs.txt Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.example?

Have these tests ever actually been run?! 🤦

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn. When changing the test collector, I was kind of worried that it's easy to accidentally write a non-test, since it needs the exact number of backticks and example.

Would it make sense to warn about lines that have a large number of backticks but don't match, or something like that?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Accepting anything with more than 8 backticks is another alternative.

We could also warn about things with a hamming distance lower than 4 for instance or matching a regular expression similar to the accepted values to avoid things like .example, but probably not so useful and prone to false positives.

Comment thread pulldown-cmark/specs/cjk_friendly_emphasis.txt Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants