Refactor test generation to reduce diff noise when adding new options - #1060

I have definitely thought the same. This should make the spec test system easier to scale.

I'm wondering if this needs a separate type, or if we could get away with using Options directly, maybe with a function that gives the "test defaults"?

There's actually already a new class of tests merged into branch-0.14, which would be the next release. Could you change your merge target to that and add the new class introduced in #1043?

(We have been doing branching like this, where master is the stable version and breaking changes are first merged into a separate branch. Maybe we should discuss changing that to the more typical approach where master/main/dev is the main integration branch with a static name. Non-breaking changes can then be backported on top of the stable version if deemed necessary.)

I started a thread about branching in the old Zulip:

#pulldown-cmark > Branching strategy

cc @Martin1887 @notriddle

There's actually already a new class of tests merged into branch-0.14, which would be the next release. Could you change your merge target to that and add the new class introduced in #1043?

sure! I'll do that.

I'm wondering if this needs a separate type, or if we could get away with using Options directly, maybe with a function that gives the "test defaults"?

Yeah, I could use Options directly in the generated test, I think this is better too.

@ollpu I have updated the generated tests to use Options directly, and also included the new test class container_extensions.

Thanks, I will take a closer look soon.

We clarified our branching strategy. master/main will be the integration point including breaking changes, so I changed the merge target back.

Sorry, it's been a while. I have an alternative approach in #1122, which requires no changes to the test generator when adding new Options, and there are no global defaults.

No worries! Feel free to close this PR if this is no longer needed.