Skip to content

Render field lists in template-format docstrings - #33

Open
hakonhagland wants to merge 3 commits into
OPM:masterfrom
hakonhagland:param-fix
Open

hakonhagland wants to merge 3 commits into
OPM:masterfrom
hakonhagland:param-fix

Conversation

@hakonhagland

Copy link
Copy Markdown
Collaborator

Every :param: and :type: on the generated simulators API page is printed as literal text instead of becoming a parameter table. On the current master build there are 45 occurrences of the raw string :param in simulators.html. A method renders like this:

advance(report_step: int) → None
Advances the simulation to a specific report step.
:param report_step: Target report step to advance to.
:type report_step: int

The opm-common page is unaffected, which turns out to be the clue to the cause.

Split docstrings on a real newline (commit 1)

  • process_template_docstrings split each docstring with doc.split('\\n'). In Python source '\\n' is a two-character string — a backslash followed by an n — not a newline.
  • json.load has already converted the \n escapes in the JSON file into real newline characters by that point, so nothing in the docstring matches, split returns a single piece, and the whole docstring is appended to the ViewList as one line.
  • docutils parses line by line, so a :param ...: in the middle of that single line never starts a line and is never recognized as a field list.
  • Fixed by splitting on '\n', which is what the flat-format path a few lines below already does. That is why only one of the two pages was affected: docstrings_common.json is in the flat format and goes down the correct path, while docstrings_simulators.json is in the template format and goes down this one.
  • The comment on the line said "Handle escaped newlines", which is the mistaken assumption itself — the escapes do not survive json.load — so it is removed rather than kept.
  • Both loops had the same separator, the constructor loop and the method loop, so both are changed.

Introduced in 069db36 ("Add template support to sphinx_ext_docstrings.py"), whose diff contains all three splits together — the class-level one correct, these two not.

Add a regression test (commit 2)

  • tests/test_sphinx_ext_docstrings.py builds a minimal Sphinx project in a temporary directory and asserts that a :param: in the JSON becomes a parameter table rather than visible text.
  • Two tests, because constructors and methods go through separate loops and each had its own copy of the bug.
  • Needed a new fixture: the existing tests/files/docstrings_simulators.json is in the older flat format, so it exercises the code path that was already correct and cannot catch this. docstrings_simulators_template.json is added alongside it — a minimal template-format file with one simulator, one constructor and one method.
  • Both tests fail on the parent of commit 1 and pass on it. The existing test_view_docs.py continues to pass.

Remove a debug print (commit 3)

  • read_doc_strings printed the JSON path it was about to open on every call, straight to standard output. Across two directives and three branch builds that is six bare absolute paths in the build log, interleaved with sphinx-versioned's own output and unlabeled.
  • Removed rather than converted to a logger call, since the path is already fixed by conf.py. If it is wanted as a diagnostic, sphinx.util.logging's logger.verbose() would put it behind sphinx-build -v instead of printing unconditionally — happy to do that instead if reviewers prefer.

Effect on the built documentation

  • Literal :param occurrences in simulators.html: 45 → 0, with 57 real field-list blocks in their place.
  • No new Sphinx warnings. The same tree built with and without the change reports build succeeded, 45 warnings either way, and the warning list is identical apart from one line number that moves because the generated reStructuredText is now split into lines rather than being one long line.
  • That moving warning is Inline literal start-string without end-string, three times. It is not introduced here — it is a three-backtick inline literal in the setupMpi docstring in opm-simulators, appearing three times because one template entry expands into the three simulator classes. It is being fixed separately in opm-simulators; this change only makes the surrounding docstring parse, which is what lets Sphinx notice it at all.

Docstrings coming from the template format were passed to docutils as a
single line, so reStructuredText field lists in them were never parsed.
The published API page showed the markup itself:

  :param init: Whether to call MPI_Init() or not.

instead of a parameter table.

The cause is the separator in

  for line in doc.split('\\n'):  # Handle escaped newlines

In Python source '\\n' is a two-character string, a backslash followed
by an n. json.load has already turned the \n escapes in the JSON file
into real newline characters, so nothing in the docstring matches and
split returns one piece: the whole docstring, newlines and all, appended
to the ViewList as a single line. docutils parses line by line, so the
":param" never starts a line and is treated as ordinary text.

Split on a real newline instead, as the flat-format path a few lines
below already does. That path was unaffected, which is why the
opm-common page has always rendered its parameter tables correctly while
the opm-simulators page did not.

Both the constructor loop and the method loop had the same separator.
Builds a minimal Sphinx project from a small template-format docstrings
file and asserts that a ":param" in the JSON becomes a parameter table
rather than visible text, for both a method and a constructor since they
go through separate loops.

The existing tests/files/docstrings_simulators.json is in the older flat
format, which uses the code path that was already correct, so it cannot
exercise this. A small template-format file is added alongside it.

Both tests fail on the previous commit's parent and pass on it.
The function printed the JSON path it was about to open, on every call,
to the build's standard output. With two directives across three
branch builds that is six bare absolute paths in the log, interleaved
with sphinx-versioned's own output and with no indication of what they
are.

Removed rather than converted to a logger call, since the path is
already determined by conf.py and visible there. If it is wanted as a
diagnostic, sphinx.util.logging's logger.verbose() would put it behind
sphinx-build -v instead of printing it unconditionally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant