Repository navigation
make doctest in CPython has failures #71387
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/Mariatta'
closed_at = <Date 2018-10-30.20:00:05.069>
created_at = <Date 2016-06-03.21:13:27.978>
labels = ['3.7', 'type-bug', 'tests', 'docs']
title = 'make doctest in CPython has failures'
updated_at = <Date 2018-10-30.20:00:05.067>
user = 'https://github.com/JelleZijlstra'bugs.python.org fields:
activity = <Date 2018-10-30.20:00:05.067>
actor = 'mdk'
assignee = 'Mariatta'
closed = True
closed_date = <Date 2018-10-30.20:00:05.069>
closer = 'mdk'
components = ['Documentation', 'Tests']
creation = <Date 2016-06-03.21:13:27.978>
creator = 'JelleZijlstra'
dependencies = ['27202', '27203', '27204', '27205', '27206', '27207', '27208', '27209', '27210', '28860', '34962']
files = ['45747', '45792', '45818', '46456']
hgrepos = []
issue_num = 27200
keywords = ['patch']
message_count = 34.0
messages = ['267166', '267209', '267275', '267278', '267284', '267373', '272315', '272316', '272317', '272355', '282309', '282310', '282330', '282651', '282788', '284499', '284520', '284548', '284560', '284628', '286413', '286463', '286490', '286510', '286511', '286514', '286615', '290167', '290347', '291617', '292431', '328887', '328928', '328943']
nosy_count = 13.0
nosy_names = ['brett.cannon', 'rhettinger', 'terry.reedy', 'ezio.melotti', 'r.david.murray', 'python-dev', 'berker.peksag', 'zach.ware', 'marco.buttu', 'JelleZijlstra', 'mdk', 'Mariatta', 'xtreak']
pr_nums = ['240', '401', '604', '616', '821']
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27200'
versions = ['Python 3.7']
Activity
JelleZijlstra commentedon Jun 3, 2016
I'm going to gather together a patch to fix some of these issues.
JelleZijlstra commentedon Jun 4, 2016
Added a number of more specific issues with patches. I'll add more patches after the current ones are resolved.
berkerpeksag commentedon Jun 4, 2016
Thanks for all of these patches, but I'm -1 on committing them. We already have an extensive test suite and I'm pretty sure that all of these examples are already tested in Lib/test [*]. Learning reST is already hard for contributors and we shouldn't make it harder by introducing new directives like 'testsetup'.
JelleZijlstra commentedon Jun 4, 2016
I would consider this a test of the quality of the documentation, not of the implementation. It's important that the examples in the documentation are actually correct. For example, a doctest in Doc/library/datetime.rst passes an "hours=" keyword argument to datetime.time when "hour=" is correct. It's going to be hard to find such bugs in examples without an automated check like
make doctest.I'm open to simplifying the patches by removing testsetup blocks. For what it's worth though, the docs already contain 17 testsetup blocks.
bitdancer commentedon Jun 4, 2016
Yes, the point here is to test the docs, not the code, and is definitely worthwhile. If we can get make doctest to pass, then the docs buildbot can start running it.
berkerpeksag commentedon Jun 5, 2016
I have also found some markup errors via 'make doctest' so I agree that this can be helpful :)
python-dev commentedon Aug 10, 2016
New changeset f3a04d19e5cb by Zachary Ware in branch '3.5':
Issue bpo-27200: Fix doctests in Doc/library/hashlib.rst
https://hg.python.org/cpython/rev/f3a04d19e5cb
New changeset d0302d8ecbc1 by Zachary Ware in branch 'default':
Issue bpo-27200: Merge with 3.5
https://hg.python.org/cpython/rev/d0302d8ecbc1
zware commentedon Aug 10, 2016
All of the current dependencies have been committed. I still see failures in:
Doc/faq/programming.rst
Doc/library/configparser.rst (also leaves Doc/example.ini)
Doc/library/copyreg.rst
Doc/library/ctypes.rst
Doc/library/datetime.rst
Doc/library/functions.rst
Doc/library/ipaddress.rst
Doc/library/pathlib.rst
Doc/library/re.rst
Doc/library/reprlib.rst
Doc/library/shlex.rst
Doc/library/turtle.rst (possibly only due to lack of usable tkinter)
Doc/library/unittest.mock-examples.rst
Doc/library/unittest.mock.rst
Doc/library/urllib.parse.rst
Doc/library/weakref.rst
For a total of 450 failures (1 in setup code) out of 1726 tests on 3.6. Also, some test leaves 'Doc/foo'.
JelleZijlstra commentedon Aug 10, 2016
Thanks for committing all of these patches! I have local patches for some more tests; I'll try to find some time to upload them.
zware commentedon Aug 10, 2016
Thanks for creating all the patches! If you create further issues for this, feel free to make me nosy.
terryjreedy commentedon Dec 3, 2016
bpo-28860 has a patch for configparser. Jelle, if you have more patches for some of the items listed above, please upload them so Marco can focus on the others.
13 remaining items
marco-buttu commentedon Jan 30, 2017
Thanks Brett :-) Following the suggestion of Ezio to split the patch in more patches (msg284560), and according to msg284628, here is a first patch for just three files. To run their doctests:
$ sphinx-build -b doctest . build/doctest \ library/configparser.rst \ library/ctypes.rst \ library/copyreg.rstrhettinger commentedon Jan 31, 2017
Mariatta, would you please add your review comments to the proposed patches. For the most part, we want the patches to make the minimal changes necessary to pass the doctests. Also, clarity of the examples is more important than making the examples doctestable (i.e. perhaps added sorted() to the output makes it more predictable but it might also obfuscate the point being made).
In general, we would like to have the convenience of doc testing but should be unwilling to commit any atrocities along the way. Ideally, the examples should be written in a way that doesn't garbage them up with <blankline> and whatnot (that is generally distracting to the reader and only serves doctest). Similarly, make sure that clarity and cut-and-pastability don't get lost by adding ... PS2 prompts that uglify the examples. In all cases, the intent of the example writer and the clarity of the example takes precedence over the doctests.
You should try out the patches and make sure the tests run, build the docs and make sure the examples are clean looking from a user point of view. If you find any issues or have questions, use rietveld for discussion (that is the "review" link to the right of the patch).
rhettinger commentedon Jan 31, 2017
Also, please make this 3.7 only. There is no point in going back in time unless an example is found that is actually broken.
zware commentedon Jan 31, 2017
Raymond Hettinger wrote:
Note that Sphinx adds a ">>>" button to the upper right of the example
that, when clicked, makes all of the prompts and output disappear so
that the code (and only the code) can be copied easily. Not having
the ps2 prompts breaks that button.
marco-buttu commentedon Feb 1, 2017
Thank you Raymond for your time. I just want to note that <blankline> does not distrac the reader, because it is not show in the output. Furthermore, to doctest an example we do not need to add the prompt (>>>): we can use the testcode/testoutput Sphinx directives.
berkerpeksag commentedon Mar 24, 2017
New changeset 909a6f6 by Berker Peksag (Marco Buttu) in branch 'master':
bpo-27200: Fix doctests in programming.rst and datetime.rst (#401)
909a6f6
berkerpeksag commentedon Mar 24, 2017
New changeset b2a7c2f by Berker Peksag (Marco Buttu) in branch 'master':
bpo-27200: fix configparser, copyreg and ctypes doctests (#240)
b2a7c2f
berkerpeksag commentedon Apr 13, 2017
New changeset 7b2491a by Berker Peksag (Marco Buttu) in branch 'master':
bpo-27200: Fix pathlib, ssl, turtle and weakref doctests (GH-616)
7b2491a
berkerpeksag commentedon Apr 27, 2017
New changeset e65fcde by Berker Peksag (Marco Buttu) in branch 'master':
bpo-27200: Fix several doctests (GH-604)
e65fcde
tirkarthi commentedon Oct 30, 2018
Sorry to bump this but to add some more context doctest related issues were fixed in bpo-34962 and is now enforced in CI.
terryjreedy commentedon Oct 30, 2018
At the moment, the dependent issues are closed and there is just one open PR on this issue.
Julian, I am nosying you here because you merged bpo-34962, which is properly a dependency of this.
JulienPalard commentedon Oct 30, 2018
Closing PR and issue as already fixed and now enforced by CI.
Thanks again Marco for those already-merged fixes, it helped a lot ending the work at the last PyCon Fr sprints.