Repository navigation
Build 3.x documentation using python3.x #54433
Description
| BPO | 10224 |
|---|---|
| Nosy | @loewis, @brettcannon, @birkenfeld, @terryjreedy, @cben, @abalkin, @ned-deily, @ezio-melotti, @merwok, @bitdancer, @asvetlov, @durban, @cjerdonek, @ericsnowcurrently, @zware |
| Dependencies | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2014-04-14.19:24:45.466>
created_at = <Date 2010-10-28.22:16:37.845>
labels = ['type-feature', 'build', 'docs']
title = 'Build 3.x documentation using python3.x'
updated_at = <Date 2014-07-02.03:49:21.282>
user = 'https://github.com/abalkin'bugs.python.org fields:
activity = <Date 2014-07-02.03:49:21.282>
actor = 'ezio.melotti'
assignee = 'none'
closed = True
closed_date = <Date 2014-04-14.19:24:45.466>
closer = 'georg.brandl'
components = ['Build', 'Documentation']
creation = <Date 2010-10-28.22:16:37.845>
creator = 'belopolsky'
dependencies = ['16471']
files = ['22080', '29525']
hgrepos = []
issue_num = 10224
keywords = ['patch']
message_count = 31.0
messages = ['119819', '119821', '119851', '119854', '119856', '119861', '119863', '119864', '119910', '120765', '120766', '136629', '136637', '136814', '137971', '138297', '140252', '175768', '183706', '183838', '184838', '184872', '184874', '184884', '184892', '184893', '190396', '216161', '216185', '216187', '222066']
nosy_count = 19.0
nosy_names = ['loewis', 'brett.cannon', 'georg.brandl', 'terry.reedy', 'cben', 'belopolsky', 'syeberman', 'ned.deily', 'ezio.melotti', 'eric.araujo', 'r.david.murray', 'asvetlov', 'zbysz', 'daniel.urban', 'chris.jerdonek', 'docs@python', 'tshepang', 'eric.snow', 'zach.ware']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue10224'
versions = ['Python 3.4', 'Python 3.5']
Activity
abalkin commentedon Oct 28, 2010
Opening a ticket to track the progress of getting python 3.x build its own documentation.
So far I have installed Sphinx from http://bitbucket.org/birkenfeld/sphinx, but I get the following error:
There is a syntax error in your configuration file: invalid syntax (patchlevel.py, line 71)
Did you change the syntax from 2.x to 3.x?
abalkin commentedon Oct 28, 2010
I had some success after running 2to3 on Doc/conf.py and Doc/tools. The build succeeded, but I have not compared the output yet. Running doctest, however, still reported lots of errors and hang in turtle.rst.
abalkin commentedon Oct 29, 2010
Incremental build was not that successful. After editing Doc/faq/programming.rst, I get the following in the error log:
birkenfeld commentedon Oct 29, 2010
I fixed that bug in Sphinx rev 49747f5b0c70 (which I will push as soon as bitbucket is up again); incremental build now works for me.
birkenfeld commentedon Oct 29, 2010
In r85910, I ported the Python-specific modules in tools/ so that they don't need to be 2to3-converted.
Now you can basically easy_install Sphinx on 3.1 and run its sphinx-build without touching the Doc/ tree.
loewis commentedon Oct 29, 2010
Did you know break building the 3.x documentation with Python 2? If so, please revert that change.
birkenfeld commentedon Oct 29, 2010
Nope, these files run just as fine in Python 2.
birkenfeld commentedon Oct 29, 2010
(The usual build process via Makefile still uses Python 2, and that won't change for 3.2.)
abalkin commentedon Oct 29, 2010
Would you consider changing "doctest" make target to check out sphinx trunk (or 3.x compatible release) and run it with the py3k python?
This should not affect regular doc builds and running doctest under anything other than the current version of python makes very little sense.
Note that I am still targeting documentation fixes for 3.2. See bpo-10225.
abalkin commentedon Nov 8, 2010
As discussed on #python-dev, building 3.x documentation using python3.x will require Sphinx 1.1 which will not be released in time for 3.2.
I am adding bpo-10225 as a dependency because I feel that fixing actual errors in ReST doctests is more important than upgrading documentation tool chain. Hopefully this will bring more eyes to that issue.
This said, I believe it is important for python-dev to "eat their own dogfood" and make 3.2 self-hosting. In other words, if we want to convince users that 3.x is ready, we should not require 2.x in the build process.
Fortunately, Doc/tools is not shipped with python releases, so the transition can be made at any time regardless of python release schedule.
birkenfeld commentedon Nov 8, 2010
I agree that 3.2 should eventually be independent of a 2.x Python. Since the "port to 3.x" part of Sphinx 1.1 is basically done, I might just do an early alpha release and use that for Doc/tools before 3.2 final.
8 remaining items
zware commentedon Mar 9, 2013
I have offered an alternative fix for that issue as a part of bpo-17386.
cben commentedon Mar 21, 2013
After a lot of experimentation, this worked for me to build AND doctest using the local python:
cd Doc/
../python -m venv toolsenv
curl http://python-distribute.org/distribute_setup.py | toolsenv/bin/python
toolsenv/bin/easy_install pip
toolsenv/bin/pip install Sphinx
# EDIT conf.py: add 'toolsenv' to exclude_trees.
toolsenv/bin/sphinx-build -b html -d build/doctrees -D latex_paper_size= . build/html
toolsenv/bin/sphinx-build -b doctest -d build/doctrees . build/doctest
The tool versions I got:
HTML: http://dl.dropbox.com/u/132894668/python3-Doc-build/html/index.html
The HTML has no significant differences vs the output of "make html".
It's mostly sphinx html tweaks. The only user-visible diff I noticed is s/v3.4.0a0/3.4.0a0/.
The doctests are pretty horrible - 559 failed out of 1954! (attached)
But it's better than 900+ failed with python 2 ("make doctest") and 700+ with python 3.2 that I originally tried via "apt-get install python3-sphinx"...
So let's update the tools ASAP, and start using PYTHON=../python.
The doctests won't get any love unless "make doctest" does the right thing out of the box.
Should I write a Makefile patch using the venv/install from pypi approach?
Or could somebody upload new versions to http://svn.python.org/projects/external/?
(Lack of access there was the reason I went the venv way.
IMHO the ease of future upgrades outweighs the concern of not depending on external sites, but I don't feel strongly about it.)
ezio-melotti commentedon Mar 21, 2013
I looked at this issue over the past few days and discussed briefly with Georg about it. The 3 main issues are:
IMHO for 1) it would be better to provide Python3-ready code. For 2) it would also be nice to get rid of the SVN dependency for the docs so if we are going to use a repo it might be better to make a new one on HG. For 3), I'm not sure it makes sense to use repos at all if we don't want the history. If we don't, we would have to put the files somewhere and write some code to download and extract the zip/tarball. OTOH, if we do, we will be able to use hg to clone and update the repo easily, and, if we still decide not to download the history, we can just use the zip/tarballs already provided by hg.python.org.
birkenfeld commentedon Mar 21, 2013
A problem with using automatic tarballs from hg.python.org is that they take server load to generate, every time someone builds the docs from a new checkout.
ezio-melotti commentedon Mar 21, 2013
Apparently they are cached, and unless the cache expires often, the server doesn't need to regenerate the tarball too often. However I don't know the details of the caching, and if they can be tweaked.
birkenfeld commentedon Mar 21, 2013
Ok, that sounds good then.
BTW, for 3.3+ would it make sense instead to create a virtualenv and directly install the required versions from pypi?
brettcannon commentedon Mar 21, 2013
I guess if we are willing to write a mini-pip to download the latest release directly from PyPI we could. Since there are no C extensions we don't have to care about the lack of header files in Include if we are going to build from the repo build directly. Or are you more thinking that we just require 3.3 externally to build and not rely on the repo build unless absolutely necessary? Just wondering about when the venv environment copies instead of symlinks the binary.
cben commentedon May 31, 2013
I was only thinking of 3.4, which will have venv and a pip bootstrapper.
Is changing the doc build / doctest in scope for minor releases of 3.3 (or even earlier)?
The commands I listed (using setup_distribute.py) also work with 3.3.
(But they're unsecure -- https://bitbucket.org/tarek/distribute/issue/374/)
3.2 is harder as it doesn't even have builtin venv.
ned-deily commentedon Apr 14, 2014
For 3.4.1 and 3.5, the Doc/Makefile no longer explicitly vendors sphinx and its dependencies (via svn), rather it assumes that an externally-supplied sphinx-build is available, for example, via a venv and pip. And the current versions of sphinx and its dependencies are all supported on current Py3's. So can this issue be closed, Georg?
birkenfeld commentedon Apr 14, 2014
Yep, thanks for the reminder.
merwok commentedon Apr 14, 2014
Do we need to change Doc/Makefile to have “PYTHON = python3” or even “PYTHON = ./python” (so doctests run with the development version)?
ezio-melotti commentedon Jul 2, 2014
Éric, I think you should open a separate issue for this.