Repository navigation
Accept arbitrary files for distutils' upload command #57153
Description
| BPO | 12944 |
|---|---|
| Nosy | @tarekziade, @glyph, @merwok, @asvetlov, @asmeurer, @dstufft |
| Superseder |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/merwok'
closed_at = <Date 2018-01-28.23:28:32.906>
created_at = <Date 2011-09-09.06:32:48.812>
labels = ['3.7', 'easy', 'type-feature', 'library']
title = "Accept arbitrary files for distutils' upload command"
updated_at = <Date 2018-01-28.23:28:32.904>
user = 'https://bugs.python.org/illume'bugs.python.org fields:
activity = <Date 2018-01-28.23:28:32.904>
actor = 'eric.araujo'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2018-01-28.23:28:32.906>
closer = 'eric.araujo'
components = ['Distutils']
creation = <Date 2011-09-09.06:32:48.812>
creator = 'illume'
dependencies = []
files = []
hgrepos = []
issue_num = 12944
keywords = ['easy']
message_count = 11.0
messages = ['143754', '144303', '144382', '144471', '144516', '148874', '178249', '179343', '213221', '311029', '311030']
nosy_count = 10.0
nosy_names = ['frougon', 'exarkun', 'illume', 'tarek', 'glyph', 'eric.araujo', 'asvetlov', 'Aaron.Meurer', 'alexis', 'dstufft']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = '21060'
type = 'enhancement'
url = 'https://bugs.python.org/issue12944'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']
Activity
illume commentedon Sep 9, 2011
We need to specify files to upload to pypi. Otherwise we have to use the web interface.
'Regarding the "setup.py upload" command, this isn't very helpful because it will not upload a package that was already built - for example, a Windows package built on another host while I am trying to release from a Linux desktop. Since automatic package building across a build farm is the only way I can actually build packages for most platforms, this rules out use of the "upload" command.'
See this post for more details:
http://as.ynchrono.us/2011/09/releasing-python-software-is-tedious.html
glyph commentedon Sep 19, 2011
This has been a serious annoyance in the Twisted and pyOpenSSL release processes; it effectively prevents us from timely uploads of win32 binaries of any kind.
merwok commentedon Sep 21, 2011
This was discussed some time ago in a thread starting here: http://mail.python.org/pipermail/catalog-sig/2011-May/003736.html I’ll summarize the important parts below.
Currently, upload will only push files that are products of a command run from the same command line, e.g. “sdist upload”.
If you run “sdist”, check the result, and then do “sdist upload”, the second sdist should be exactly the same, unless your setup.py is buggy, you made changes to your files or you have a computer problem. (I agree that’s a lot of unless). So for distutils, having to run “sdist upload” is not inefficient, and makes you be explicit about the files you want to push, which is IMO good.
A trick can be used to avoid the second sdist to redo all its work: Fist you run “python setup.py sdist --keep-temp”, then you check the sdist, and to upload you call “python setup.py sdist --dry-run upload”. I’m not in favor of adding that trick to the doc, as for normal usage, running sdist twice is okay.
As you may not be aware, distutils is under a feature freeze. To cut a long story short, let’s just say that it’s impossible to gradually improve it without breaking a lot of third-party code, so we only fix bugs and we add features in distutils2, the next gen (included in Python 3.3 and available on PyPI for 2.4-3.2 soon).
I think there are good arguments to improve upload for distutils2, mainly:
I propose a new option, say -f/--file, that takes a filename as argument:
pysetup run upload -f dist/spam-0.2.tar.gz -f dist/spam-0.2.exe
I’d prefer to avoid globs (upload -i 'dist.spam-0.1.*' e.g.) because I want to retain the original design choice of the upload command: the person doing the upload needs to be explicit about what kind of file for what versions and with what Python version to upload.
Does that sounds good?
[-]setup.py upload to pypi needs to work with specified files[/-][+]Accept arbitrary files for packaging's upload command[/+]exarkun commentedon Sep 23, 2011
I'm not sure why it's "run upload" instead of just "upload", but maybe that's the convention in pysetup. Apart from that, this looks like a vast improvement to me. As far as not supporting globs goes, I guess I don't really care either way. If globs aren't supported, then I'll probably end up globbing somewhere else (or constructing the full upload list in some other automated way). The motivation for the complaint is that too many things require manual interaction; I'm not going to use a new, easily automated utility manually. :)
merwok commentedon Sep 25, 2011
merwok commentedon Dec 5, 2011
Someone from the mentoring project has volunteered to work on a patch.
asvetlov commentedon Dec 26, 2012
Close as out of date since packaging has been removed from stdlib.
2 remaining items
merwok commentedon Mar 12, 2014
With the lift of the distutils feature freeze decided at PyCon 2013, this feature request could be done for 3.5.
[-]Accept arbitrary files for packaging's upload command[/-][+]Accept arbitrary files for distutils' upload command[/+]merwok commentedon Jan 28, 2018
I had a patch for this, but now think it would be an attractive nuisance, given that distutils does not always ensure proper HTTPS verification.
Not adding the ability to build sdists/wheels, check them and then upload them will help push people to twine, which doesn’t have all the .pypirc parsing bugs and is a good HTTPS citizen.
I will however commit a patch to clarify the message «no dist file created in earlier command».
[-]Accept arbitrary files for distutils' upload command[/-][+]Better error message for "setup.py upload" missing sdist/bdist[/+]merwok commentedon Jan 28, 2018
A doc ticket already exists, moving this one back to a feature request and closing it.
[-]Better error message for "setup.py upload" missing sdist/bdist[/-][+]Accept arbitrary files for distutils' upload command[/+]