Repository navigation
Remove the major version from sys.platform #57004
Description
| BPO | 12795 |
|---|---|
| Nosy | @malemburg, @loewis, @vstinner, @ezio-melotti |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2011-10-14.01:09:42.580>
created_at = <Date 2011-08-20.16:23:15.361>
labels = []
title = 'Remove the major version from sys.platform'
updated_at = <Date 2011-10-14.01:09:42.579>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2011-10-14.01:09:42.579>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2011-10-14.01:09:42.580>
closer = 'vstinner'
components = []
creation = <Date 2011-08-20.16:23:15.361>
creator = 'vstinner'
dependencies = []
files = ['22964']
hgrepos = []
issue_num = 12795
keywords = ['patch']
message_count = 14.0
messages = ['142540', '142542', '142552', '142564', '142704', '142706', '142709', '142712', '142723', '142726', '142730', '142737', '142738', '145495']
nosy_count = 6.0
nosy_names = ['lemburg', 'loewis', 'vstinner', 'nadeem.vawda', 'ezio.melotti', 'Arfrever']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue12795'
versions = ['Python 3.3']
Activity
vstinner commentedon Aug 20, 2011
sys.platform contains the major system version. If you test the sys.platform value (e.g. sys.platform == 'linux2'), your program doesn't work anymore with the new system major version (e.g. Linux 3). This problem is common with NetBSD, OpenBSD and FreeBSD. You don't have the problem if you test the prefix of sys.platform (e.g. sys.platform.startswith('linux')), but this code pattern is rare.
Because of the release of Linux 3, it was proposed in bpo-12326 to remove the major version from sys.platform. It is already done for Cygwin and Darwin. Example of new sys.platform values: 'linux', 'freebsd', 'hp-ux', ... (instead of 'linux2', 'freebsd8', hp-ux11', ...).
I don't know if "osf1" becomes "osf". I don't know if "irix646" becomes "irix6" or just "irix"?
What about sys.platform=="win32"? Should it be truncated to "win"? Many tests use already sys.platform.startswith("win"). And what about sys.platform=="java"? It would be nice to be consistent (e.g. never have digits in sys.platform).
Without the major version, it's much easier when you only care of the system name: you can use a dictionary with the name for the key (it's used in regrtest.py with my patch).
Another example :
------------------------
if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3',
'Darwin1.2', 'darwin',
'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5',
'freebsd6', 'freebsd7', 'freebsd8',
'bsdos2', 'bsdos3', 'bsdos4',
'openbsd', 'openbsd2', 'openbsd3', 'openbsd4'):
------------------------
becomes:
------------------------
if sys.platform in ('netbsd', 'freebsd', 'openbsd', 'bsdos',
'Darwin1.2', 'darwin'):
------------------------
('Darwin1.2'? WTF?)
This issue follows my previous commit 50f1922bc1d5:
Issue bpo-12326: don't test the major version of sys.platform
Use startswith, instead of ==, when testing sys.platform to support
new platforms like Linux 3 or OpenBSD 5.
I chose to keep sys.platform in distutils and packaging because these modules are supposed to work on older Python versions (e.g. packaging will be backported to Python 2.4-3.2).
Attached patch implements this issue. It requires platform.major(): see issue bpo-12794. The patch require to rerun "autoconf".
vstinner commentedon Aug 20, 2011
To have an idea of how much code has to be changed for the sys.platform change, there is the diffstat :
Lib/packaging/command/build_ext.py | 3 -
Lib/packaging/compiler/unixccompiler.py | 6 +-
Lib/packaging/tests/test_config.py | 4 -
Lib/sysconfig.py | 12 ++---
Lib/test/fork_wait.py | 2
Lib/test/regrtest.py | 71 ++++++++++++++------------------
Lib/test/support.py | 2
Lib/test/test_asyncore.py | 2
Lib/test/test_fcntl.py | 9 +---
Lib/test/test_locale.py | 2
Lib/test/test_logging.py | 2
Lib/test/test_multiprocessing.py | 2
Lib/test/test_os.py | 6 +-
Lib/test/test_posix.py | 2
Lib/test/test_signal.py | 11 ++--
Lib/test/test_socket.py | 7 ---
Lib/test/test_tarfile.py | 2
Lib/test/test_tempfile.py | 2
Lib/test/test_threading.py | 14 +++---
Lib/unittest/test/test_break.py | 4 -
configure.in | 17 ++-----
setup.py | 19 ++++----
26 files changed, 105 insertions(+), 121 deletions(-)
loewis commentedon Aug 20, 2011
I think this change should be much much smaller in scope. It was (nearly) agreed to drop the major version if the system is Linux. There is no consensus (that I'm aware of) to drop the major OS version for all systems.
So I would propose to do this *only* selectively, and *only* if experts of the platform request such a change. For example, hpux9, hpux10, and hpux11 are *very* different operating systems; it's not clear that users are helped if we drop the major version for them. Also, if HP-UX 12 ever gets released, it is probably again different.
I'm not sure sure about the BSDs: it is my understanding that they follow the tradition of only adding major changes to major OS releases, which would be in favor of keeping the status quo for them. OTOH, it may also be that the impact of the major version on Python and Python applications is too small so that applications would be better helped with dropping the major version.
vstinner commentedon Aug 20, 2011
At the beginning, I thaught that it would be better to remove the major version from sys.platform on all platforms. When I started to write the patch, I expected to change only 2 or 3 files. But the patch is now huge ("26 files changed, 105 insertions(+), 121 deletions(-)"), which means that it will break most third party projects.
Ok, I (now) agree. So let's close this issue.
ezio-melotti commentedon Aug 22, 2011
Having the major version in sys.platform only for a few selected platforms doesn't seem too consistent for me.
If linux 4 will introduce major changes, will we have to look for the major version of freebsd in sys.platform and for the linux major version elsewhere?
That said, removing the major version for the other platforms might introduce unnecessary breakage, but if it's ever going to happen, I think it's better to do it now for all the platforms and not waiting for the experts to speak and change it in later versions, adding even more inconsistency.
malemburg commentedon Aug 22, 2011
Moved the discussion here from bpo-12326:
Actually, with that reasoning we would need to reintroduce the
version for Mac OS, and even go a step further and add the minor
version number as well, since since major changes have happened on Mac OS
with every single minor release for the last couple of years.
IMO, a better approach is to split the information in two parts:
sys.platform, which only specifies the platform name on which
Python was built (uname -s)
sys.platform_build_version, which provides the full platform
version (uname -r; either as string or as tuple or both -
that would have to be hashed out)
It is obvious that the version number in sys.platform often
doesn't really get used (see Victor's example). In such cases,
having access to just the build platform name is better.
In other cases, where you do need to know the version (e.g.
on Mac OS X), you can then easily get if from the new attribute,
and including the minor and even patch level release version
details (e.g. to determine whether Python was compiled with
a Windows specific service pack in place or not).
vstinner commentedon Aug 22, 2011
FreeBSD or OpenBSD release major version frequently, something like one
per year, or one per two years. FreeBSD and OpenBSD developers knows
that for years, and Python programs use sys.platform.startswith() for
these OSes.
For Linux, it's different. Linux 2.0 was released in 1997 and 3.0 in
2011: it took 14 years to change the major version. It don't think that
any program working on Linux was prepared for this change: see bpo-12326
history to have an idea on the problem. It looks like
sys.platform=='linux3' breaks most programs testing sys.platform
(including Python itself because of Lib/plat-linux2/ directory).
If you want the OS name, use platform.system() or os.uname()[0].
If you want the OS version, use platform.release(). If you want the OS
version as a tuple, hum... see the issue bpo-12794.
malemburg commentedon Aug 22, 2011
STINNER Victor wrote:
Victor, you are constantly mixing build time information with
runtime information. Those are two different types of information
and we should regard them as such.
sys.platform refers to build time information, so the platform
module won't help.
The version of the build platform is important to know, but
adding it to sys.platform is not necessarily the right
thing to do, since adding just the major version is often
not enough (see Mac OS X) and can sometimes lead to breakage
due to frequent new releases of OSes (see FreeBSD).
Given that the various OSes use different schemes for versioning
and backwards compatibility, a single string cannot possibly
cover all aspects.
Separating the information into OS name and version is a much
more future proof approach.
vstinner commentedon Aug 22, 2011
When I wrote my patch, I realized that sys.platform is used to get the runtime information, which is wrong. See the tests in Lib/test/test_*.py: they use sys.platform to check features of the running OS (kernel). It is mostly to test FreeBSD major version, and it looks like Python is build on the same FreeBSD major version that it is running on. Maybe because on FreeBSD, most programs are compiled instead of using prebuild packages (pkgadd).
malemburg commentedon Aug 22, 2011
STINNER Victor wrote:
Right. FreeBSD doesn't ship prebuilt binaries for packages -
it uses a very elegant "ports" collection, where everything gets
compiled on the target machine:
So you don't really run into the issue of using a older Python build
on the system, unless you have an application which ships Python
along with the application package.
vstinner commentedon Aug 22, 2011
FreeBSD does ship prebuild Python packages.
malemburg commentedon Aug 22, 2011
The FreeBSD base distribution (which has the binaries
and, AFAIK, the only binaries that ship with FreeBSD) doesn't
include Python. It's in the ports collection, though,
and for those you have the option of installing prebuilt
binaries of the packages via pkg_add.
Still, that's not exactly the FreeBSD way of doing things :)
See the big warning in the pkg_add man page, various discussions
on the net and the pros/cons in the handbook:
http://www.freebsd.org/doc/handbook/ports-overview.html
But in general you are right: if you use system provided Python
packages, chances are that your runtime OS version will be
identical or very close to the build time OS version.
vstinner commentedon Aug 22, 2011
I asked on #freebsd-fr: even with pkg_add, the installed Python is compiled on the same major version than the running major version. So the problem is "hidden" (it doesn't affect) on FreeBSD.
vstinner commentedon Oct 14, 2011
While working on bpo-12326, it's realized that many modules and applications rely on sys.platform. Change this value should be decided for each platform because it breaks backward compatibility. Linux was a special case and now I don't think that we should drop the major version from sys.platform on all platforms. Let's close this issue as "wont fix".