Linux 3: code should avoid using sys.platform == 'linux2'

@malemburg

BPO	12326
Nosy	@malemburg, @loewis, @warsaw, @birkenfeld, @doko42, @amauryfa, @pitrou, @vstinner, @larryhastings, @benjaminp, @jwilk, @djc, @ezio-melotti, @merwok, @bitdancer, @davidmalcolm, @sandrotosi, @akheron
Files	linux3-v2.patch configure_linux2.python3.2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-09-05.20:39:30.835>
created_at = <Date 2011-06-13.14:34:41.657>
labels = ['type-bug', 'release-blocker']
title = "Linux 3: code should avoid using sys.platform == 'linux2'"
updated_at = <Date 2011-10-04.14:24:53.642>
user = 'https://bugs.python.org/neologix'

bugs.python.org fields:

activity = <Date 2011-10-04.14:24:53.642>
actor = 'barry'
assignee = 'none'
closed = True
closed_date = <Date 2011-09-05.20:39:30.835>
closer = 'vstinner'
components = []
creation = <Date 2011-06-13.14:34:41.657>
creator = 'neologix'
dependencies = []
files = ['22917', '22946']
hgrepos = []
issue_num = 12326
keywords = ['patch']
message_count = 145.0
messages = ['138251', '138252', '138253', '138254', '138255', '138271', '138294', '138295', '138298', '138299', '138300', '138306', '138308', '138527', '138582', '138597', '138711', '138730', '138757', '138759', '138816', '138817', '138823', '138824', '139166', '139208', '139224', '139243', '139248', '140012', '140017', '140061', '140062', '140063', '140064', '140065', '140394', '140397', '140684', '140703', '140708', '140710', '140801', '140802', '140830', '140831', '140939', '140944', '140949', '140955', '140985', '141088', '141651', '141659', '142195', '142196', '142197', '142219', '142226', '142272', '142290', '142291', '142297', '142298', '142313', '142344', '142345', '142346', '142350', '142353', '142354', '142356', '142358', '142379', '142380', '142381', '142382', '142384', '142385', '142387', '142389', '142394', '142396', '142398', '142401', '142406', '142407', '142408', '142409', '142410', '142417', '142418', '142419', '142420', '142425', '142440', '142441', '142444', '142446', '142450', '142451', '142453', '142457', '142459', '142460', '142461', '142463', '142482', '142490', '142493', '142498', '142499', '142500', '142524', '142531', '142539', '142541', '142570', '142571', '142574', '142579', '142593', '142605', '142614', '142615', '142628', '142635', '142678', '142694', '142697', '142698', '142700', '142707', '142711', '142762', '142766', '142769', '143455', '143470', '143479', '143557', '143561', '143562', '144889', '144890']
nosy_count = 25.0
nosy_names = ['lemburg', 'loewis', 'barry', 'georg.brandl', 'doko', 'amaury.forgeotdarc', 'gagern', 'foom', 'pitrou', 'vstinner', 'larry', 'benjamin.peterson', 'jwilk', 'djc', 'ezio.melotti', 'eric.araujo', 'Arfrever', 'Retro', 'r.david.murray', 'dmalcolm', 'sandro.tosi', 'rosslagerwall', 'python-dev', 'petri.lehtinen', 'Ramchandra Apte']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12326'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

Linus recently decided that the next Linux kernel version would 3.0 [1].
As a consequence, sys.platform (which is actually MACHDEP) will be 'linux3' on machines running Linux 3 kernels, and tests checking explicitely against 'linux2' will either break and won't run.
A quick grep through the code base returns only a couple problematic places, but this should definitely be fixed.
For information, here's a - probably incomplete - list of such occurrences:

"""
./Lib/test/test_logging.py: if sys.platform in ('linux2', 'darwin'):
./Lib/test/test_sysconfig.py: sys.platform = 'linux2'
./Lib/test/regrtest.py: 'linux2':
./Lib/test/test_socket.py: if sys.platform == 'linux2':
./Lib/test/test_tarfile.py: if sys.platform == "linux2":
./Lib/distutils/tests/test_bdist_rpm.py: if sys.platform != 'linux2':
./Lib/distutils/tests/test_bdist_rpm.py: if sys.platform != 'linux2':
./Lib/distutils/tests/test_util.py: sys.platform = 'linux2'
./Lib/packaging/tests/test_config.py: inotify (0.0.1); sys.platform == 'linux2'
./Lib/packaging/tests/test_config.py: "inotify (0.0.1); sys.platform == 'linux2'"]
./setup.py: if (platform in ('linux2', 'freebsd4', 'freebsd5', 'freebsd6',
"""

[1] http://thread.gmane.org/gmane.linux.kernel/1147415

Why should be used instead?

sys.platform.startswith('linux')
os.uname()[0] == 'Linux'
platform.system() == 'Linux'
other?

Tests like sys.platform in ('linux2', 'darwin') can be replaced by sys.platform in ('linux2', 'linux3', 'darwin'). We will have to patch this test again for Linux 4.

I would expect changing sys.platform will also break a lot of third-party code. Perhaps sys.platform can still be 'linux2' under Linux 3.x? After all, there's no significant change that deserves changing sys.platform.

sys.platform comes from Py_GetPlatform() which comes from PLATFORM define. On Linux, this define comes from Makefile: MACHDEP variable which comes from configure. Finally, MACHDEP is defined by:

ac_sys_system=`uname -s`
if test "$ac_sys_system" = "AIX" \
-o "$ac_sys_system" = "UnixWare" -o "$ac_sys_system" = "OpenUNIX"; then
	ac_sys_release=`uname -v`
else
	ac_sys_release=`uname -r`
fi
ac_md_system=`echo $ac_sys_system |
		   tr -d '/ ' | tr '[A-Z]' '[a-z]'`
ac_md_release=`echo $ac_sys_release |
		   tr -d '/ ' | sed 's/^[A-Z]\.//' | sed 's/\..*//'`
MACHDEP="$ac_md_system$ac_md_release"

I would expect changing sys.platform will also break a lot of third-
party code.

Maybe, but this would be an application bug.

Here's sys.platform current implementation:

const char *
Py_GetPlatform(void)
{
    return PLATFORM;
}

And here's the documentation, from http://docs.python.org/c-api/init.html

"""
Return the platform identifier for the current platform. On Unix, this is formed from the “official” name of the operating system, converted to lower case, followed by the major revision number"""

So it's actually documented.

Perhaps sys.platform can still be 'linux2' under Linux 3.x? After
all, there's no significant change that deserves changing
sys.platform.

Sounds like a recipe for confusion.

I suggest to change sys.platform to 'linux' in future releases (3.3). For bugfix releases (2.7, 3.2), I'd keep it as 'linux2' even on Linux 3. For security-only releases (2.6, 3.1), no change should be made.

This change is reasonable for the long term. But it *will* break a lot of code.

This change is reasonable for the long term. But it *will* break a lot of code.

[If you favor a specific change, please indicate what that is. I'm
assuming you support my proposal for the moment :-]

I agree it will break a lot of code, but it's also somewhat urgent
because we will get 'linux3' if we don't act, which will also break
a lot of code (but more subtly, since people testing their code may
do so on Linux 2, only to get bug reports that it breaks on "some"
Linux systems). I'm sure Linus Torvalds is fully aware of the possible
consequences of the version change, and just accepted the breakage
that this would cause.

It's important that we set a policy before the Linux distributions
do (which may end up choosing different policies). We don't actually
have to *release* this change quickly, since Linux distributions
who release 3.x kernels will fix their Python packages themselves.

The change to sys.platform=='linux' would break code even on current platforms.
OTOH, we have sys.platform=='win32' even on Windows 64bit; would this favor keeping 'linux2' on all versions of Linux as well?

The change to sys.platform=='linux' would break code even on current platforms.

Correct. Compared to introducing 'linux3', I consider this the better
change - it likely breaks earlier (i.e. when porting to Python 3.3).

OTOH, we have sys.platform=='win32' even on Windows 64bit; would this
favor keeping 'linux2' on all versions of Linux as well?

While this has better compatibility, it's also a constant source of
irritation. Introducing 'win64' would have been a worse choice (just
as introducing 'linux3' would: incompatibility for no gain, since
the distinction between win32 and win64, from a Python POV, is
irrelevant). Plus, Microsoft dislikes the term Win64 somewhat, and
rather wants people to refer to the "Windows API".

I personally disliked 'linux2' when it was introduced, for its
incompatibilities. Anticipating that, some day, we may have 'Linux 4',
and so on, I still claim it is better to fix this now. We could even
come up with a 2to3 fixer for people who dual-source their code.

I'm sure Linus Torvalds is fully aware of the possible
consequences of the version change, and just accepted the breakage
that this would cause.

Any application relying on sys.platform == 'linux2' is already broken.
It's exactly the same if an application checks for 'freebsd6' or
'openbsd4' : if you want to check for a specific operating system,
there's already POSIX' struct utsname sysname field, i.e. uname()[0].

Martin v. Löwis wrote:

Martin v. Löwis <[email protected]> added the comment:

> The change to sys.platform=='linux' would break code even on current platforms.

Correct. Compared to introducing 'linux3', I consider this the better
change - it likely breaks earlier (i.e. when porting to Python 3.3).

> OTOH, we have sys.platform=='win32' even on Windows 64bit; would this
> favor keeping 'linux2' on all versions of Linux as well?

While this has better compatibility, it's also a constant source of
irritation. Introducing 'win64' would have been a worse choice (just
as introducing 'linux3' would: incompatibility for no gain, since
the distinction between win32 and win64, from a Python POV, is
irrelevant). Plus, Microsoft dislikes the term Win64 somewhat, and
rather wants people to refer to the "Windows API".

I personally disliked 'linux2' when it was introduced, for its
incompatibilities. Anticipating that, some day, we may have 'Linux 4',
and so on, I still claim it is better to fix this now. We could even
come up with a 2to3 fixer for people who dual-source their code.

I think we should consider adding a better mechanism and just
keep the old mechanism for determining sys.platform in place
(letting it break on Linux to raise awareness) and add a new better
attribute along the lines of what Martin suggested:

sys.system == 'linux', 'freebsd', 'openbsd', 'windows', etc.
(without version)

and a new

sys.system_info == system release info (named) tuple similar to
sys.version_info

to query a specific system version.

As already noted, direct sys.platform testing already breaks for
OpenBSD, FreeBSD and probably a few other OSes as well with
every major OS release, so the Linux breakage is not really new
in any way.

> The change to sys.platform=='linux' would break code even on current platforms.

Correct. Compared to introducing 'linux3', I consider this the better
change - it likely breaks earlier (i.e. when porting to Python 3.3).

FWIW, I also agree that sys.platform == 'linux' would be the better
choice. It seems there's little point having the kernel's major version
there. It's both disruptive and useless (because features are introduced
at any point, not just at new major versions).

Martin’s sys.platform = 'linux' sounds good to me too.

Something like:

diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -699,20 +699,21 @@ always available.
    This string contains a platform identifier that can be used to append
    platform-specific components to :data:`sys.path`, for instance.
 
-   For Unix systems, this is the lowercased OS name as returned by ``uname -s``
-   with the first part of the version as returned by ``uname -r`` appended,
-   e.g. ``'sunos5'`` or ``'linux2'``, *at the time when Python was built*.
-   Unless you want to test for a specific system version, it is therefore
-   recommended to use the following idiom::
+   For Unix systems, except on Linux, this is the lowercased OS name as
+   returned by ``uname -s`` with the first part of the version as returned by
+   ``uname -r`` appended, e.g. ``'sunos5'`` or ``'linux2'``, *at the time when
+   Python was built*.  Unless you want to test for a specific system version,
+   it is therefore recommended to use the following idiom::
 
-      if sys.platform.startswith('linux'):
-          # Linux-specific code here...
+      if sys.platform.startswith('freebsd'):
+          # Freebsd-specific code here...
 
    For other systems, the values are:
 
    ================

===========================
System :data:platform value
================ ===========================

Linux 'linux2'
Windows 'win32'
Windows/Cygwin 'cygwin'
Mac OS X 'darwin'

?

I don't think that I need a :versionchanged:`2.7.3`.

I think the doc patch should mention that:

it's 'linux2' also on Linux 3;
why it's not 'linux3';
The why can be done in a footnote and explain that Linux 3 doesn't introduce new major features and that changing the string to 'linux3' would have just broken needlessly code that was checking for sys.platform == 'linux2'.
It should probably also mention that in future versions (i.e. 3.3+) this string has been changed to be just 'linux' and that keep checking for sys.platform == 'linux2' is therefore discouraged even after this change.

I've updated 3.2 docs in e11b4c945f7e (currently in the release clone, will be merged to upstream after the release of 3.2.2.)

Please commit a similar change to the 2.7 branch.

Please delete Lib/plat-linux3 directories on 2.7 and 3.2 branches.

New changeset 265da863d017 by Victor Stinner in branch '3.2':
Issue bpo-12326: sys.platform is now always 'linux2' on Linux
http://hg.python.org/cpython/rev/265da863d017

New changeset e11b4c945f7e by Georg Brandl in branch '3.2':
Update sys.platform doc for bpo-12326.
http://hg.python.org/cpython/rev/e11b4c945f7e

New changeset d95c4b030eac by Victor Stinner in branch '2.7':
Issue bpo-12326: Remove plat-linux3 directory
http://hg.python.org/cpython/rev/d95c4b030eac

New changeset cb47cf5138a4 by Victor Stinner in branch '3.2':
Issue bpo-12326: Remove plat-linux3 directory
http://hg.python.org/cpython/rev/cb47cf5138a4

New changeset 0fe571d43317 by Victor Stinner in branch '2.7':
Update sys.platform doc for bpo-12326.
http://hg.python.org/cpython/rev/0fe571d43317

I removed Lib/plat-linux3 from Python 2.7 and 3.2, and updated doc of Python 2.7. I close this issue (again).

I have a better idea... Why don't we change the "linux2" string into just "linux". That way we will never run into this kind of issue, even in the future when Linux kernel version 4 is going to exist. Any thoughts on this?

On Oct 04, 2011, at 01:03 PM, Boštjan Mejak wrote:

I have a better idea... Why don't we change the "linux2" string into just
"linux". That way we will never run into this kind of issue, even in the
future when Linux kernel version 4 is going to exist. Any thoughts on this?

Python 3.3 already sets sys.platform to 'linux'. It can't be done for older
versions due to backward compatibility.

cmake: sys.platform should be 'linux' instead of 'linux2'

Navigation Menu

Repository navigation

Linux 3: code should avoid using sys.platform == 'linux2' #56535

125 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Linux 3: code should avoid using sys.platform == 'linux2' #56535

Description

Activity

neologix commented on Jun 13, 2011

vstinner commented on Jun 13, 2011

pitrou commented on Jun 13, 2011

vstinner commented on Jun 13, 2011

neologix commented on Jun 13, 2011

loewis commented on Jun 13, 2011

amauryfa commented on Jun 14, 2011

loewis commented on Jun 14, 2011

amauryfa commented on Jun 14, 2011

loewis commented on Jun 14, 2011

neologix commented on Jun 14, 2011

malemburg commented on Jun 14, 2011

pitrou commented on Jun 14, 2011

merwok commented on Jun 17, 2011

125 remaining items

vstinner commented on Aug 22, 2011

ezio-melotti commented on Aug 22, 2011

birkenfeld commented on Sep 3, 2011

Arfrever commented on Sep 3, 2011

python-dev commented on Sep 4, 2011

python-dev commented on Sep 5, 2011

python-dev commented on Sep 5, 2011

vstinner commented on Sep 5, 2011

Retro commented on Oct 4, 2011

warsaw commented on Oct 4, 2011

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions