Repository navigation
Adding platform information to stdlib/VERSIONS #11260
Description
It might be a good idea to add platform availability information to stdlib/VERSIONS, instead of using the sys.platform hack we're currently using in the source files. This has the same rationale for introducing VERSIONS in the first place:
- Remove platform checks and the required indentation from stub files.
- Enables clearer type checker messages (e.g. "Module does not exist on this platform", instead of "Item X doesn't exist in module Y")
- Generally an unambiguous way of indicating that a module doesn't exist.
- Edit: Collect all information about module availability in one place.
One way to do this is an HTTP/E-Mail header-like key/value syntax:
abc: 3.0-; platforms=linux,darwin
def: 3.7-3.10; platforms=!windows,!linux
ghi: 3.9-
This would allow future extension when type checkers are equipped to ignore unknown keys:
abc: 3.0-; header1=foo; header2=bar
Do we need a way do indicate that a certain platform only supports a certain module starting from a certain Linux version?
Activity
srittau commentedon Jan 8, 2024
Cc @JukkaL @ilevkivskyi @rchen152 @erictraut
erictraut commentedon Jan 8, 2024
Is there a concrete reason for doing this now, or is this this just theoretical? If it's just theoretical, I'd prefer to hold off until there's a proven need for this facility.
srittau commentedon Jan 8, 2024
The need is the same as for initially adding VERSIONS: Addressing a deficiency in how type checkers can detect whether stubs are available in a certain Python version. But the concrete trigger was #6749/#11241: Having a solid way to say "This package doesn't exist on this platform in the stdlib" would enable a third-party stubs package to reliably add that package without interfering with a stdlib package.
srittau commentedon Jan 8, 2024
Also just to clarify: There would be no need to immediately implement this on the type checker side and there is also no pressing need on typeshed's side to remove the version_info checks when this gets implemented. This is as possible timeline:
line = line.split(";")[0]).Then type checkers have the option to implement the new-style checks. I expect typeshed to keep the current in-stub checks for at least as long as the major type checkers haven't implemented the new-style checks.
erictraut commentedon Jan 9, 2024
OK, makes sense. I've added provisional support for the above proposal in pyright. If we end up modifying the proposal, I will adjust accordingly.
I presume that if there are one or more platforms listed without
!characters, that all other platforms are assumed to be unsupported.Added support for python/typeshed#11260.
Akuli commentedon Jan 10, 2024
I'm not convinced that this is a good idea. Currently the syntax of the VERSIONS file is super simple, and I'd prefer to leave it that way instead of inventing a complex language that is only used to describe one file.
Also, the VERSIONS file is supposed to contain info about standard library specific things. To me, "Python version added" is clearly a standard-library thing while "supported platforms" could apply to any stub file. So if we do this for standard-library stubs, it would be tempting to do something similar for third-party stubs, which would introduce even more complexity for little benefit.
To me, the only convincing reason is better type checker error messages, because users would see it.
srittau commentedon Jan 10, 2024
That's a good point. A generic way to tell type checkers "this module is not available on that platform" would be preferable. That said, amending VERSIONS is better than the status quo.
I don't think keeping VERSIONS super simple should be a design goal, as it's not user facing. Also, to me, the proposed format is simple enough, and only used for a few modules.
Akuli commentedon Jan 21, 2024
A bit weird, but maybe type checkers could just check if everything in the stub file (except imports) is defined inside a
if sys.platform ==statement that is being skipped?Advantages:
Disadvantages:
cursessituationrchen152 commentedon Jan 23, 2024
Echoing @Akuli, adding platform information to
stdlib/VERSIONSfeels like a bit of an awkward fit, since it's not stdlib-specific. A couple other ideas off the top of my head:PLATFORMSfile. The stdlib could have one, and any stub distribution that needs to communicate platform info could add its own.assert sys.platform != "windows"at the top. (I think I've seen this suggestion go by before, so apologies if it's already been discussed and rejected.)srittau commentedon Jan 24, 2024
Ideally, both platform and Python version availability would be encoded in the stub file itself instead of an external file. But for now I think Rebecca's suggestion to use a
PLATFORMSfile sounds like the best idea, as it's named unambiguously (as opposed toVERSIONS) and backwards-compatible.JelleZijlstra commentedon Jan 24, 2024
I think I'd prefer to implement Rebecca's second suggestion. This would not require a new special file format, and it would generalize to third-party packages more easily.
Suggested specification: The first line of code in a stub file may be an
assertstatement. It may contain the same set of conditions as are supported inifstatements in stubs (checks onsys.platformandsys.version_info). If this assert is present, type checkers should evaluate it and if the condition statically evaluates to false, they should treat the stub file as if it does not exist.erictraut commentedon Jan 24, 2024
That won't work for pyright and pylance. We need to determine which type stubs are applicable without loading, parsing and analyzing every type stub in the environment. This is important for language server features like completion suggestions with auto-import.
A static file like
PLATFORMSor an extension toVERSIONSwould work much better for our use case.