Unexpected type 'unicode' for os.path.join() with (Any, str) args

@Michael0x2a

The revealed type for the os.path.join call in the program below is unicode, even though I'd expect either Any or str, as the arguments have types Any and str:

import os
def f(): pass
reveal_type(os.path.join(f(), 'x'))  # builtins.unicode

Here's a self-contained example:

# a.pyi
from typing import overload
@overload
def join(__p1: bytes, *p: bytes) -> bytes: ...
@overload
def join(__p1: unicode, *p: unicode) -> unicode: ...

# program.py
from a import join
def f(): pass
reveal_type(join(f(), 'x'))  # builtins.unicode

Run as mypy -2 program.py.

@Michael0x2a Do you have time to have a look at this? This is causing issues in internal Dropbox repos. If you are busy, I can try to fix this. This was apparently introduced by f61c2ba.

I am looking at this now.

I think the issue is that due to the way we handle promotions in mypy, bytes is actually technically considered to be a subtype of unicode -- so the union math logic concludes that it's valid to return unicode instead of Any since the former is more precise.

Not sure what the best way to handle this is -- I'd personally prefer just removing that promotion altogether, but that's a very invasive and difficult-to-implement change. Alternatively, we could special-case this in the union math somehow, but that would make it inconsistent with the rest of mypy. E.g. this currently typechecks, for example:

def expects_unicode(x):
    # type: (unicode) -> None
    pass

b = b'foo'  # type: bytes

expects_unicode(b)

I guess other options include:

Modifying the "is the last return type a supertype of the previous ones" checks so promotions are disabled altogether.
Removing that check altogether so we always return Any.

My original motivation for requesting it was that this example should work

@overload
def f(x: List[int]) -> List[int]: ...
@overload
def f(x: List[Any]) -> List[Any]: ...

arg: Any
f(arg)  # should be List[Any], _not_ just Any

Also now I think that we should really check both ways, because the logic is that replacing a precise type with Any should ideally not cause new errors. For example:

def f() -> int: ...

@overload
def g(x: int) -> int: ...
@overload
def g(x: float) -> float: ...

x: int = g(f())  # this passes, but will fail if I remove the annotation from 'f'

I will now prepare a PR to return last type only if it is compatible both ways with all other return types (like in the first example with List[Any]).

Fix overload interaction with Any (#5233)

Navigation Menu

Repository navigation

Unexpected type 'unicode' for os.path.join() with (Any, str) args #5232

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Unexpected type 'unicode' for os.path.join() with (Any, str) args #5232

Description

Activity

ilevkivskyi commented on Jun 18, 2018

Michael0x2a commented on Jun 18, 2018

Michael0x2a commented on Jun 18, 2018

ilevkivskyi commented on Jun 18, 2018

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions