Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

Stub method being overshadowed by module #690

Closed
Closed
Stub method being overshadowed by module#690

Description

@malmaud
Jonathan Malmaud (malmaud)
opened on Mar 2, 2019

With the new PyTorch type stubs (pytorch/pytorch#12500), the stub for the tensor method seems to not be utilized by intellisense (example below). I think this is related to the existence of a module called tensor.py in the PyTorch package - if I delete that file, then the stub is utilized. Maybe the language server is somehow getting confused between whether torch.tensor should be inferred to refer to the module or the method defined in the stub. In this case, it should refer to the stub.

The stub for tensor indicates it has a Tensor return type, but we see vscode not showing Tensor methods here:

screen shot 2019-03-02 at 5 00 39 pm

Meanwhile the similar as_tensor method, which has the same return type, is working:
screen shot 2019-03-02 at 5 01 03 pm

Here are the stub definitions:

def tensor(data: Any, dtype: Optional[_dtype]=None, device: Union[_device, str, None]=None, requires_grad: bool=False) -> Tensor: ...

def as_tensor(data: Any, dtype: _dtype=None, device: Optional[_device]=None) -> Tensor: ...

I don't have a minimal example at the moment. PyTorch is a big package and in my simple attempts to reduce the problem to a minimal version, VSCode did the right thing, so it's hard for me to tell what exactly is going wrong.

This is with Python 3.7, extension version 2019.5.5433 and VSCode 1.31.1.

Activity

  1. jakebailey commented on Mar 5, 2019

    @jakebailey
    Jake Bailey (jakebailey)
    on Mar 5, 2019
    Member

    I'm checking this against the LS in master (very different than the released one) which does a much better job with stubs, and can at least do:

    image

    However, neither torch.tensor nor torch.as_tensor are things which appear in the torch module's __all__ list, which means that they aren't being exposed:

    __all__ = [
    'typename', 'is_tensor', 'is_storage', 'set_default_tensor_type',
    'set_rng_state', 'get_rng_state', 'manual_seed', 'initial_seed',
    'save', 'load', 'set_printoptions', 'chunk', 'split', 'stack', 'matmul',
    'no_grad', 'enable_grad', 'rand', 'randn',
    'DoubleStorage', 'FloatStorage', 'LongStorage', 'IntStorage',
    'ShortStorage', 'CharStorage', 'ByteStorage',
    'DoubleTensor', 'FloatTensor', 'LongTensor', 'IntTensor',
    'ShortTensor', 'CharTensor', 'ByteTensor', 'Tensor',
]

    https://github.com/pytorch/pytorch/blob/master/torch/__init__.py#L19

    Are you on a different version of Torch which exposes those functions? I'm testing with:

  2. jakebailey commented on Mar 5, 2019

    @jakebailey
    Jake Bailey (jakebailey)
    on Mar 5, 2019
    Member

    Oh. Gross.

    from torch._C import *

__all__ += [name for name in dir(_C)
            if name[0] != '_' and
            not name.endswith('Base')]

  3. malmaud commented on Mar 6, 2019

    @malmaud
    Jonathan Malmaud (malmaud)
    on Mar 6, 2019
    Author

    Thanks for looking into this!

    Ya, it's because of dynamic code like that that a type stub was added to aid code completion. You'll see a __init__.pyi in the 'torch' package folder, and that stub explicitly declares a def tensor(...).

    I opened this issue because I thought the language server would use the information in the stub, and it does seem to with the exception of the tensor method. It's on the nightly pytorch that I see as_tensor is being suggested as a completion while tensor is not, despite them both being in the stub file.

    I installed the nightly with pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html.

  4. jakebailey commented on Mar 6, 2019

    @jakebailey
    Jake Bailey (jakebailey)
    on Mar 6, 2019
    Member

    The version of torch I have installed doesn't have those two functions in the stub, so I'll have to switch up and see. With some __all__ filtering removed, I can see loads of Torch functions already.

    Just to say so, the visual difference between -> Tensor and -> 'Tensor' is visual only; the tooltips in the new language server are essentially copies from the declaration to the output, but the analysis itself parses out the string and does something with it.

  5. jakebailey commented on Mar 6, 2019

    @jakebailey
    Jake Bailey (jakebailey)
    on Mar 6, 2019
    Member

    Alright, testing with the nightly build of torch:

    image

    image

    image

    image

    I'm going to close this issue, as it's working as expected in the new LS (which should be released into daily and beta soon).

  6. Jake Bailey (jakebailey)
    closed this as completedon Mar 6, 2019

  7. malmaud commented on Mar 6, 2019

    @malmaud
    Jonathan Malmaud (malmaud)
    on Mar 6, 2019 via email
    Author

  8. jakebailey commented on Mar 6, 2019

    @jakebailey
    Jake Bailey (jakebailey)
    on Mar 6, 2019
    Member

    0.2.16 is now available on beta channel. You can set this in your VS Code settings.json to get it:

    "python.analysis.downloadChannel": "beta"

  9. malmaud commented on Mar 6, 2019

    @malmaud
    Jonathan Malmaud (malmaud)
    on Mar 6, 2019
    Author

    Confirmed that switching to the beta channel fixes things on my end too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions