Repository navigation
pytorch fails signature tests with odd error message #159
Description
This was discussed on slack. There are errors like
FAILED array_api_tests/test_signatures.py::test_func_signature[max] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[mean] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.float32 with fill value 0.0
FAILED array_api_tests/test_signatures.py::test_func_signature[min] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[prod] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[std] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.float32 with fill value 0.0
FAILED array_api_tests/test_signatures.py::test_func_signature[sum] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[var] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.float32 with fill value 0.0
which are not easy to read
The problem is that hypothesis tries to pass an integer to full, but torch only supports a tuple. I think a simple fix would be if hypothesis only tried passing a tuple to full to avoid this issue.
Metadata
Metadata
Assignees
Labels
No labels
Activity
asmeurer commentedon Jan 10, 2023
This error actually occurs throughout the test suite, not just in the signature tests.
honno commentedon Jan 11, 2023
Not a big fan of changes like this, where
hypothesis.extra.array_apiis nominally built with the mindset to happily do whatever the spec allows (such as passing ints as the shape arg). But given this is probably not going to change anytime soon for pytorch, most likely a worthy change beyond just for this test suite, so I'll submit a patch.For onlookers, ideally we could fallback to using
inspect, but alasso we instead pass arrays for un-inspectable functions. They could be manual examples, but at the time I found it convenient to use Hypothesis, as it could easily translate a functions valid input dtypes and create an array from that... using manual examples could be revisited too, esp. if there's some blocker to changing Hypothesis behaviour.