Replace OverflowError with ValueError when calculating length of range objects > PY_SIZE_MAX #80733

Activity

1. tonybaloney commented on Apr 8, 2019

   

   tonybaloneymannequin

   on Apr 8, 2019

   MannequinAuthor

   More actions

   When calculating length of range() objects that have an r->length > PY_SIZE_MAX, the underlying PyLong_AsSsize_t() function will raise an OverflowError:

   >>> a = list(range(2**256))
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   OverflowError: Python int too large to convert to C ssize_t
   >>> a = range(2**256)
   >>> len(a)
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   OverflowError: Python int too large to convert to C ssize_t

   This is expected behaviour, but to the average user, who won't know what ssize_t is, or what this has to do with Python int, the user message is confusing and OverflowError is the symptom but not the cause. The cause is that the length sent to range was in a value too large to calculate.

   This patch changes OverflowError to ValueError to hint to the user that the value sent to the range object constructor is too large.

   >>> a = list(range(2**256))
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   ValueError: Range object too large to calculate length (Overflow Error)
   >>> a = range(2**256)
   >>> len(a)
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   ValueError: Range object too large to calculate length (Overflow Error)
2. tonybaloney

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Apr 8, 2019
3. serhiy-storchaka commented on Apr 8, 2019

   

   serhiy-storchaka

   on Apr 8, 2019

   Member

   More actions

   For large but smaller than the C limit ranges the list constructor raises a MemoryError. It should raise the same error for larger ranges.

   Raising an OverflowError in range.__len__ is legitimate.
4. zooba commented on Apr 12, 2019

   

   zooba

   on Apr 12, 2019

   Member

   More actions

   We should at least have consistent error messages:

   >>> class O:
   ...  def __len__(self):
   ...   return 2**100
   ...
   >>> o=O()
   >>> len(o)
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   OverflowError: cannot fit 'int' into an index-sized integer

   I'd argue for replacing 'int' here with the rendered value, but I think OverflowError is the right type. Mentioning "C ssize_t" is the problem.

   As for the list constructor, I'd be okay with chaining a MemoryError here, provided the OverflowError sticks around. But in this context a MemoryError is "recoverable" while an OverflowError very likely indicates a programming error, so we shouldn't hide it from the user.
5. tonybaloney

   self-assigned this

   on May 9, 2019
6. serhiy-storchaka

   closed this as completedon May 9, 2019
7. ezio-melotti

   transferred this issue fromon Apr 10, 2022