This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Replace OverflowError with ValueError when calculating length of range objects > PY_SIZE_MAX
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: anthonypjshaw Nosy List: anthony shaw, anthonypjshaw, serhiy.storchaka, steve.dower
Priority: normal Keywords: patch

Created on 2019-04-08 04:38 by anthony shaw, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12720 closed anthonypjshaw, 2019-04-08 04:39
Messages (3)
msg339589 - (view) Author: anthony shaw (anthony shaw) Date: 2019-04-08 04:38
When calculating length of range() objects that have an r->length > PY_SIZE_MAX, the underlying PyLong_AsSsize_t() function will raise an OverflowError:

>>> a = list(range(2**256))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C ssize_t
>>> a = range(2**256)
>>> len(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C ssize_t

This is expected behaviour, but to the average user, who won't know what ssize_t is, or what this has to do with Python int, the user message is confusing and OverflowError is the symptom but not the cause. The cause is that the length sent to range was in a value too large to calculate. 

This patch changes OverflowError to ValueError to hint to the user that the value sent to the range object constructor is too large.

>>> a = list(range(2**256))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Range object too large to calculate length (Overflow Error)
>>> a = range(2**256)
>>> len(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Range object too large to calculate length (Overflow Error)
msg339592 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-04-08 07:25
For large but smaller than the C limit ranges the list constructor raises a MemoryError. It should raise the same error for larger ranges.

Raising an OverflowError in range.__len__ is legitimate.
msg340080 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-04-12 16:27
We should at least have consistent error messages:

>>> class O:
...  def __len__(self):
...   return 2**100
...
>>> o=O()
>>> len(o)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer

I'd argue for replacing 'int' here with the rendered value, but I think OverflowError is the right type. Mentioning "C ssize_t" is the problem.

As for the list constructor, I'd be okay with chaining a MemoryError here, provided the OverflowError sticks around. But in this context a MemoryError is "recoverable" while an OverflowError very likely indicates a programming error, so we shouldn't hide it from the user.
History
Date User Action Args
2022-04-11 14:59:13adminsetgithub: 80733
2019-05-09 05:59:19serhiy.storchakasetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2019-05-09 04:53:31anthonypjshawsetassignee: anthonypjshaw

nosy: + anthonypjshaw
2019-04-12 16:27:15steve.dowersetnosy: + steve.dower
messages: + msg340080
2019-04-08 07:25:41serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg339592
2019-04-08 04:39:47anthonypjshawsetkeywords: + patch
stage: patch review
pull_requests: + pull_request12646
2019-04-08 04:38:16anthony shawcreate