Repository navigation
Remove bytearray to bytes copies in stdlib using take_bytes #141968
Description
Feature or enhancement
Proposal:
This is a meta issue for this one pattern which is fairly small and well contained (ba = bytearray(), later does bytes(ba)). More complex refactors (ex. binacii.b2a_base64 usage) should get their own / separate issues.
Using .take_bytes() (gh-139871) can remove an allocation + copy in cases where a function builds up data in a temporary bytearray then at the end of the function returns bytes(ba). This can significantly speed up some code. For instance gh-141863 for asyncio.streams improved the asyncio_tcp pyperformance benchmark over 10%.
- base64
_b32encode,_b32decode - wave
_byteswap - encodings.punycode
- encodings.idna
-
urllib.parse-- Theunquote_to_bytesbytearray.extenddramatically dominates time in profile, more complex refactor needed for measurable improvement. - re._compiler charmap
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Linked PRs
- gh-141968: Use take_byes in base64 _b32encode and _b32decode #141971
- gh-141968: Use take_bytes in wave._byteswap #141973
- gh-141968: Use take_bytes in encodings.punycode #141974
- gh-141968: Use take_bytes in encodings.idna #141975
- gh-141968: Use take_bytes in re._compiler #141995
- gh-141968: Use take_bytes in zipfile._ZipDecrypter #142240
- gh-141968: Use
take_bytesto remove copy in_pyio.BytesIO.read()#149850 - gh-141968: Use
take_bytesto simplify and remove copy from pyreplBaseEventQueue#149852 - gh-141968: Use bytearray.take_bytes() in more I/O paths #156989
Activity
_b32encode,_b32decode#141969_b32encode,_b32decode#141969pythongh-141968: Use take_byes in base64 _b32encode and _b32decode
pythongh-141968: Use take_bytes in wave._byteswap
pythongh-141968: Use take_byes in encodings.punycode
pythongh-141968: Use take_bytes in encodings.idna
pythongh-141968: Use take_bytes in encodings.idna
pythongh-141968: Use take_bytes in encodings.idna
gh-141968: Use `bytearray.take_bytes` in `base64` `_b32encode` and `_…
gh-141968: use `bytearray.take_bytes` in `wave._byteswap` (#141973)
2 remaining items
gh-141968: Use take_bytes in re._compiler (#141995)
gh-141968: Use take_bytes in encodings.punycode (#141974)
pythongh-141968: Use take_byes in zipfile._ZipDecrypter
pythongh-141968: Use `bytearray.take_bytes` in `base64` `_b32encode` …
pythongh-141968: use `bytearray.take_bytes` in `wave._byteswap` (pyth…
pythongh-141968: use `bytearray.take_bytes` in `encodings.idna` (pyth…
pythongh-141968: Use take_bytes in re._compiler (python#141995)
pythongh-141968: Use take_bytes in encodings.punycode (python#141974)
gh-141968: Use `take_bytes` to remove copy in `_pyio.BytesIO.read()` (#…
gh-141968: Use `take_bytes` to simplify and remove copy from pyrepl `…
pythongh-141968: Use bytearray.take_bytes() in more I/O paths
pythongh-141968: Inline traditional ZIP key updates
pythongh-141968: Use bytearray.take_bytes() in more I/O paths
pythongh-141968: Inline traditional ZIP key updates