Repository navigation
Honor allow_extra_data=False for unknown-size frames in the C backend - #376
Open
SakayanagiYuna wants to merge 2 commits into
Open
SakayanagiYuna wants to merge 2 commits into
SakayanagiYuna wants to merge 2 commits into
Conversation
When a frame omits its content size and max_output_size exceeds the decoded size, the output-buffer resize branch prevents the subsequent else-if trailing-data check from running. Trailing input is then accepted even with allow_extra_data=False. Run the trailing-data check independently after successful decompression and any output-buffer resize. Add regression coverage for unknown-size empty and nonempty frames, exact and spare output capacity, trailing junk and concatenated frames, and decoder reuse after rejection. Verify that the default behavior and explicit allow_extra_data=True continue to accept trailing input. Related to indygreg#181.
Check the error return from PyObject_IsTrue before testing for trailing input. Clear the output object and use the existing cleanup path so the original exception propagates instead of producing SystemError. Add regression coverage for __bool__ and __len__ failures with known and unknown content sizes, exact and spare output capacity, and input with and without trailing data. Verify that the original exception object is preserved and that the decompressor remains reusable after the error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The C backend ignores
allow_extra_data=Falsewhen a frame omits itscontent size and
max_output_sizeexceeds the decoded size.For example:
This returns
b"foobar"instead of raisingZstdErrorfor the fourunused bytes. The output-buffer resize branch prevents the subsequent
else iftrailing-data check from running.This change makes that check independent of buffer resizing. The default
behavior and explicit
allow_extra_data=Trueremain unchanged.Errors raised while evaluating the truth value of
allow_extra_dataarealso propagated correctly, preserving the original exception instead of
returning a result with an exception set and producing
SystemError.Regression coverage includes unknown-size empty and nonempty frames,
exact and spare output capacity, trailing junk and a second frame, and
decoder reuse after rejection. Additional cases cover exceptions from
__bool__and__len__, with and without trailing data, and verify thatthe original exception object is preserved and the decoder remains usable.
Validation on Windows with CPython 3.12.14:
all pass with this change.
SystemErrorbefore the error-handling fix and pass afterward.
skips.
Related to #181.