Repository navigation
fix(msgpack): raise DecodeError for unhashable map keys - #1209
Merged
Merged
Conversation
Translate TypeError from PyDict_SetItem in mpack_decode_dict into DecodeError so map/ext keys no longer escape the documented error contract. Fixes msgspec#1208
vyrnsynx
marked this pull request as ready for review
September 26, 2026 01:12
sobolevn
approved these changes
Sep 29, 2026
sobolevn
added this pull request to the merge queue
Sep 29, 2026
Closed
pull Bot
pushed a commit
to Mu-L/msgspec
that referenced
this pull request
Sep 29, 2026
Entries for msgspec#1194, msgspec#1196, msgspec#1197, msgspec#1199 and msgspec#1209 are missing from the changelog of the 0.22.0 release. Of the changes merged after msgspec#1211 moved the unreleased entries into the 0.22.0 section, only msgspec#1207 added its own entry. This adds the missing entries next to the related entries in that section, and marks msgspec#1196 as a breaking change: objects that expose `__dataclass_fields__` only through instance attribute access are no longer encoded as dataclasses. Co-authored-by: Tseluiko Aleksandr <[email protected]>
simpleqt
added a commit
to simpleqt/msgspec
that referenced
this pull request
Nov 8, 2026
`msgspec.json.decode`, `msgspec.msgpack.decode` and `msgspec.convert` raise a
bare `TypeError` when a value can't be added to a `set`/`frozenset`:
>>> msgspec.json.decode(b'[{"a": 1}]', type=set)
TypeError: unhashable type: 'dict'
`DecodeError` is documented as "An error occurred while decoding an object",
so `except msgspec.DecodeError` doesn't catch this. The same bug was fixed for
msgpack map keys in msgspec#1209; the set item paths still leak it.
`mpack_decode_set`, `json_decode_set` and `convert_seq_to_set` now translate
the `TypeError` raised by `PySet_Add` into a `ValidationError` (a subclass of
`DecodeError`) that points at the offending item:
>>> msgspec.json.decode(b'[{"a": 1}]', type=set)
msgspec.ValidationError: Expected a hashable value, got `dict` - at `$[0]`
`convert_dict_to_dict` leaks it the same way when a key converts to an
unhashable value, so the `PyDict_SetItem` call there gets the same treatment:
>>> msgspec.convert({(1, 2): 0}, dict[list[int], int])
msgspec.ValidationError: Expected a hashable value, got `list` - at `key` in `$`
Note the msgpack decoder decodes arrays as tuples when a hashable value is
required, so arrays still decode into sets there; only maps, `Ext` values and
unhashable `dec_hook` results error out.
Tests: all 19 new cases raise a bare `TypeError` without the fix.
- `tests/unit/test_common.py`: json and msgpack, a `set`/`frozenset` item and a
nested `$.values[0]` path.
- `tests/unit/test_json.py`: `[[1, 2]]` decodes as a list, which can't be added
to a `set`, `frozenset` or `set[list[int]]`.
- `tests/unit/test_msgpack.py`: a map and an `Ext` value as set items.
- `tests/unit/test_convert.py`: `convert` with a dict/list item, a struct field
path, and an unhashable dict key (4 cases, 3 of them parametrized over
`dict`/`dict` subclass/duck mapping).
`pytest tests/unit` (Python 3.11, Windows): 6549 passed, 243 skipped, 2 errors.
Both errors are `test_cpylint.py` reading `_core.c` with the local cp936
default encoding, and they fail identically on `main`.
`ruff check`, `ruff format --diff` and `codespell` pass.
Signed-off-by: simpleqt <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
msgspec.msgpack.decodecurrently raises a bareTypeErrorwhen a MessagePack map orextis used as a map key. That escapes the documentedDecodeErrorcontract, soexcept msgspec.DecodeErrordoes not catch it.Fix
In
mpack_decode_dict, translateTypeErrorfromPyDict_SetItemintoDecodeError. Other errors (e.g. OOM) are left unchanged.mpack_decode_frozendictalready goes through this path.Tests
Adds coverage for map/ext keys (
818001,878048,8bd4ec5c03) and keeps the hashable array-key control (819001→{(): 1}).Fixes #1208