Skip to content

fix(msgpack): raise DecodeError for unhashable map keys - #1209

Merged
sobolevn merged 3 commits into
msgspec:mainfrom
vyrnsynx:fix-msgpack-unhashable-map-key
Sep 29, 2026
Merged

sobolevn merged 3 commits into
msgspec:mainfrom
vyrnsynx:fix-msgpack-unhashable-map-key

Conversation

@vyrnsynx

Copy link
Copy Markdown
Contributor

Problem

msgspec.msgpack.decode currently raises a bare TypeError when a MessagePack map or ext is used as a map key. That escapes the documented DecodeError contract, so except msgspec.DecodeError does not catch it.

Fix

In mpack_decode_dict, translate TypeError from PyDict_SetItem into DecodeError. Other errors (e.g. OOM) are left unchanged. mpack_decode_frozendict already goes through this path.

Tests

Adds coverage for map/ext keys (818001, 878048, 8bd4ec5c03) and keeps the hashable array-key control (819001 → {(): 1}).

Fixes #1208

Translate TypeError from PyDict_SetItem in mpack_decode_dict into
DecodeError so map/ext keys no longer escape the documented error
contract.

Fixes msgspec#1208
@vyrnsynx
vyrnsynx marked this pull request as ready for review September 26, 2026 01:12

@sobolevn sobolevn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@sobolevn
sobolevn added this pull request to the merge queue Sep 29, 2026
@sobolevn sobolevn mentioned this pull request Sep 29, 2026
Merged via the queue into msgspec:main with commit 55f5cdd Sep 29, 2026
24 checks passed
pull Bot pushed a commit to Mu-L/msgspec that referenced this pull request Sep 29, 2026
Entries for msgspec#1194, msgspec#1196, msgspec#1197, msgspec#1199 and msgspec#1209 are missing from the
changelog of the 0.22.0 release. Of the changes merged after msgspec#1211
moved the unreleased entries into the 0.22.0 section, only msgspec#1207 added
its own entry. This adds the missing entries next to the related
entries in that section, and marks msgspec#1196 as a breaking change: objects
that expose `__dataclass_fields__` only through instance attribute
access are no longer encoded as dataclasses.

Co-authored-by: Tseluiko Aleksandr <[email protected]>
simpleqt added a commit to simpleqt/msgspec that referenced this pull request Nov 8, 2026
`msgspec.json.decode`, `msgspec.msgpack.decode` and `msgspec.convert` raise a
bare `TypeError` when a value can't be added to a `set`/`frozenset`:

    >>> msgspec.json.decode(b'[{"a": 1}]', type=set)
    TypeError: unhashable type: 'dict'

`DecodeError` is documented as "An error occurred while decoding an object",
so `except msgspec.DecodeError` doesn't catch this. The same bug was fixed for
msgpack map keys in msgspec#1209; the set item paths still leak it.

`mpack_decode_set`, `json_decode_set` and `convert_seq_to_set` now translate
the `TypeError` raised by `PySet_Add` into a `ValidationError` (a subclass of
`DecodeError`) that points at the offending item:

    >>> msgspec.json.decode(b'[{"a": 1}]', type=set)
    msgspec.ValidationError: Expected a hashable value, got `dict` - at `$[0]`

`convert_dict_to_dict` leaks it the same way when a key converts to an
unhashable value, so the `PyDict_SetItem` call there gets the same treatment:

    >>> msgspec.convert({(1, 2): 0}, dict[list[int], int])
    msgspec.ValidationError: Expected a hashable value, got `list` - at `key` in `$`

Note the msgpack decoder decodes arrays as tuples when a hashable value is
required, so arrays still decode into sets there; only maps, `Ext` values and
unhashable `dec_hook` results error out.

Tests: all 19 new cases raise a bare `TypeError` without the fix.

- `tests/unit/test_common.py`: json and msgpack, a `set`/`frozenset` item and a
  nested `$.values[0]` path.
- `tests/unit/test_json.py`: `[[1, 2]]` decodes as a list, which can't be added
  to a `set`, `frozenset` or `set[list[int]]`.
- `tests/unit/test_msgpack.py`: a map and an `Ext` value as set items.
- `tests/unit/test_convert.py`: `convert` with a dict/list item, a struct field
  path, and an unhashable dict key (4 cases, 3 of them parametrized over
  `dict`/`dict` subclass/duck mapping).

`pytest tests/unit` (Python 3.11, Windows): 6549 passed, 243 skipped, 2 errors.
Both errors are `test_cpylint.py` reading `_core.c` with the local cp936
default encoding, and they fail identically on `main`.
`ruff check`, `ruff format --diff` and `codespell` pass.

Signed-off-by: simpleqt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

msgpack.decode leaks TypeError: unhashable type: 'dict' instead of DecodeError when a map or ext is used as a map key

3 participants