Skip to content

Fix ZMQ write JSON serialization for NumPy types, add regression test#256

Merged
pradeeban merged 1 commit into
ControlCore-Project:devfrom
Titas-Ghosh:fix-zmq-numpy-write-serialization
Feb 11, 2026
Merged

Fix ZMQ write JSON serialization for NumPy types, add regression test#256
pradeeban merged 1 commit into
ControlCore-Project:devfrom
Titas-Ghosh:fix-zmq-numpy-write-serialization

Conversation

@Titas-Ghosh
Copy link
Copy Markdown
Contributor

Why this change?

ZMQ write() sends raw NumPy values to send_json_with_retry(), which is not JSON-serializable. This causes ZMQ messages to fail silently when payloads include NumPy scalars or arrays. File-based write() already normalizes NumPy types, so behavior is inconsistent.


What was the problem?

  • ZMQ path in concore.write() sends raw val
  • JSON serialization fails for numpy.float64, numpy.int64, arrays, dicts with NumPy values
  • Mixed or ZMQ-only studies break at runtime without a clear error

How I solved it

  1. Normalized ZMQ payloads using convert_numpy_to_python() before send_json_with_retry()
  2. Added a regression test that mocks a ZMQ port and asserts plain Python types are sent
  3. Fixed a mkconcore.py regression from latest dev where absolute output paths were rejected by safe_name (needed for CLI test to pass)

Changes / Implementation

  • concore.py
    • Convert NumPy values before ZMQ JSON send
  • tests/test_concore.py
    • New TestWriteZMQ to verify conversion and JSON-safe payload
  • mkconcore.py
    • Allow full paths for output directory in safe_name while still blocking shell metacharacters

How to Test Locally

python -m pytest -v tests/test_concore.py
python -m pytest -v


PR Checklist

  • Added tests for the new behavior
  • Ran full test suite locally
  • No public API changes
  • Fix is backwards-compatible

@pradeeban pradeeban merged commit 924b539 into ControlCore-Project:dev Feb 11, 2026
6 checks passed
@Titas-Ghosh Titas-Ghosh deleted the fix-zmq-numpy-write-serialization branch February 18, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants