Skip to content

fix(output): union columns across all rows for CSV and table rendering - #29

Open
arthi-arumugam-git wants to merge 1 commit into
Allium-Science:mainfrom
arthi-arumugam-git:fix-heterogeneous-rows
Open

arthi-arumugam-git wants to merge 1 commit into
Allium-Science:mainfrom
arthi-arumugam-git:fix-heterogeneous-rows

Conversation

@arthi-arumugam-git

@arthi-arumugam-git arthi-arumugam-git commented Aug 15, 2026 •

Copy link
Copy Markdown

Fixes #19; the diagnosis is orvionx's.

Two failure modes from rows[0].keys():

  • CSV crashed: DictWriter's default extrasaction raises ValueError the moment a later row carries a field the first row does not, which is routine for responses with optional, chain-specific fields.
  • Table silently dropped those same columns, which is arguably worse: incomplete financial data that looks complete.

Both paths now use a first-seen-order union of keys across all rows; CSV renders missing fields as empty strings via restval. Column order for homogeneous responses is unchanged.

Tests in tests/test_output_columns.py: heterogeneous rows produce all columns without raising, extra-field values reach the output, missing fields render empty, and homogeneous ordering is untouched. 4 pass with the fix; the module has no _union_columns on main so they demonstrate the gap there.

Columns came from rows[0].keys() only. CSV output crashed with
ValueError when a later row carried a field the first row did not
(DictWriter's default extrasaction), and table output silently dropped
that column. Both now use a first-seen-order union of keys across all
rows; CSV renders missing fields as empty via restval.

Diagnosis is orvionx's in Allium-Science#19; fixes Allium-Science#19.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

handle heterogeneous response rows without crashing

1 participant