Skip to content

Fix unit extraction in structured_to_unstructured for StructuredQuantity#19106

Open
ReemHamraz wants to merge 3 commits into
astropy:mainfrom
ReemHamraz:fix-structured-unit-extraction
Open

Fix unit extraction in structured_to_unstructured for StructuredQuantity#19106
ReemHamraz wants to merge 3 commits into
astropy:mainfrom
ReemHamraz:fix-structured-unit-extraction

Conversation

@ReemHamraz
Copy link
Copy Markdown
Contributor

Fix unit extraction in structured_to_unstructured for StructuredQuantity


Description

This PR addresses a bug where numpy.lib.recfunctions.structured_to_unstructured failed when applied to a StructuredQuantity, even if all fields within the quantity possessed compatible units.

The Problem

The function was failing during the unit extraction phase. Because a StructuredQuantity has a StructuredUnit instead of a standard Unit object, the existing helper logic was attempting to access attributes or iterators that did not exist for that object type, leading to either an AttributeError or a StopIteration.

The Fix

I modified the unit extraction logic in astropy/units/quantity_helper/function_helpers.py to safely handle StructuredUnit objects:

  • The logic now uses a try...except block to attempt standard unit extraction first.
  • If standard extraction fails (as it does with structured units), it falls back to extracting the first unit found within the StructuredUnit values.
  • This ensures the "target unit" for the resulting unstructured array is correctly identified, allowing the conversion to proceed if the fields are compatible.

###Testing
I also added a formal regression test to verify the fix and prevent future regressions. This test specifically targets the scenario where a StructuredQuantity with multiple identical units is converted to an unstructured Quantity.

Regression Test Details
File: astropy/units/tests/test_structured.py

Test Class: TestStructuredQuantityFunctions

New Method: test_structured_to_unstructured_success

The test performs the following checks:

  1. Type Check: Ensures the result of rfn.structured_to_unstructured is a standard astropy.units.Quantity.

  2. Unit Consistency: Verifies that the resulting quantity inherits the correct base unit (e.g., u.m).

  3. Data Integrity: Validates that the reshaped values match the original data.


@github-actions github-actions Bot added the units label Dec 29, 2025
@github-actions
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

# For the other tests of ``structured_to_unstructured``, see
# ``test_quantity_non_ufuncs.TestRecFunctions.test_unstructured_to_structured``

def test_structured_to_unstructured_success(self):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a crucial property of a test that accompanies a change is that, in addition to passing with the test, it should also fail without the change. This way we know that the test properly captures the change in behavior that we intend it to check. This test doesn't fail on main, so, maybe it's still valuable in itself (I haven't attempted to evaluate this yet), but it is demonstrably insufficient to check your change.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes more sense in that other module

ReemHamraz added a commit to ReemHamraz/astropy that referenced this pull request Jan 9, 2026
@neutrinoceros neutrinoceros added Bug backport-v7.2.x on-merge: backport to v7.2.x labels Mar 13, 2026
@neutrinoceros neutrinoceros added this to the v7.2.1 milestone Mar 13, 2026
@neutrinoceros
Copy link
Copy Markdown
Contributor

thanks @ReemHamraz ! your test is now failing as expected on main, so it qualifies as a proper regression test !
Now we need to make sure that all CI passes, including linting (pre-commit.ci) and docs (readthedocs). Can you rebase your branch on top of main and squash your commits to a single one ?

@neutrinoceros
Copy link
Copy Markdown
Contributor

good news: docs failures are unrelated, and seen across all PRs

ReemHamraz added a commit to ReemHamraz/astropy that referenced this pull request Mar 13, 2026
@ReemHamraz ReemHamraz force-pushed the fix-structured-unit-extraction branch from 674767c to 39aa191 Compare March 13, 2026 17:54
@ReemHamraz
Copy link
Copy Markdown
Contributor Author

pre-commit.ci autofix

@pllim
Copy link
Copy Markdown
Member

pllim commented Mar 13, 2026

Errr I don't see any code changes, just a change log entry.

@neutrinoceros
Copy link
Copy Markdown
Contributor

Huho, this is probably the result of a failed rebase+squash. Luckily I remember having checked-out this branch locally last time I reviewed it so I should be able to re-play the intended changes authored by @ReemHamraz

@neutrinoceros neutrinoceros marked this pull request as draft March 18, 2026 13:55
@neutrinoceros neutrinoceros force-pushed the fix-structured-unit-extraction branch from 7f053e3 to 0606db8 Compare March 18, 2026 13:56
Comment thread astropy/units/quantity_helper/function_helpers.py Outdated
@neutrinoceros neutrinoceros marked this pull request as ready for review March 18, 2026 14:09
with pytest.raises(ValueError, match="The length of the last dimension"):
rfn.unstructured_to_structured(self.q_pv, dtype=self.q_pv.dtype)

# For the other tests of ``structured_to_unstructured``, see
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just seeing this comment now... should your test be moved there ? (I'll try to answer my own question right now)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-v7.2.x on-merge: backport to v7.2.x Bug units

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants