Skip to content

Remove use of numpy structured arrays in table operations - #3179

Merged
taldcroft merged 8 commits into
astropy:masterfrom
taldcroft:table-ops
Dec 9, 2014
Merged

taldcroft merged 8 commits into
astropy:masterfrom
taldcroft:table-ops

Conversation

@taldcroft

Copy link
Copy Markdown
Member

PR #2790 did not address the high-level table operations hstack, vstack, and join. These are implemented in two-tiered approach: the low-level tier in np_utils.py used pure numpy structured arrays to do the heavy lifting while the high-level operations.py provided the Table interface. The problem after #2790 is that these operations required creating temporary numpy structured arrays to represent the tables.

This PR updates the low-level code to do the operations directly with Table objects and largely eliminates use of numpy structured arrays.

In the current version of the PR, the module np_utils.py is basically orphaned, at least from the Table perspective. The functions there were nominally public so we need to decide whether to deprecate them or just leave as a useful utility.

@taldcroft taldcroft added table Affects-dev PRs and issues that do not impact an existing Astropy release labels Dec 5, 2014
@taldcroft taldcroft self-assigned this Dec 5, 2014
@taldcroft taldcroft added this to the v1.0.0 milestone Dec 5, 2014
@taldcroft

Copy link
Copy Markdown
Member Author

By moving the low-level code into operations.py, all visibility in to the code changes is lost in the final diff with master. However, the changes can be seen in the individual commits.

@astrofrog

Copy link
Copy Markdown
Member

@taldcroft - very nice! For the np_utils.py, I personally doubt anyone was using these, but I may be wrong. Maybe you could ping the list about it? If no one objects, you can add deprecation warnings?

Otherwise, 👍 from me!

@taldcroft

Copy link
Copy Markdown
Member Author

In anticipation of no objections I've put in the deprecation warnings.

Comment thread astropy/table/np_utils.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should be the alternative to using these? Is it worth mentioning one?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if the alternative is just "don't use these because they were only meant to support functionality that no longer exists" :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By design there is no real astropy dependency in np_utils.py, so it can be ripped out quite easily if needed. So two options are:

  • Copy np_utils.py (and _np_utils.pyx) to your project and use it.
  • Ask the astropy devs about making this into a standalone package.

I don't know exactly how to say this in a deprecation message.

@taldcroft

Copy link
Copy Markdown
Member Author

@embray - something like this good enough?

message = ('The %(func)s %(obj_type)s is deprecated and may '
                    'be removed in a future version. '
                    'Contact the Astropy developers if you need '
                    'continued support for this function.')

@embray

embray commented Dec 9, 2014

Copy link
Copy Markdown
Member

Works for me--I absolutely agree it seems unlikely to ever be an issue though.

taldcroft added a commit that referenced this pull request Dec 9, 2014
Remove use of numpy structured arrays in table operations
@taldcroft
taldcroft merged commit da2bf58 into astropy:master Dec 9, 2014
@taldcroft
taldcroft deleted the table-ops branch December 9, 2014 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Affects-dev PRs and issues that do not impact an existing Astropy release table

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants