ENH: ArrayMethod sorting for built-in dtypes by MaanasArora · Pull Request #31278 · numpy/numpy

MaanasArora · 2026-04-20T00:27:21Z

Adds new-style sorting and argsorting array-methods (#29737) to built-in NumPy dtypes (bools, integers, floats, dates). This allows the new path in PyArray_Sort to be functional for these, which will enable writing descending sorts soon after this. Supersedes #31273 to be much leaner. ping @seberg @mhvk - thanks!

Still have to check the resolve descriptors setup and see if string/unicode can be added, but otherwise this is basically complete (modulo below refcount comment).

AI Disclosure

I used Claude to write the initial template *_(arg)sort_loop functions but I cleaned it to the point there is little left of it.

MaanasArora · 2026-04-20T00:27:59Z

+        return -1;
+    }
+    NPY_DT_SLOTS(dtypemeta)->sort_meth = sort_method_@name@->method;
+    Py_INCREF(sort_method_@name@->method);


For some reason, decrefing sort_method_@name@ causes a segfault, but this is leaking otherwise...

Rather cryptic, I agree. The problem is that:

if (PyType_Ready(&PyArrayMethod_Type) < 0) { return -1; } if (PyType_Ready(&PyBoundArrayMethod_Type) < 0) { return -1; }

needs to be called first. Remind me: object and void already use this due to being the default fallback path, I guess?
(But I agree that they are different enough, either way.)

No I don't think so actually! object and void have a comparison function that is set here:

(PyArray_CompareFunc*)@from@_compare,

So, they are using the old path. That said it shouldn't at all be hard to use npy_default_sort_loop to take in a new comparison function for these.

Hmm, if those use @from@_compare, can we then not do the same in #30415 (we discussed there somewhat at #30415 (review))

Right, we could add that slot, but in the end, I suppose either way we'll be using a hard-coded path for those. (We'll probably just want to change the signature of the internal sort function to re-align, but it is hard-coded either way.)
One could add a "uses compare slot" auto-fill, but I am not sure it is worth it.

Agreed, probably will hard-code those, so unclear whether filling the new slot with legacy sort is worth the tradeoff.

So for the DECREF... I guess this entire registration needed to be moved, so I added a function and called that after that block instead of within set_typeinfo, thanks!

That should work yeah. If it is nicer, you can also just move the finalization of ArrayMethod_Type and BoundArrayMethod_Type to happen first, they don't have any dependencies and can happen much earlier.

Ah okay, thanks! That does look nicer (less verbose), pushed!

seberg

Thanks, small comments, mostly how to fix the decref issue.

seberg · 2026-04-20T09:45:42Z

+    }
+
+    return method->casting;
+}


Hmmm, I guess we these them for timedelta64/datetime64 unfortunately, because it is parametric? (Just too bad, since this is all very simple in principle...)

Unfortunate indeed, yes, it's because they're parametric. Given how often we're running into this, probably good to fix the issue somehow...

seberg · 2026-04-20T09:46:36Z

+        return -1;
+    }
+    NPY_DT_SLOTS(dtypemeta)->sort_meth = sort_method_@name@->method;
+    Py_INCREF(sort_method_@name@->method);


Rather cryptic, I agree. The problem is that:

if (PyType_Ready(&PyArrayMethod_Type) < 0) { return -1; } if (PyType_Ready(&PyBoundArrayMethod_Type) < 0) { return -1; }

needs to be called first. Remind me: object and void already use this due to being the default fallback path, I guess?
(But I agree that they are different enough, either way.)

mhvk

This looks nice! Main question I have is whether we can know these routines are actually used! You probably found out implicitly from errors... Can one test using the RuntimeError? Though it seems difficult to trigger in practice. So perhaps it can wait until we support descending...

MaanasArora · 2026-04-20T22:13:23Z

Thanks @mhvk! Yes I found out via errors indeed, and confirmed with printf statements (I know...) Do you mean a runtime error in PyArray_Sort? I think that could make sense, something that checks if one of these dtypes has made it past the array method path, it should be fast so we probably don't care about penalizing other dtypes. Going to try this if it makes sense!

mhvk · 2026-04-21T05:10:47Z

+            return timsort_@suff@(data[0], dimensions[0], NULL);
+#endif
+        default:
+            PyErr_Format(PyExc_RuntimeError,


I was wondering if it was possible to hit this RuntimeError -- but ideally from python. But that does seem tricky; having checked with print statements really is good enough!

Yeah, not sure we can reach it easily. Although, once we follow up with reverse sorting, I think we'll have our proof at least?

I think we can reach it by setting descending=True right? That should hit this?

Edit: sorry, I didn't recall we don't have it in Python yet for some reason! Yeah, that could be easier to test... no kwarg seems to hit this path right now in Python.

…method types instead

seberg · 2026-04-29T06:38:23Z

@MaanasArora happy to put this in if it helps with the other PR, just let me know. I think we'll want to move much of this to C++ to avoid that c.src -> C++ template transition (which is just really verbose).
(So in the end, maybe we can just go with the other PR directly and skip this also.)

MaanasArora · 2026-04-29T10:42:23Z

@seberg Yeah I rebased the other PR and moved sort loop creation to C++, so we can just go with that one I think! I'll close this. Thanks!

ENH: ArrayMethod sorting for built-in dtypes

f4bbfc2

MaanasArora commented Apr 20, 2026

View reviewed changes

MaanasArora mentioned this pull request Apr 20, 2026

ENH: ArrayMethod sorts for builtin dtypes #31273

Closed

REF: Remove unused template variable

bec707a

seberg reviewed Apr 20, 2026

View reviewed changes

mhvk reviewed Apr 20, 2026

View reviewed changes

REF: Simplify output descriptor in sort_resolve_descriptors

64e7d0a

mhvk reviewed Apr 21, 2026

View reviewed changes

MaanasArora added 2 commits April 21, 2026 05:06

BUG: Move builtin sort registrations after array method types are ready

45b558f

REF: Remove set_sorts function back to set_typeinfo, move ready array…

66c183e

…method types instead

MaanasArora mentioned this pull request Apr 27, 2026

ENH: Descending sorts with built-in dtype implementations #31345

Merged

MaanasArora closed this Apr 29, 2026

Conversation

MaanasArora commented Apr 20, 2026

AI Disclosure

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

MaanasArora commented Apr 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaanasArora Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seberg commented Apr 29, 2026

Uh oh!

MaanasArora commented Apr 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MaanasArora Apr 21, 2026 •

edited

Loading