BUG: apply the NEP 50 scalar rules in concatenate and choose, and make casting="no" match "equiv" by ngoldbaum · Pull Request #32497 · numpy/numpy

ngoldbaum · 2026-09-03T21:46:30Z

PR summary

This makes np.concatenate(axis=None) and np.choose cast the temporary array made from a Python scalar instead of converting the scalar with the resolved dtype.

Before:

>>> np.concatenate((np.ones(2, "int8"), 300), axis=None)
array([ 1,  1, 44], dtype=int8)
>>> np.choose([0], (-1, np.array([1], dtype=np.uint8)))
array([255], dtype=uint8)

After:

>>> np.concatenate((np.ones(2, "int8"), 300), axis=None)
OverflowError: Python integer 300 out of bounds for int8
>>> np.choose([0], (-1, np.array([1], dtype=np.uint8)))
OverflowError: Python integer -1 out of bounds for uint8

This also makes casting='no' an error in spots where casting='equiv' is an error. Closes #32491, see discussion there for details.

While we're at it, also adds reference docs for the NumPy scalar casting rules.

I searched for open source codebases that this change would break and couldn't find any, but I could have easily missed things. Maybe the new errors should be deprecations instead?

The first commit duplicates #32496 because the tests in this PR crash without that fix. I'm making that PR separately from this one to backport the crash fix.

AI Disclosure

I iterated with an AI model on the code changes and new tests.

ngoldbaum · 2026-09-03T21:48:55Z

Ping @mhvk since you pointed out the inconsistency this fixes in NumPy's python scalar casting rules in code review for #32356.

ngoldbaum · 2026-09-03T21:49:28Z

-    for casting in ["no", "equiv", "safe", "same_kind", "unsafe"]:
-        res = np.concatenate((arr, "z"), axis=None, casting=casting)
-        assert res[1] == "z"
-


this change hasn't made it into a release yet, so there's no backward compatibility concern

ngoldbaum · 2026-09-03T21:50:10Z

-        [(1000, np.array([1], dtype=np.uint8)),
-         (-1, np.array([1], dtype=np.uint8)),
+        [(100, np.array([1], dtype=np.uint8)),
+         (-1, np.array([1], dtype=np.int8)),


This changed because 1000 now triggers an overflow error. Should it merely be a deprecation instead?

mhvk

Looks good! And especially good to update the docs -- so that is also where my attention went: a few in-line comments to help ensure we do not forget that for float we allow loss of precision with "safe".

mhvk · 2026-09-03T23:45:42Z

+treats a Python ``int``, ``float`` or ``complex`` by its kind, since the
+scalar has no precision of its own. Converting a Python ``int`` to any NumPy
+integer dtype or a Python ``float`` to any NumPy floating point dtype counts
+as "safe", even though the value may not fit. A value that does not fit


"may not fit or loose precision" (for float).

mhvk · 2026-09-03T23:47:01Z

+scalar has no precision of its own. Converting a Python ``int`` to any NumPy
+integer dtype or a Python ``float`` to any NumPy floating point dtype counts
+as "safe", even though the value may not fit. A value that does not fit
+raises ``OverflowError`` on conversion, as shown above. Lowering the kind,


"While lowering a python float to a lower-precision type is considered safe,, lowering it to an integer dtype requires casting="unsafe". ..."

mhvk · 2026-09-03T23:48:11Z

+as "safe", even though the value may not fit. A value that does not fit
+raises ``OverflowError`` on conversion, as shown above. Lowering the kind,
+such as a Python ``float`` into an integer dtype, requires
+``casting="unsafe"``. `numpy.choose` applies the same rules with "safe"


Why is numpy.choose mentioned here (could perhaps be at the end?)

Thanks. I just dropped it because this section isn't really about that and it's an obscure detail.

ngoldbaum · 2026-09-09T22:09:14Z

@seberg any chance I can get your opinion here? This is docs and two fixes for things that were missed in the NEP 50 transition.

ngoldbaum · 2026-09-15T16:52:22Z

I went ahead and pushed another commit that removes use of asanyarray in the np.append implementation, which nicely makes it end up as a one-line alias for np.concatenate!

…ed dtype

seberg

Thanks this looks nice. The question is if we dare do the concatenate change as it could well break someone.

A deprecation is always nice I guess, but overall, I think I can live with it (until someone sounds the warning bells that they are hitting it of course!).
Really, I have never seen anyone rely on concatenate with axis=None and even if someone does it, the plausible issue is niche around:

concatenate((-1, arr, -1), dtype="uint8")

or so. Plausible, but strange.

But maybe I am getting a bit too relaxed about this these days. I just really think this'll break at most a number of people countable on one hand (of course if one of those is a large library).

seberg · 2026-09-15T17:00:30Z

+scalars with the result dtype, as ufuncs and `numpy.copyto` do. A Python
+integer that does not fit the result dtype raises ``OverflowError`` instead of
+wrapping. For example, ``np.concatenate((np.ones(2, "int8"), 300), axis=None)``
+previously returned ``44`` for the new element.


Yeah, I guess this change would be the one to worry about. But concatenating numbers with axis=None seems so awkward that...
(I have seen concatenate(([1], array)) more, although this actually probably is what those would want to use :))

I just did another AI-assisted scan and couldn't find cases where the new concatenate behavior would cause breakage in popular downstream libraries.

However the append change does lead to breakage (numba encodes our current behavior, for one), so I'm going to go ahead and back that out from this PR and then do a second PR that introduces a deprecation warning cycle.

seberg · 2026-09-15T17:02:46Z

+    ...
+  TypeError: Cannot cast scalar from dtype('float64') to dtype('int8') according to the rule 'same_kind'
+
+Under ``casting="equiv"`` and ``casting="no"`` a Python scalar must convert


I might be tempted to not worry about this, but I guess for completeness sake it makes sense.
(I really don't see why a user would ever want to do this, except maybe being surprised by it actually being so strict.)

Which is why I think it's safe to change. I'm only changing this because Marten noticed it looking at another PR and because I like fixing consistency holes like this.

seberg · 2026-09-15T17:07:22Z

-        if arr.ndim != 1:
-            arr = arr.ravel()
-        values = ravel(values)
-        axis = arr.ndim - 1


seberg · 2026-09-15T17:16:41Z

(Just saw the release note trimming, looks good too, and thanks, I like them short!)

mhvk

Super, all looks good!

ngoldbaum added 00 - Bug 04 - Documentation component: numpy.ufunc labels Sep 3, 2026

ngoldbaum commented Sep 3, 2026

View reviewed changes

mhvk reviewed Sep 3, 2026

View reviewed changes

ngoldbaum force-pushed the pyscalar-concat-choose branch from 285de86 to c677093 Compare September 4, 2026 17:59

ngoldbaum added the component: numpy._core label Sep 4, 2026

ngoldbaum mentioned this pull request Sep 5, 2026

BUG: apply NEP 50 scalar rules in concatenate and choose, and make casting="no" match "equiv" #32510

Closed

ngoldbaum added this to the 2.6.0 Release milestone Sep 8, 2026

ngoldbaum added the triage review Issue/PR to be discussed at the next triage meeting label Sep 15, 2026

ngoldbaum force-pushed the pyscalar-concat-choose branch from b2f2a63 to 59accf2 Compare September 15, 2026 16:49

ngoldbaum added 6 commits September 15, 2026 11:09

BUG: convert Python scalars in concatenate and choose with the resolv…

b8e7510

…ed dtype

BUG: reject casting='no' for a Python scalar where 'equiv' raises

ebb7a4c

DOC: apply marten's review suggestions

712c9b7

MAINT: fix rebase error

732f33c

DOC: add release notes

5ccfdaa

ENH: make append use NEP-50 promotion as well

0449880

seberg approved these changes Sep 15, 2026

View reviewed changes

DOC: trim release note fragments

2a2aaa7

ngoldbaum force-pushed the pyscalar-concat-choose branch from 59accf2 to 2a2aaa7 Compare September 15, 2026 17:14

ngoldbaum changed the title ~~BUG: apply the NEP 50 scalar rules in concatenate and choose, and make casting="no" match "equiv"~~ BUG: apply the NEP 50 scalar rules in append, concatenate, and choose, and make casting="no" match "equiv" Sep 15, 2026

ngoldbaum changed the title ~~BUG: apply the NEP 50 scalar rules in append, concatenate, and choose, and make casting="no" match "equiv"~~ BUG: apply the NEP 50 scalar rules in concatenate and choose, and make casting="no" match "equiv" Sep 15, 2026

MAINT: back out changes to append

833aea4

mhvk approved these changes Sep 15, 2026

View reviewed changes

ngoldbaum merged commit b245332 into numpy:main Sep 15, 2026
91 checks passed

ngoldbaum mentioned this pull request Sep 21, 2026

NEP: update NEP 58 text to reflect current discussion/prototype #32745

Merged

Uh oh!

Conversation

ngoldbaum commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR summary

AI Disclosure

Uh oh!

ngoldbaum commented Sep 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngoldbaum commented Sep 15, 2026

Uh oh!

seberg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seberg commented Sep 15, 2026

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ngoldbaum commented Sep 3, 2026 •

edited

Loading

ngoldbaum Sep 3, 2026 •

edited

Loading

ngoldbaum commented Sep 9, 2026 •

edited

Loading

ngoldbaum Sep 15, 2026 •

edited

Loading