Skip to content

ENH: add np.positive ufunc - #8967

Merged
eric-wieser merged 2 commits into
numpy:masterfrom
shoyer:positive
May 1, 2017
Merged

eric-wieser merged 2 commits into
numpy:masterfrom
shoyer:positive

Conversation

@shoyer

@shoyer shoyer commented Apr 21, 2017

Copy link
Copy Markdown
Member

xref #8932

The strategy here was to search for every use of "negative" in NumPy's C code, so I quite possibly missed something. But it builds, and seems to work.

Still needs full test coverage. The existing test coverage for ufuncs like absolute and negative is rather sparse, but that's no excuse for new code.

Questions to resolve:

  • Is there a better way to do this, perhaps still making use of PyArray_Copy?
  • Do I need to write an simd loop for floats? This looked a little tricky, so I didn't bother. But it does mean that positive is slightly slower than negative for floats (I measured 7% for a large array), which is a little strange.
  • Do we need to deprecate the existing behavior for the unary op instead of suddenly raising for operations like +np.array('foo')? How do we even detect these cases? I guess we can check for dtypes that are no longer valid (e.g., strings), but there's no way to know if unary + is a valid on an dtype=object array without trying it on each of the scalars.
  • Should + on a boolean array be valid? I guess the answer is no, given that -x raises for boolean arrays and based on the principle that +x should only be valid when -x is. This strengths the case for a deprecation cycle for the behavior of ndarray.__pos__, given that we went through such a deprecation cycle for negative on booleans (DEP: Handle expired deprecations. #8297).

@shoyer

shoyer commented Apr 21, 2017

Copy link
Copy Markdown
Member Author

For now, it might be better to skip trying to fix up ndarray.__pos__, and only add the new ufunc for use with NDArrayOperatorsMixin.

Comment thread numpy/core/src/umath/loops.c.src Outdated

@eric-wieser eric-wieser Apr 21, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do *out = +in here (and below), for clarity?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then you haven't pushed!

Comment thread numpy/core/src/umath/loops.c.src Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize that you're just copying negative here - but what's the heuristic for when to use UNARY_LOOP_FAST and when to just use UNARY_LOOP? @juliantaylor, I suspect you can shed some light here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you use UNARY_LOOP_FAST in cases where compiler vectorization is profitable.
It involves several copies of the loop so it is just bloat in cases where no vectorization can be done like in the negative half case.
You could use a fast loop for positive here (though it is hardly worth the extra generated code)
In general you could use a memcpy special case instead of the fast loop here too, like the logical_not in gh-8924. Some compilers might not be able to do that for you.

Comment thread numpy/core/src/multiarray/number.c Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function can be removed now

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With some regret, I decided to undo this change, because of the potential for a backwards compatibility break (we're pretty paranoid about those).

@shoyer shoyer changed the title ENH: add np.positive ufunc and use it for ndarray.__pos__ ENH: add np.positive ufunc Apr 22, 2017
@eric-wieser eric-wieser added the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Apr 22, 2017
@shoyer

shoyer commented Apr 26, 2017

Copy link
Copy Markdown
Member Author

Any further concerns here?

Note that I rolled back the changes to ndarray.__pos__

@eric-wieser

Copy link
Copy Markdown
Member

Ah, but needs a release note

@eric-wieser

eric-wieser commented Apr 26, 2017

Copy link
Copy Markdown
Member

Proposal - can all the positive loops be combined into the same template as the negative loops? So add a repeat with @op = +, -@, @func = positive, negative@. Just in the interest of keeping what is an already large file to a somewhat reasonable size.

@shoyer shoyer added this to the 1.13.0 release milestone Apr 27, 2017
@charris

charris commented Apr 30, 2017

Copy link
Copy Markdown
Member

@shoyer Needs a rebase and release note.

@homu

homu commented Apr 30, 2017

Copy link
Copy Markdown
Contributor

☔ The latest upstream changes (presumably #8885) made this pull request unmergeable. Please resolve the merge conflicts.

@shoyer

shoyer commented May 1, 2017

Copy link
Copy Markdown
Member Author

Proposal - can all the positive loops be combined into the same template as the negative loops? So add a repeat with @op = +, -@, @func = positive, negative@. Just in the interest of keeping what is an already large file to a somewhat reasonable size.

This is a good idea, but it turned out to be easier said that done:

  • For integers, negative uses a non-trivial SIMD trick
  • For timedelta, negative needs to check for NaT
  • For floats, negative also uses SIMD
  • For half, negative also uses something clever

So I just stuck with separate definitions.

Needs a rebase and release note

Done and done.

@shoyer shoyer removed the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label May 1, 2017
@eric-wieser
eric-wieser merged commit d46df62 into numpy:master May 1, 2017
@eric-wieser

Copy link
Copy Markdown
Member

Thanks @shoyer!

@mhvk

mhvk commented May 8, 2017

Copy link
Copy Markdown
Contributor

I'm puzzled, this one function does not seem to work well with __array_ufunc__:

class A(np.ndarray):
    def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
        return "A"

a = np.array(1.).view(A)
-a
# 'A'
+a
# 1.0

I think it is due to positive still being defined as _array_copy_nice in number.c, but simply changing that and adding to GET and SET seemed insufficient:

numpy/core/src/multiarray/number.c:573:50: error: ‘NumericOps {aka struct <anonymous>}’ has no member named ‘positive’; did you mean ‘negative’?
     return PyArray_GenericUnaryFunction(m1, n_ops.positive);

(will try to investigate later, but have to run now; if someone else understands what is happening, do go ahead and fix...)

@eric-wieser

Copy link
Copy Markdown
Member

@shoyer deliberately did not connect up np.positive to __pos__ yet, so I think this is expected behaviour?

@mhvk

mhvk commented May 8, 2017

Copy link
Copy Markdown
Contributor

OK, rereading the comments I see that is the case. I guess I'll have to keep Quantity.__pos__ for another release then...

@shoyer
shoyer deleted the positive branch May 8, 2017 19:51
@shoyer

shoyer commented May 8, 2017

Copy link
Copy Markdown
Member Author

@mhvk if you think it would be OK from a compatibility perspective, we could switch ndarray.__pos__. But again, I am concerned that this might cause a minor compatibility break.

@mhvk

mhvk commented May 8, 2017

Copy link
Copy Markdown
Contributor

Probably being careful is good, but should we at least add the deprecation warning? Also, maybe just pass on to np.positive for classes that define __array_ufunc__? (Since those obviously know what is happening.)

@shoyer

shoyer commented May 8, 2017

Copy link
Copy Markdown
Member Author

@mhvk Those both sound like fine ideas to me.

@mhvk

mhvk commented May 8, 2017

Copy link
Copy Markdown
Contributor

@shoyer - would you be able to do it? Or have an idea what I should do beyond what I tried (#8967 (comment))

@shoyer

shoyer commented May 9, 2017

Copy link
Copy Markdown
Member Author

@mhvk I'm not sure I'll have time to get to this before the branch for 1.13. Probably, the desired behavior is something like:

def __pos__(self):
    try:
        return np.positive(self)
    except TypeError:
        issue deprecation warning
        return self.copy()

(This may be a little tricky, though, because it will need to be done in C.)

For your ndarray subclass, you could overwrite __pos__ to call np.positive, though would probably need a fallback option for older versions of numpy where np.positive is not available.

@mhvk

mhvk commented May 9, 2017

Copy link
Copy Markdown
Contributor

The override is indeed what I do for Quantity; I'm also not sure I'll have time for another crack at this.

@shoyer

shoyer commented May 9, 2017

Copy link
Copy Markdown
Member Author

I created #9081 so we don't forget about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants