Skip to content

ENH: Avoid re-encapsulation of the default handler - #20345

Merged
mattip merged 2 commits into
numpy:mainfrom
eliaskoromilas:default-handler
Nov 13, 2021
Merged

mattip merged 2 commits into
numpy:mainfrom
eliaskoromilas:default-handler

Conversation

@eliaskoromilas

@eliaskoromilas eliaskoromilas commented Nov 10, 2021 •

Copy link
Copy Markdown
Contributor

The default handler is now capsuled once (singleton), which makes testing for object identity, using == in C, or is in Python sufficient.

#20193 and #20343 discussions imply that PyDataMem_Handler.name should not be used to encode information neither to suggest allocator uniqueness, so this will allow user libraries to tell if a 'default_allocator' is the 'default_allocator'.

The question is whether it would make more sense to also export that capsule in the NumPy API (e.g. like Py_None singleton).

@eliaskoromilas

Copy link
Copy Markdown
Contributor Author

@mattip @seberg What do you think?

@seberg

seberg commented Nov 10, 2021

Copy link
Copy Markdown
Member

Seems reasonable to me. Do we expect users to test identity? Because if not, we could just not make this a public symbol, but I don't really care either way.

@eliaskoromilas

eliaskoromilas commented Nov 10, 2021 •

Copy link
Copy Markdown
Contributor Author

Do we expect users to test identity?

Define users :). My points in favor of exporting it publicly are:

  • There is already a way to get (a borrowed reference to) it, but can be considered an overkill:
    PyObject *handler = PyDataMem_SetHandler(NULL);
    PyObject *default_handler = PyDataMem_SetHandler(handler);
    Py_XDECREF(handler);
    Py_XDECREF(default_handler);
  • Maybe in the future NumPy could offer more than one native allocator (e.g. pooled/unpooled versions). PyDataMem_SetHandler(NULL) will always set the default one but still all the available options should be accessible by the users. For example:
    Py_XDECREF(PyDataMem_SetHandler(NULL)); // sets the default allocator
    Py_XDECREF(PyDataMem_SetHandler(PyDataMem_ArenaHandler)); // sets the arena allocator
    Py_XDECREF(PyDataMem_SetHandler(PyDataMem_DefaultHandler)); // sets the default allocator (for API consistency)
  • It could enable a feature in the NumPy Allocator project, roughly equivalent to this:
    import numpy
    import numpy_allocator
    
    class default_allocator(metaclass=numpy_allocator.type):
        _handler_ = numpy_allocator.default_handler()
    
    with another_allocator:
        # do stuff
        with default_allocator:
            test = numpy.ndarray(())
    assert default_allocator.handles(test)

@seberg

seberg commented Nov 10, 2021

Copy link
Copy Markdown
Member

Good, any reason is enough of a reason for me in this case ;).

@seberg

seberg commented Nov 12, 2021

Copy link
Copy Markdown
Member

@mattip, if you like this, lets put it in for 1.22?

@seberg
seberg requested a review from mattip November 12, 2021 18:34
@mattip
mattip merged commit 4b19c7c into numpy:main Nov 13, 2021
@mattip

mattip commented Nov 13, 2021

Copy link
Copy Markdown
Member

LGTM, thanks. It simplifies things a bit, at the cost of another exported API value

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants