Need to make class destruction more robust

@leofang

Possible solutions include

using weakref.finalize() (as you also suggested)

cythonize everything, make all classes cdef'd, and switch from __del__ to __dealloc__

using a lightweight shutdown detection hack (see, e.g., Memory Leak in libllvmlite.so when using Numba numba/numba#9731 (comment) and numba/llvmlite@7347d77)

Originally posted by @leofang in #87 (comment)

See also #87 (comment). This doesn't seem trivial to me by a curtesy look at the Python atexit docs...

Moving this to beta 2 and bumping to P0

I spent a few minutes playing with weakref.finalize() and also looked around.

This seems to work great, even with self.close as the weakref.finalize() callback:

import weakref

class ShopKeeper:
    def __init__(self, serno, exception_harness):
        self.serno = serno
        self.exception_harness = exception_harness
        weakref.finalize(self, self.close)

    def close(self):
        if self.exception_harness:
            try:
                print("close", self.serno, 1 / (self.serno - 2))
            except Exception:
                print("problem", self.serno)
        else:
                print("close", self.serno, 1 / (self.serno - 2))

for exception_harness in [False, True]:
    for serno in range(5):
        ShopKeeper(serno, exception_harness)

$ python3 --version
Python 3.10.2

$ python3 ShopKeeper.py
close 4 0.5
close 3 1.0
problem 2
close 1 -1.0
close 0 -0.5
close 4 0.5
close 3 1.0
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/weakref.py", line 667, in _exitfunc
    f()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/weakref.py", line 591, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/Users/rgrossekunst/Downloads/ShopKeeper.py", line 16, in close
    print("close", self.serno, 1 / (self.serno - 2))
ZeroDivisionError: division by zero
close 1 -1.0
close 0 -0.5

Is it better than __del__?

Maybe not a lot? — See https://peps.python.org/pep-0442/ (Python 3.4 was released in 2014)

But I also found https://groups.google.com/g/dev-python/c/iFlQm0j5lpU/m/viOe4hQeCAAJ, with postings from 2019/2020.

Quoting for easy reference:

The main thing going for weakref.finalize is that it is called at the beginning of the interpreter shutdown while __del__ may be called much later in the process.

My own conclusion: Replacing __del__ with weakref.finalize() seems to be super easy** and will make the code (slightly) more robust.

** I'm counting only 5 __del__ in cuda_core

@rwgk

Great, @rwgk mind I re-assign this task to you? 😛

Done :-)

I think this is great for me to learn the cuda-python developer workflow.

xref: #1063 (comment)

Navigation Menu

Repository navigation

Need to make class destruction more robust #141

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Need to make class destruction more robust #141

Description

Activity

leofang commented on Oct 10, 2024

leofang commented on Oct 15, 2024

rwgk commented on Nov 15, 2024

leofang commented on Nov 15, 2024

rwgk commented on Nov 15, 2024

leofang commented on Oct 4, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions