Fix thread leak in Python 3.7 prometheus#340 (prometheus#356)

sbrandtb · brian-brazil · commit 5aa256d8aab3 · 2018-12-11T09:24:54.000Z
* Fix thread leak in Python 3.7 prometheus#340 The leak is caused by the fact that in Python 3.7, the default behavior of the `ThreadingMixin` is to use no daemon threads, but to request to block on threads on close. Because of that, it collects references to all created threads, creating the "leak": https://github.com/python/cpython/blob/v3.7.0/Lib/socketserver.py#L661 * Python 3.7: `block_on_close` is `True`: https://github.com/python/cpython/blob/v3.7.0/Lib/socketserver.py#L635 * Python 3.6: `_block_on_close` is `False`: https://github.com/python/cpython/blob/v3.6.7/Lib/socketserver.py#L639 * Python 2.7: There is no `block_on_close`, thus no logic for collecting references: https://github.com/python/cpython/blob/v2.7.15/Lib/SocketServer.py#L582 Fix by setting `daemon_threads` to `True`, which in our case should be a reasonable setting for all Python versions. Also, the new in Python 3.7 `ThreadingHTTPServer` stdlib class also sets it by default: https://github.com/python/cpython/blob/v3.7.0/Lib/http/server.py#L144 Signed-off-by: Sebastian Brandt <sebastian.brandt@friday.de>
diff --git a/prometheus_client/exposition.py b/prometheus_client/exposition.py
@@ -173,6 +173,12 @@ def factory(cls, registry):
 
 class _ThreadingSimpleServer(ThreadingMixIn, HTTPServer):
     """Thread per request HTTP server."""
+    # Make worker threads "fire and forget". Beginning with Python 3.7 this
+    # prevents a memory leak because ``ThreadingMixIn`` starts to gather all
+    # non-daemon threads in a list in order to join on them at server close.
+    # Enabling daemon threads virtually makes ``_ThreadingSimpleServer`` the
+    # same as Python 3.7's ``ThreadingHTTPServer``.
+    daemon_threads = True
 
 
 def start_http_server(port, addr='', registry=REGISTRY):
