Skip to content

HTTP server leaks memory in python3.7.1 #340

Closed
#356
Closed
HTTP server leaks memory in python3.7.1#340
#356
@josecv

Description

@josecv
josecv
opened on Nov 10, 2018

I'm using version 0.4.2, and it seems like the default http server, started via start_http_server leaks memory on every request.

Repro script:

#!/usr/bin/env python3
import gc
import time
import tracemalloc
from random import randint

from prometheus_client import Summary, start_http_server

# Create a metric to track time spent and requests made.
REQUEST_TIME = Summary('request_processing_seconds',
                       'Time spent processing request')


# Decorate function with metric.
@REQUEST_TIME.time()
def process_request(t):
    """A dummy function that takes some time."""
    time.sleep(t)


if __name__ == '__main__':
    # Start up the server to expose the metrics.
    start_http_server(8000)
    tracemalloc.start()
    snapshot = tracemalloc.take_snapshot()
    # Generate some requests.
    while True:
        process_request(randint(1, 5))
        gc.collect()
        snapshot2 = tracemalloc.take_snapshot()
        top_stats = snapshot2.compare_to(snapshot, 'lineno')
        print("[ Top 5 differences ]")
        for stat in top_stats[:5]:
            print(stat)

If you kick off the above script and then do:

while true; do curl localhost:8000; done

you can observe its memory footprint growing on every iteration:

[ Top 5 differences ]
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:799: size=946 KiB (+946 KiB), count=1682 (+1682), average=576 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:365: size=868 KiB (+868 KiB), count=1683 (+1683), average=528 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:500: size=422 KiB (+422 KiB), count=6753 (+6753), average=64 B
/Users/jose/code/prometheus-mem-leak/.venv/lib/python3.7/site-packages/prometheus_client/core.py:802: size=354 KiB (+354 KiB), count=5027 (+5027), average=72 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:899: size=263 KiB (+263 KiB), count=3366 (+3366), average=80 B
[ Top 5 differences ]
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:799: size=992 KiB (+992 KiB), count=1763 (+1763), average=576 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:365: size=910 KiB (+910 KiB), count=1764 (+1764), average=528 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:500: size=443 KiB (+443 KiB), count=7078 (+7078), average=64 B
/Users/jose/code/prometheus-mem-leak/.venv/lib/python3.7/site-packages/prometheus_client/core.py:802: size=370 KiB (+370 KiB), count=5266 (+5266), average=72 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:899: size=276 KiB (+276 KiB), count=3528 (+3528), average=80 B
[ Top 5 differences ]
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:799: size=1063 KiB (+1063 KiB), count=1889 (+1889), average=576 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:365: size=974 KiB (+974 KiB), count=1889 (+1889), average=528 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:500: size=474 KiB (+474 KiB), count=7584 (+7584), average=64 B
/Users/jose/code/prometheus-mem-leak/.venv/lib/python3.7/site-packages/prometheus_client/core.py:802: size=397 KiB (+397 KiB), count=5639 (+5639), average=72 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:899: size=295 KiB (+295 KiB), count=3778 (+3778), average=80 B
[ Top 5 differences ]
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:799: size=1115 KiB (+1115 KiB), count=1982 (+1982), average=576 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:365: size=1022 KiB (+1022 KiB), count=1983 (+1983), average=528 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:500: size=497 KiB (+497 KiB), count=7956 (+7956), average=64 B
/Users/jose/code/prometheus-mem-leak/.venv/lib/python3.7/site-packages/prometheus_client/core.py:802: size=416 KiB (+416 KiB), count=5922 (+5922), average=72 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:899: size=310 KiB (+310 KiB), count=3966 (+3966), average=80 B
[ Top 5 differences ]
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:799: size=1151 KiB (+1151 KiB), count=2046 (+2046), average=576 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:365: size=1055 KiB (+1055 KiB), count=2047 (+2047), average=528 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:500: size=514 KiB (+514 KiB), count=8213 (+8213), average=64 B
/Users/jose/code/prometheus-mem-leak/.venv/lib/python3.7/site-packages/prometheus_client/core.py:802: size=430 KiB (+430 KiB), count=6108 (+6108), average=72 B
/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:899: size=320 KiB (+320 KiB), count=4094 (+4094), average=80 B

it's clearly coming from the server, because as soon as you stop the curl, memory usage stops growing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions