When you've used a lot of pids and have a lot of metrics files lying around, there is a high probability that mark_process_dead
$ prometheus_multiproc_dir=/tmp/prometheus_metrics python
Python 3.7.3 (default, May 8 2019, 05:31:59)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from prometheus_client import multiprocess
>>> from prometheus_client import generate_latest, CollectorRegistry
>>> registry = CollectorRegistry()
>>> multiprocess.MultiProcessCollector(registry)
<prometheus_client.multiprocess.MultiProcessCollector object at 0x7ffb0b1efa58>
>>> data = generate_latest(registry)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/grocker/app.venv/lib/python3.7/site-packages/prometheus_client/exposition.py", line 89, in generate_latest
for metric in registry.collect():
File "/home/grocker/app.venv/lib/python3.7/site-packages/prometheus_client/registry.py", line 75, in collect
for metric in collector.collect():
File "/home/grocker/app.venv/lib/python3.7/site-packages/prometheus_client/multiprocess.py", line 30, in collect
return self.merge(files, accumulate=True)
File "/home/grocker/app.venv/lib/python3.7/site-packages/prometheus_client/multiprocess.py", line 43, in merge
d = MmapedDict(f, read_mode=True)
File "/home/grocker/app.venv/lib/python3.7/site-packages/prometheus_client/mmap_dict.py", line 39, in __init__
self._f = open(filename, 'rb' if read_mode else 'a+b')
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/prometheus_metrics/gauge_livesum_37367.db'
Hello,
When you've used a lot of pids and have a lot of metrics files lying around, there is a high probability that
mark_process_deadclient_python/prometheus_client/multiprocess.py
Lines 140 to 147 in 8a4f2de
will be called between the
glob.globcall and theself.mergeone:client_python/prometheus_client/multiprocess.py
Lines 136 to 137 in 8a4f2de
leading to:
An easy fix could be #424.