core/jobs.py:133-138, invoked from server.py:778 every 50 requests
if (job.end_time or job.start_time) < cutoff_time:
Falling back to start_time is deliberate — it reaps jobs that crashed without ever setting end_time. But there is no status check, so a job that is genuinely still RUNNING and simply older than max_age_hours is evicted too.
Failure
Index a large monorepo that takes longer than the threshold (default 24h). Mid-run, a routine cleanup deletes the job record. Afterwards:
update_job calls silently no-op — they are guarded by if job_id in self.jobs, so the worker keeps running with its progress going nowhere.
check_job_status reports not_found for a job that is actively writing to the graph.
The user is told the job does not exist while it continues to mutate the database.
Suggested fix
Skip jobs whose status is RUNNING (or PENDING) when aging by start_time, and reap only terminal-state jobs plus jobs with no recent progress update. Tracking a last_update_time would let genuinely-hung jobs still be reclaimed without evicting healthy long-running ones.
Environment
|
|
| Commit |
c0e0bed (main) |
| Python |
3.12.3, Linux |
Found during a source audit. The logic is unambiguous on reading; not reproduced, since triggering it needs a >24h job.
core/jobs.py:133-138, invoked fromserver.py:778every 50 requestsFalling back to
start_timeis deliberate — it reaps jobs that crashed without ever settingend_time. But there is nostatuscheck, so a job that is genuinely stillRUNNINGand simply older thanmax_age_hoursis evicted too.Failure
Index a large monorepo that takes longer than the threshold (default 24h). Mid-run, a routine cleanup deletes the job record. Afterwards:
update_jobcalls silently no-op — they are guarded byif job_id in self.jobs, so the worker keeps running with its progress going nowhere.check_job_statusreportsnot_foundfor a job that is actively writing to the graph.The user is told the job does not exist while it continues to mutate the database.
Suggested fix
Skip jobs whose status is
RUNNING(orPENDING) when aging bystart_time, and reap only terminal-state jobs plus jobs with no recent progress update. Tracking alast_update_timewould let genuinely-hung jobs still be reclaimed without evicting healthy long-running ones.Environment
c0e0bed(main)Found during a source audit. The logic is unambiguous on reading; not reproduced, since triggering it needs a >24h job.