Skip to content

Fix memory-safety and correctness bugs surfaced by Coverity audit (part 1) - #22266

Merged
stelfrag merged 15 commits into
netdata:masterfrom
stelfrag:cov_fix_part1
Apr 24, 2026
Merged

Fix memory-safety and correctness bugs surfaced by Coverity audit (part 1)#22266
stelfrag merged 15 commits into
netdata:masterfrom
stelfrag:cov_fix_part1

Conversation

@stelfrag

@stelfrag stelfrag commented Apr 24, 2026

Copy link
Copy Markdown
Collaborator
Summary

Summary by cubic

Fixes memory-safety, resource leaks, and race conditions found by Coverity across the agent. Improves startup robustness, journal migration, alert serialization, and worker shutdown.

  • Bug Fixes
    • Free leaked buffers in claim split-file flow; always release metadata cleanup and UUID-deletion Judy lists on shutdown.
    • Close leaked descriptors: close duplicate DIRs in systemd-journal (including on dictionary_set() NULL), and close migrated journal fds on mmap/build failures.
    • Remove access()/mkdir() race for machine GUID; on EEXIST verify the path is a directory.
    • Clone alert config keys to avoid a use-after-free; require closing brace in ${label:...} parsing.
    • Clamp SQLite non_clear_duration to uint32 range.
    • Make workers_exit checks atomic in functions evloop; fix fatal-signal log to include the thread id.
    • In worker utilization, only free workname after successful JudyHSDel(), zero-initialize JError_t, and adjust memory accounting; log unexpected delete failures.

Written for commit 39d8bde. Summary will update on new commits.

ktsaou added 15 commits April 24, 2026 18:11
Coverity CID 457948 (TOCTOU): machine_guid_get_or_create() checked the
registry path with access() before calling mkdir() while startup still runs
with elevated privileges. Use mkdir() with EEXIST handling directly, matching
existing Netdata directory-creation idioms and removing the check/use race.
Coverity CID 440042 (INTEGER_OVERFLOW): health_alarm_log_populate() read
non_clear_duration from SQLite directly into the uint32_t ACLK field.
Clamp negative values to 0 and oversized values to UINT32_MAX before
serializing alert log entries.
Coverity CID 468190 (CHECKED_RETURN): worker_unregister() freed the last workname even when JudyHSDel() failed, which can leave a freed workname still indexed in the JudyHS table. Only free and account away the workname after a successful delete, and log unexpected JudyHS delete failures.
mkdir() returning EEXIST can mean either an existing directory (the case
we want to accept) or an existing non-directory file at the same path.
Treat the latter as a failure so callers get a clear error instead of
ENOTDIR on subsequent writes.
Initialize the local `JError_t` so the `JU_ERRNO` / `JU_ERRID` values
logged on a JudyHSDel() JERR return are always defined, even if Judy
leaves the struct partially populated in some future path.
Coverity CID 455300 (RESOURCE_LEAK): start_metadata_hosts() skipped
store_ctx_cleanup_list() once shutdown was requested, leaving a
worker-owned Judy list unreleased after ownership moved out of
metadata_event_loop(). Call the helper unconditionally so shutdown still
frees the list while its internal guard suppresses database work.
Coverity CID 471887 (RESOURCE_LEAK): metadata workers transferred
pending_uuid_deletion out of the event loop, but skipped
do_pending_uuid_deletion() after shutdown started. Always run the helper
so shutdown still frees the Judy list and queued UUIDs, while keeping
metadata cleanup disabled during shutdown.
Coverity CID 405474 (RESOURCE_LEAK): initialize `fd_v2` before
`nd_mmap_advanced()` and close it on both migration failure paths.
This keeps the success-path ownership transfer through
`journalfile_v2_data_set()` unchanged while preventing leaked file
descriptors when mmap setup or journal build fails.
Coverity CID 459644 (RESOURCE_LEAK): nd_journal_directory_scan_recursively()
opened a directory before checking whether it had already been scanned.
Close the duplicate-path DIR handle on the early-return path so repeated
directories do not leak file descriptors.
Coverity CID 439996 (INTEGER_OVERFLOW): only process `${label:...}` placeholders when the token is complete and ends with `}`. This prevents the label-name truncation math from indexing before the local buffer on unterminated placeholders.
Treat a NULL return from dictionary_set() the same as an existing entry:
close the opendir() handle and return, so the directory descriptor is not
leaked if the dict insertion fails.
alerts_v2_insert_callback() created `t->configs` with
DICT_OPTION_NAME_LINK_DONT_CLONE. alerts_v2_add() then inserted a UUID
derived from nd_uuid_unparse_full() into the dictionary — but the buffer
holding that UUID was a local on the caller's stack. In
LINK_DONT_CLONE mode the dictionary stores the caller pointer verbatim,
so every inserted key became a dangling stack reference as soon as
alerts_v2_add() returned. Later dictionary operations (including
dictionary_destroy()) would then dereference freed stack memory via
strlen(item->caller_name), a use-after-free on the /api/v2/alerts path.

Drop DICT_OPTION_NAME_LINK_DONT_CLONE from the `t->configs` dictionary so
names are cloned into stable storage. `t->nodes` is left unchanged
because its names come from persistent rrdhost->machine_guid strings, not
from stack buffers.

Related: Coverity CID 414658 flagged an OVERRUN on the insertion line;
the trace itself was a tool-model FP against the commented-out XXH3
hashtable path, but investigating it surfaced this real stack-escape
lifetime bug.
Coverity CID 442342 (RESOURCE_LEAK): release the token and rooms
buffers returned by read_by_filename() after
claim_agent_from_split_files() finishes using them.

This normal-exit cleanup also fixes sibling CID 442346, which
reports the same leak root cause for rooms.
Coverity CID 457745 (STRING_OVERFLOW): the reported overflow is not real
because strcatz() bounds the buffer, but the same log path dropped the
thread id by not advancing len after print_uint64(). Update len after the
integer write so the fatal signal message preserves the thread id.
Coverity CID 425867 (MISSING_LOCK): use atomic load/store for the shared workers_exit latch in the worker event loop. This removes the unlocked cross-thread race without changing the existing mutex and condition-variable flow.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 11 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.
Architecture diagram
sequenceDiagram
    participant Daemon as "Daemon / Signal Handler"
    participant MGUID as "Machine GUID Service"
    participant JM as "Journal Migrator"
    participant MW as "Metadata Worker"
    participant DB as "SQLite / DB Engine"
    participant WR as "Worker Registry"
    participant FS as "File System / OS"

    Note over Daemon, FS: Startup & Machine Initialization
    Daemon->>MGUID: machine_guid_get_or_create()
    MGUID->>FS: CHANGED: mkdir(pathname)
    alt Dir exists
        FS-->>MGUID: EEXIST
        MGUID->>FS: NEW: stat(pathname) to verify S_ISDIR
    else Created
        FS-->>MGUID: Success
    end
    MGUID->>FS: Open/Read GUID file

    Note over JM, DB: DB Engine Journal Migration (v1 to v2)
    JM->>FS: nd_mmap_advanced() (opens fd_v2)
    alt mmap or build fails
        JM->>FS: NEW: close(fd_v2) to prevent leak
        JM->>FS: unlink(path)
    else Success
        JM->>FS: Continue migration
    end

    Note over MW, DB: Metadata Persistence & Shutdown
    loop Background Tasks
        MW->>DB: store_alert_transitions()
        MW->>DB: store_ctx_cleanup_list()
        Note right of MW: NEW: Always release Judy lists<br/>even during shutdown
        MW->>DB: NEW: do_pending_uuid_deletion()
        opt !SHUTDOWN_REQUESTED
            MW->>DB: run_metadata_cleanup()
        end
    end

    Note over Daemon, WR: Shutdown & Worker Cleanup
    Daemon->>Daemon: Signal received (SIGTERM/SIGINT)
    Daemon->>WR: CHANGED: Atomic store workers_exit = true
    WR->>WR: Worker loop detects exit via atomic load
    WR->>WR: worker_unregister()
    WR->>WR: NEW: Check JudyHSDel() success before free
    WR->>WR: CHANGED: Update memory accounting with Judy pulse

    Note over Daemon, FS: Fatal Signal Logging
    alt Crash / Fatal Signal
        Daemon->>FS: NEW: Log thread ID (gettid) in async-safe handler
    end
Loading

@stelfrag
stelfrag marked this pull request as ready for review April 24, 2026 16:17
@stelfrag
stelfrag requested a review from thiagoftsm as a code owner April 24, 2026 16:17
Copilot AI review requested due to automatic review settings April 24, 2026 16:17
@stelfrag
stelfrag marked this pull request as draft April 24, 2026 16:17
@sonarqubecloud

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a set of Coverity-reported memory-safety and correctness issues across core agent components (worker utilization, functions event loop, health variable parsing, SQLite metadata/ACLK alert serialization, dbengine journal migration, systemd-journal scanning, claim flow, and daemon utilities).

Changes:

  • Fix resource-management and memory-safety issues (fd/DIR leaks, correct freeing on Judy delete outcomes, free claim split-file buffers).
  • Harden correctness and robustness (atomic worker-exit flag reads/writes, safer ${label:...} parsing, clamp SQLite duration narrowing, safer machine-guid directory creation).
  • Prevent lifetime bugs in API v2 alert contexts by ensuring dictionary keys are cloned.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/libnetdata/worker_utilization/worker_utilization.c Avoid freeing workname when JudyHS delete fails; improve Judy error reporting and memory accounting.
src/libnetdata/functions_evloop/functions_evloop.c Make workers_exit access atomic to avoid data races between mutex and non-mutex checks.
src/health/rrdcalc.c Require a closing } for ${label:...} variable substitution to prevent malformed parsing issues.
src/database/sqlite/sqlite_metadata.c Always release worker-owned Judy lists for ctx cleanup and UUID deletion, even during shutdown.
src/database/sqlite/sqlite_aclk_alert.c Clamp non_clear_duration to the uint32_t range when reading from SQLite.
src/database/engine/journalfile.c Close migrated journal fd on mmap/build failure paths to prevent fd leaks.
src/database/contexts/api_v2_contexts_alerts.c Ensure alert config dictionary clones keys (prevents stack/key lifetime issues).
src/daemon/signal-handler.c Correctly advance buffer length when printing thread id in fatal-signal logging.
src/daemon/machine-guid.c Remove access()/mkdir() TOCTOU; on EEXIST verify the path is a directory.
src/collectors/systemd-journal.plugin/systemd-journal-files.c Close DIR handle on duplicate directory detection and on dictionary_set() failure.
src/claim/claim-with-api.c Free token/rooms buffers on the success path of split-file claim.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/collectors/systemd-journal.plugin/systemd-journal-files.c
@stelfrag
stelfrag removed the request for review from Ferroin April 24, 2026 16:40
@stelfrag
stelfrag marked this pull request as ready for review April 24, 2026 16:40

@thiagoftsm thiagoftsm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR is working as expected. No coredump or issues found during runtime. LGTM!

@stelfrag
stelfrag merged commit a50db9c into netdata:master Apr 24, 2026
162 checks passed
@stelfrag
stelfrag deleted the cov_fix_part1 branch April 25, 2026 13:02
@stelfrag stelfrag mentioned this pull request Jun 22, 2026
Ferroin pushed a commit that referenced this pull request Jul 15, 2026
…rt 1) (#22266)

* daemon: remove machine guid access precheck

Coverity CID 457948 (TOCTOU): machine_guid_get_or_create() checked the
registry path with access() before calling mkdir() while startup still runs
with elevated privileges. Use mkdir() with EEXIST handling directly, matching
existing Netdata directory-creation idioms and removing the check/use race.

* sqlite: clamp alert non_clear_duration for aclk

Coverity CID 440042 (INTEGER_OVERFLOW): health_alarm_log_populate() read
non_clear_duration from SQLite directly into the uint32_t ACLK field.
Clamp negative values to 0 and oversized values to UINT32_MAX before
serializing alert log entries.

* worker_utilization: handle JudyHSDel failure

Coverity CID 468190 (CHECKED_RETURN): worker_unregister() freed the last workname even when JudyHSDel() failed, which can leave a freed workname still indexed in the JudyHS table. Only free and account away the workname after a successful delete, and log unexpected JudyHS delete failures.

* daemon: verify machine guid path is a directory on EEXIST

mkdir() returning EEXIST can mean either an existing directory (the case
we want to accept) or an existing non-directory file at the same path.
Treat the latter as a failure so callers get a clear error instead of
ENOTDIR on subsequent writes.

* worker_utilization: zero-initialize JError_t on the JudyHSDel path

Initialize the local `JError_t` so the `JU_ERRNO` / `JU_ERRID` values
logged on a JudyHSDel() JERR return are always defined, even if Judy
leaves the struct partially populated in some future path.

* sqlite: free metadata cleanup list on shutdown

Coverity CID 455300 (RESOURCE_LEAK): start_metadata_hosts() skipped
store_ctx_cleanup_list() once shutdown was requested, leaving a
worker-owned Judy list unreleased after ownership moved out of
metadata_event_loop(). Call the helper unconditionally so shutdown still
frees the list while its internal guard suppresses database work.

* sqlite: fix pending uuid deletion leak on shutdown

Coverity CID 471887 (RESOURCE_LEAK): metadata workers transferred
pending_uuid_deletion out of the event loop, but skipped
do_pending_uuid_deletion() after shutdown started. Always run the helper
so shutdown still frees the Judy list and queued UUIDs, while keeping
metadata cleanup disabled during shutdown.

* dbengine: close migrated journal fds on failure

Coverity CID 405474 (RESOURCE_LEAK): initialize `fd_v2` before
`nd_mmap_advanced()` and close it on both migration failure paths.
This keeps the success-path ownership transfer through
`journalfile_v2_data_set()` unchanged while preventing leaked file
descriptors when mmap setup or journal build fails.

* systemd-journal: fix duplicate scan dir handle leak

Coverity CID 459644 (RESOURCE_LEAK): nd_journal_directory_scan_recursively()
opened a directory before checking whether it had already been scanned.
Close the duplicate-path DIR handle on the early-return path so repeated
directories do not leak file descriptors.

* health: fix malformed ${label:...} parsing

Coverity CID 439996 (INTEGER_OVERFLOW): only process `${label:...}` placeholders when the token is complete and ends with `}`. This prevents the label-name truncation math from indexing before the local buffer on unterminated placeholders.

* systemd-journal: guard dictionary_set() NULL return in recursive scan

Treat a NULL return from dictionary_set() the same as an existing entry:
close the opendir() handle and return, so the directory descriptor is not
leaked if the dict insertion fails.

* contexts: clone alert config keys to avoid stack-escape UAF

alerts_v2_insert_callback() created `t->configs` with
DICT_OPTION_NAME_LINK_DONT_CLONE. alerts_v2_add() then inserted a UUID
derived from nd_uuid_unparse_full() into the dictionary — but the buffer
holding that UUID was a local on the caller's stack. In
LINK_DONT_CLONE mode the dictionary stores the caller pointer verbatim,
so every inserted key became a dangling stack reference as soon as
alerts_v2_add() returned. Later dictionary operations (including
dictionary_destroy()) would then dereference freed stack memory via
strlen(item->caller_name), a use-after-free on the /api/v2/alerts path.

Drop DICT_OPTION_NAME_LINK_DONT_CLONE from the `t->configs` dictionary so
names are cloned into stable storage. `t->nodes` is left unchanged
because its names come from persistent rrdhost->machine_guid strings, not
from stack buffers.

Related: Coverity CID 414658 flagged an OVERRUN on the insertion line;
the trace itself was a tool-model FP against the commented-out XXH3
hashtable path, but investigating it surfaced this real stack-escape
lifetime bug.

* claim: free split-file claim buffers

Coverity CID 442342 (RESOURCE_LEAK): release the token and rooms
buffers returned by read_by_filename() after
claim_agent_from_split_files() finishes using them.

This normal-exit cleanup also fixes sibling CID 442346, which
reports the same leak root cause for rooms.

* daemon: fix thread id in deadly signal log

Coverity CID 457745 (STRING_OVERFLOW): the reported overflow is not real
because strcatz() bounds the buffer, but the same log path dropped the
thread id by not advancing len after print_uint64(). Update len after the
integer write so the fatal signal message preserves the thread id.

* functions_evloop: make workers_exit checks atomic

Coverity CID 425867 (MISSING_LOCK): use atomic load/store for the shared workers_exit latch in the worker event loop. This removes the unlocked cross-thread race without changing the existing mutex and condition-variable flow.

---------

Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
(cherry picked from commit a50db9c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants