Skip to content

fix(journal): improve journal file access error handling and validation - #22310

Merged
stelfrag merged 4 commits into
netdata:masterfrom
stelfrag:fix_upd_retention
Apr 30, 2026
Merged

fix(journal): improve journal file access error handling and validation#22310
stelfrag merged 4 commits into
netdata:masterfrom
stelfrag:fix_upd_retention

Conversation

@stelfrag

@stelfrag stelfrag commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator
Summary
  • Access header fields after the PROTECTED_ACCESS_SETUP

Summary by cubic

Hardened journal file access in the retention scan and simplified the search loop to avoid invalid indexing. Prevents out-of-bounds reads and cleanly skips truncated or invalid journals.

  • Bug Fixes
    • Use journal file size from journalfile_v2_data_acquire_with_hint and validate metric_offset/metric_count with overflow-safe bounds; skip when the metric list would exceed the file.
    • Read and use header fields only after PROTECTED_ACCESS_SETUP; handle mmap/protection failures.
    • Simplify journal search: remove redundant checks/counters and break early when journal_search_start >= journal_metric_count.
    • Improve logs: report access failures; include metric list vs file size details.
    • On access failure, release and continue, skipping the early-release path to keep state consistent.

Written for commit 7870cb2. Summary will update on new commits. Review in cubic

@stelfrag
stelfrag marked this pull request as ready for review April 28, 2026 20:04
Copilot AI review requested due to automatic review settings April 28, 2026 20:04
@stelfrag
stelfrag marked this pull request as draft April 28, 2026 20:04

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.
Architecture diagram
sequenceDiagram
    participant RE as RRDEngine
    participant J2 as JournalV2 (mmap)
    participant Mem as Protected Memory Area
    participant Log as Logger

    Note over RE, Mem: Retention Scan: find_uuid_first_time loop

    RE->>J2: NEW: journalfile_v2_data_acquire_with_hint()
    J2-->>RE: Return j2_header pointer & NEW: journal_v2_file_size

    alt Header acquisition failed
        RE->>RE: Skip to next datafile
    end

    RE->>RE: PROTECTED_ACCESS_SETUP (mmap guards)

    alt Access Signal (SIGBUS/SIGSEGV) or no_signal_received is false
        RE->>Log: CHANGED: "failed to access journalfile"
    else Access Successful
        RE->>Mem: CHANGED: Read header fields (start_time, offsets, count)
        
        Note over RE: NEW: Strict Bounds Validation
        RE->>RE: Check __builtin_mul_overflow(metric_count, size)
        
        alt NEW: Validation Error (Offset/Size > File Size)
            RE->>Log: NEW: "metric list exceeds journal file size"
            RE->>RE: Set journal_access_failed = true
        else Validation Success
            loop For each UUID entry
                RE->>RE: NEW: Check if journal_search_start >= metric_count
                RE->>Mem: Access uuid_list[index]
                Note right of Mem: Binary search or direct access
            end
        end
    end

    RE->>J2: journalfile_v2_data_release()

    opt NEW: Access Failure skip
        RE->>RE: Skip early-release logic
        RE->>RE: Continue to next datafile
    end

    RE->>RE: datafile_release_and_acquire_next_for_retention()
Loading

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts rrdengine’s journal v2 scanning so journal header fields are accessed only after PROTECTED_ACCESS_SETUP, and adds additional bounds validation when walking the journal metric list (to better handle corrupted/unreadable journal files).

Changes:

  • Pass data_size to journalfile_v2_data_acquire_with_hint() and use it for validating metric-list bounds.
  • Move journal header reads (start_time_ut, metric_offset, metric_count) under PROTECTED_ACCESS_SETUP.
  • Add guard logic for running past metric_count and skip journals when protected access / validation fails.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/database/engine/rrdengine.c Outdated
@stelfrag
stelfrag requested a review from Copilot April 28, 2026 20:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Improves robustness of journal v2 file access during retention scanning by performing header/metric list validation under protected access and skipping truncated/invalid journals safely.

Changes:

  • Read header fields only after PROTECTED_ACCESS_SETUP and treat protection failures as access failures.
  • Validate metric_offset/metric_count against the journal file size using overflow-safe arithmetic and skip invalid files.
  • Guard against journal_search_start >= journal_metric_count to avoid invalid indexing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/database/engine/rrdengine.c
Comment thread src/database/engine/rrdengine.c Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens journal v2 access during DBENGINE retention scanning by moving header-derived reads under protected access and validating metric directory bounds against the mapped file size to avoid out-of-bounds access on truncated/corrupt journals.

Changes:

  • Acquire journal v2 mapped size via journalfile_v2_data_acquire_with_hint() and validate metric_offset/metric_count with overflow-safe arithmetic before indexing.
  • Read header fields (start_time_ut, metric_offset, metric_count) only under PROTECTED_ACCESS_SETUP and treat access faults as “skip and continue”.
  • Add guard for journal_search_start >= journal_metric_count to prevent invalid indexing during the scan loop.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/database/engine/rrdengine.c Outdated
Comment thread src/database/engine/rrdengine.c Outdated
@sonarqubecloud

Copy link
Copy Markdown

@stelfrag
stelfrag marked this pull request as ready for review April 30, 2026 06:33

@thiagoftsm thiagoftsm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found during runtime, LGTM!

@stelfrag
stelfrag merged commit 26b26ac into netdata:master Apr 30, 2026
158 checks passed
@stelfrag
stelfrag deleted the fix_upd_retention branch April 30, 2026 12:44
@stelfrag stelfrag mentioned this pull request Jun 22, 2026
Ferroin pushed a commit that referenced this pull request Jul 15, 2026
…on (#22310)

* fix(journal): improve journal file access error handling and validation

* Address review comment

* Address review comment 2

* fix(rrdengine): simplify journal search logic and remove redundant checks

(cherry picked from commit 26b26ac)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants