Skip to content

Conversation

@ktsaou
Copy link
Member

@ktsaou ktsaou commented Nov 25, 2025


Summary by cubic

Adds float support end-to-end for dimensions. Plugins can mark DIMENSION type=float and send float values in SET, and the pipeline (calculation, streaming, exporting, health) now carries doubles with backward-compatible fallbacks.

  • New Features

    • plugins.d: DIMENSION supports options type=float; SET accepts floats; stores last_collected as double when float.
    • Core: RRDDIM gains type-aware storage (int/float) with helpers; new rrddim_set_by_pointer_double; calculations use doubles while preserving int overflow/reset logic for incremental.
    • Streaming: adds FLOATBASELINE capability; sends float baselines when supported and includes type=int|float in chart definitions; falls back to int baseline when not supported.
    • Exporters/Health: Graphite, JSON, OpenTSDB, Prometheus, and health APIs emit last_collected as double for float dims.
    • go.d/ping: RTT and packet_loss now reported as floats (RTT in seconds); charts mark these dimensions as float.
  • Migration

    • Plugin authors: add type=float to DIMENSION options to use floats, and send float values in SET lines.
    • No action needed for streaming peers; older peers receive int baselines (downcast) automatically.

Written for commit 6abba18. Summary will update on new commits.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8 issues found across 21 files

Prompt for AI agents (all 8 issues)

Understand the root cause of the following 8 issues and fix them.


<file name="src/streaming/protocol/command-begin-set-end-v2.c">

<violation number="1" location="src/streaming/protocol/command-begin-set-end-v2.c:61">
Encoding integer baselines now loses precision for values &gt;2^53 because the value is routed through a double before being cast back to int64, corrupting streamed state for large counters.</violation>
</file>

<file name="src/streaming/protocol/command-begin-set-end-v1.c">

<violation number="1" location="src/streaming/protocol/command-begin-set-end-v1.c:32">
Casting the collected double to int64 before printing truncates all fractional dimension values, so streaming SET lines still transmit integers instead of floats.</violation>
</file>

<file name="src/streaming/stream-replication-sender.c">

<violation number="1" location="src/streaming/stream-replication-sender.c:244">
Casting the integer baseline through rrddim_last_collected_as_double() truncates large counters (&gt;2^53) before sending them. Use the raw int accessor so replication baselines remain exact.</violation>
</file>

<file name="src/database/rrdset-collection.c">

<violation number="1" location="src/database/rrdset-collection.c:678">
Reset handling for PCENT_OVER_DIFF_TOTAL now calls the float setter unconditionally, so integer dimensions never update their last value after an overflow and continue to report negative deltas.</violation>

<violation number="2" location="src/database/rrdset-collection.c:681">
Summing rrddim_*_as_double() into total_number (long long) truncates every float dimension’s contribution, breaking percentage calculations for non-integer dimensions.</violation>
</file>

<file name="src/plugins.d/pluginsd_parser.c">

<violation number="1" location="src/plugins.d/pluginsd_parser.c:1111">
Float dimensions are reset using the int-only accessor, so their collected_value fields remain stale after END2; use the float setter when rrddim_is_float(rd).</violation>
</file>

<file name="src/database/rrddim.c">

<violation number="1" location="src/database/rrddim.c:618">
`rrddim_set_by_pointer_double()` truncates floats to 64‑bit integers before storing, so the new float DIMENSION support never records fractional data.</violation>

<violation number="2" location="src/database/rrddim.c:645">
The float branch of `rrddim_timed_set_by_pointer()` returns the last value cast to `collected_number`, so even the new double API always reports an integer result.</violation>
</file>

Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds end-to-end float support for dimensions in Netdata, allowing plugins to mark dimensions as float type and send float values through the entire data pipeline (collection, streaming, exporting, and health APIs).

Key Changes

  • Streaming Protocol: Adds STREAM_CAP_FLOAT_BASELINE capability for float baseline transmission, with backward-compatible fallback to int baselines for older peers
  • Core Data Structures: Introduces type-aware storage using a union in RRDDIM struct with separate lanes for int and float values, selected by RRDDIM_OPTION_VALUE_FLOAT flag
  • Algorithm Support: Updates collection algorithms (INCREMENTAL, ABSOLUTE, PCENT_*) to handle float dimensions with appropriate overflow/reset detection logic

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/streaming/stream-capabilities.h Defines new STREAM_CAP_FLOAT_BASELINE capability flag
src/streaming/stream-capabilities.c Registers FLOATBASELINE capability string and enables it by default
src/streaming/stream-replication-sender.c Sends float or int baselines based on dimension type and peer capability
src/streaming/protocol/command-chart-definition.c Includes type=float or type=int in dimension definition options
src/streaming/protocol/command-begin-set-end-v2.c Handles float baseline encoding in SET_V2 command with capability check
src/streaming/protocol/command-begin-set-end-v1.c Outputs float or int values in SET command based on dimension type
src/plugins.d/pluginsd_replication.c Parses float or int baselines during dimension state replay
src/plugins.d/pluginsd_parser.c Parses type=float option in DIMENSION command; handles float values in SET and SET_V2; resets collected values by type in END_V2
src/plugins.d/README.md Documents the new type=float and type=int options (misplaced in CHART section)
src/health/rrdvar.c Outputs float dimensions as doubles in health API JSON responses
src/health/health_variable.c Uses type-aware accessor for dimension lookups in health expressions
src/exporting/prometheus/remote_write/remote_write.c Exports float dimension values correctly
src/exporting/prometheus/prometheus.c Handles float dimensions in as-collected metric output
src/exporting/opentsdb/opentsdb.c Formats float dimension values using double format specifiers
src/exporting/json/json.c Outputs float dimension values as doubles in JSON export
src/exporting/graphite/graphite.c Formats float dimension values using double format specifiers
src/database/rrdset-collection.c Implements float-aware collection algorithms with separate int/float overflow detection; uses type-aware accessors throughout
src/database/rrddim.h Defines union structure for int/float storage, adds type-checking macros and accessor functions
src/database/rrddim.c Implements rrddim_set_by_pointer_double() and updates existing functions to handle float dimensions
src/database/engine/dbengine-unittest.c Updates test code to use new accessor functions
src/database/engine/dbengine-stresstest.c Updates stress test to use new accessor functions
src/collectors/proc.plugin/proc_diskstats.c Updates to use rrddim_last_collected_raw_int() accessor

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

@ktsaou ktsaou changed the title Plugins.d doubles + OTEL deltas Plugins.d doubles Nov 25, 2025
@ilyam8 ilyam8 self-requested a review as a code owner November 27, 2025 18:15
@ktsaou
Copy link
Member Author

ktsaou commented Jan 13, 2026

@ilyam8 have you tested this? We should merge it, no?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +956 to +957
rrddim_set_collected_int(rd, 0);
rrddim_set_collected_float(rd, 0.0);
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both int and float collection values are being reset unconditionally. This could be inefficient since only one lane is actually in use per dimension. Consider resetting only the active lane based on the dimension type to avoid redundant operations.

Suggested change
rrddim_set_collected_int(rd, 0);
rrddim_set_collected_float(rd, 0.0);
if (rrddim_is_float(rd))
rrddim_set_collected_float(rd, 0.0);
else
rrddim_set_collected_int(rd, 0);

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants