-
Notifications
You must be signed in to change notification settings - Fork 6.3k
Plugins.d doubles #21349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Plugins.d doubles #21349
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
8 issues found across 21 files
Prompt for AI agents (all 8 issues)
Understand the root cause of the following 8 issues and fix them.
<file name="src/streaming/protocol/command-begin-set-end-v2.c">
<violation number="1" location="src/streaming/protocol/command-begin-set-end-v2.c:61">
Encoding integer baselines now loses precision for values >2^53 because the value is routed through a double before being cast back to int64, corrupting streamed state for large counters.</violation>
</file>
<file name="src/streaming/protocol/command-begin-set-end-v1.c">
<violation number="1" location="src/streaming/protocol/command-begin-set-end-v1.c:32">
Casting the collected double to int64 before printing truncates all fractional dimension values, so streaming SET lines still transmit integers instead of floats.</violation>
</file>
<file name="src/streaming/stream-replication-sender.c">
<violation number="1" location="src/streaming/stream-replication-sender.c:244">
Casting the integer baseline through rrddim_last_collected_as_double() truncates large counters (>2^53) before sending them. Use the raw int accessor so replication baselines remain exact.</violation>
</file>
<file name="src/database/rrdset-collection.c">
<violation number="1" location="src/database/rrdset-collection.c:678">
Reset handling for PCENT_OVER_DIFF_TOTAL now calls the float setter unconditionally, so integer dimensions never update their last value after an overflow and continue to report negative deltas.</violation>
<violation number="2" location="src/database/rrdset-collection.c:681">
Summing rrddim_*_as_double() into total_number (long long) truncates every float dimension’s contribution, breaking percentage calculations for non-integer dimensions.</violation>
</file>
<file name="src/plugins.d/pluginsd_parser.c">
<violation number="1" location="src/plugins.d/pluginsd_parser.c:1111">
Float dimensions are reset using the int-only accessor, so their collected_value fields remain stale after END2; use the float setter when rrddim_is_float(rd).</violation>
</file>
<file name="src/database/rrddim.c">
<violation number="1" location="src/database/rrddim.c:618">
`rrddim_set_by_pointer_double()` truncates floats to 64‑bit integers before storing, so the new float DIMENSION support never records fractional data.</violation>
<violation number="2" location="src/database/rrddim.c:645">
The float branch of `rrddim_timed_set_by_pointer()` returns the last value cast to `collected_number`, so even the new double API always reports an integer result.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds end-to-end float support for dimensions in Netdata, allowing plugins to mark dimensions as float type and send float values through the entire data pipeline (collection, streaming, exporting, and health APIs).
Key Changes
- Streaming Protocol: Adds
STREAM_CAP_FLOAT_BASELINEcapability for float baseline transmission, with backward-compatible fallback to int baselines for older peers - Core Data Structures: Introduces type-aware storage using a union in
RRDDIMstruct with separate lanes for int and float values, selected byRRDDIM_OPTION_VALUE_FLOATflag - Algorithm Support: Updates collection algorithms (INCREMENTAL, ABSOLUTE, PCENT_*) to handle float dimensions with appropriate overflow/reset detection logic
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/streaming/stream-capabilities.h | Defines new STREAM_CAP_FLOAT_BASELINE capability flag |
| src/streaming/stream-capabilities.c | Registers FLOATBASELINE capability string and enables it by default |
| src/streaming/stream-replication-sender.c | Sends float or int baselines based on dimension type and peer capability |
| src/streaming/protocol/command-chart-definition.c | Includes type=float or type=int in dimension definition options |
| src/streaming/protocol/command-begin-set-end-v2.c | Handles float baseline encoding in SET_V2 command with capability check |
| src/streaming/protocol/command-begin-set-end-v1.c | Outputs float or int values in SET command based on dimension type |
| src/plugins.d/pluginsd_replication.c | Parses float or int baselines during dimension state replay |
| src/plugins.d/pluginsd_parser.c | Parses type=float option in DIMENSION command; handles float values in SET and SET_V2; resets collected values by type in END_V2 |
| src/plugins.d/README.md | Documents the new type=float and type=int options (misplaced in CHART section) |
| src/health/rrdvar.c | Outputs float dimensions as doubles in health API JSON responses |
| src/health/health_variable.c | Uses type-aware accessor for dimension lookups in health expressions |
| src/exporting/prometheus/remote_write/remote_write.c | Exports float dimension values correctly |
| src/exporting/prometheus/prometheus.c | Handles float dimensions in as-collected metric output |
| src/exporting/opentsdb/opentsdb.c | Formats float dimension values using double format specifiers |
| src/exporting/json/json.c | Outputs float dimension values as doubles in JSON export |
| src/exporting/graphite/graphite.c | Formats float dimension values using double format specifiers |
| src/database/rrdset-collection.c | Implements float-aware collection algorithms with separate int/float overflow detection; uses type-aware accessors throughout |
| src/database/rrddim.h | Defines union structure for int/float storage, adds type-checking macros and accessor functions |
| src/database/rrddim.c | Implements rrddim_set_by_pointer_double() and updates existing functions to handle float dimensions |
| src/database/engine/dbengine-unittest.c | Updates test code to use new accessor functions |
| src/database/engine/dbengine-stresstest.c | Updates stress test to use new accessor functions |
| src/collectors/proc.plugin/proc_diskstats.c | Updates to use rrddim_last_collected_raw_int() accessor |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
|
@ilyam8 have you tested this? We should merge it, no? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| rrddim_set_collected_int(rd, 0); | ||
| rrddim_set_collected_float(rd, 0.0); |
Copilot
AI
Jan 19, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both int and float collection values are being reset unconditionally. This could be inefficient since only one lane is actually in use per dimension. Consider resetting only the active lane based on the dimension type to avoid redundant operations.
| rrddim_set_collected_int(rd, 0); | |
| rrddim_set_collected_float(rd, 0.0); | |
| if (rrddim_is_float(rd)) | |
| rrddim_set_collected_float(rd, 0.0); | |
| else | |
| rrddim_set_collected_int(rd, 0); |
Plugins.d protocol support DIMENSION type=float and SET accepts floats
Undo test: make go.d/ping using floats before merging.
Summary by cubic
Adds float support end-to-end for dimensions. Plugins can mark DIMENSION type=float and send float values in SET, and the pipeline (calculation, streaming, exporting, health) now carries doubles with backward-compatible fallbacks.
New Features
Migration
Written for commit 6abba18. Summary will update on new commits.