diff --git a/OTEL_EXEMPLARS.md b/OTEL_EXEMPLARS.md
index ea7377251..1034d52be 100644
--- a/OTEL_EXEMPLARS.md
+++ b/OTEL_EXEMPLARS.md
@@ -9,8 +9,8 @@ If you want to see this in action, you can run the example from the `ExemplarsCl
```
./mvnw package
cd integration_tests/it_exemplars_otel_agent/target/
-curl -LO https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.2.0/opentelemetry-javaagent-all.jar
-java -Dotel.traces.exporter=logging -Dotel.metrics.exporter=none -javaagent:./opentelemetry-javaagent-all.jar -jar ./example-spring-boot-app.jar
+curl -LO https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.10.1/opentelemetry-javaagent.jar
+java -Dotel.traces.exporter=logging -Dotel.metrics.exporter=none -javaagent:./opentelemetry-javaagent.jar -jar ./example-spring-boot-app.jar
```
Now you have a Spring REST service running on [http://localhost:8080/hello](http://localhost:8080/hello) that is instrumented with the OpenTelemetry Java agent.
diff --git a/README.md b/README.md
index f6fe7a172..b6dd47033 100644
--- a/README.md
+++ b/README.md
@@ -46,25 +46,25 @@ version can be found on in the maven repository for
io.prometheussimpleclient
- 0.12.0
+ 0.15.0io.prometheussimpleclient_hotspot
- 0.12.0
+ 0.15.0io.prometheussimpleclient_httpserver
- 0.12.0
+ 0.15.0io.prometheussimpleclient_pushgateway
- 0.12.0
+ 0.15.0
```
@@ -130,57 +130,86 @@ when using this approach ensure the value you are reporting accounts for concurr
### Summary
-Summaries track the size and number of events.
+Summaries and Histograms can both be used to monitor distributions, like latencies or request sizes.
+
+An overview of when to use Summaries and when to use Histograms can be found on [https://prometheus.io/docs/practices/histograms](https://prometheus.io/docs/practices/histograms).
+
+The following example shows how to measure latencies and request sizes:
```java
class YourClass {
- static final Summary receivedBytes = Summary.build()
- .name("requests_size_bytes").help("Request size in bytes.").register();
- static final Summary requestLatency = Summary.build()
- .name("requests_latency_seconds").help("Request latency in seconds.").register();
- void processRequest(Request req) {
+ private static final Summary requestLatency = Summary.build()
+ .name("requests_latency_seconds")
+ .help("request latency in seconds")
+ .register();
+
+ private static final Summary receivedBytes = Summary.build()
+ .name("requests_size_bytes")
+ .help("request size in bytes")
+ .register();
+
+ public void processRequest(Request req) {
Summary.Timer requestTimer = requestLatency.startTimer();
try {
// Your code here.
} finally {
- receivedBytes.observe(req.size());
requestTimer.observeDuration();
+ receivedBytes.observe(req.size());
}
}
}
```
-There are utilities for timing code and support for [quantiles](https://prometheus.io/docs/practices/histograms/#quantiles).
-Essentially quantiles aren't aggregatable and add some client overhead for the calculation.
+The `Summary` class provides different utility methods for observing values, like `observe(double)`, `startTimer(); timer.observeDuration()`, `time(Callable)`, etc.
+
+By default, `Summary` metrics provide the `count` and the `sum`. For example, if you measure latencies of a REST service, the `count` will tell you how often the REST service was called, and the `sum` will tell you the total aggregated response time. You can calculate the average response time using a Prometheus query dividing `sum / count`.
+
+In addition to `count` and `sum`, you can configure a Summary to provide quantiles:
```java
-class YourClass {
- static final Summary requestLatency = Summary.build()
- .quantile(0.5, 0.05) // Add 50th percentile (= median) with 5% tolerated error
- .quantile(0.9, 0.01) // Add 90th percentile with 1% tolerated error
- .name("requests_latency_seconds").help("Request latency in seconds.").register();
+Summary requestLatency = Summary.build()
+ .name("requests_latency_seconds")
+ .help("Request latency in seconds.")
+ .quantile(0.5, 0.01) // 0.5 quantile (median) with 0.01 allowed error
+ .quantile(0.95, 0.005) // 0.95 quantile with 0.005 allowed error
+ // ...
+ .register();
+```
- void processRequest(Request req) {
- requestLatency.time(new Runnable() {
- public abstract void run() {
- // Your code here.
- }
- });
+As an example, a `0.95` quantile of `120ms` tells you that `95%` of the calls were faster than `120ms`, and `5%` of the calls were slower than `120ms`.
+Tracking exact quantiles require a large amount of memory, because all observations need to be stored in a sorted list. Therefore, we allow an error to significantly reduce memory usage.
- // Or the Java 8 lambda equivalent
- requestLatency.time(() -> {
- // Your code here.
- });
- }
-}
+In the example, the allowed error of `0.005` means that you will not get the exact `0.95` quantile, but anything between the `0.945` quantile and the `0.955` quantile.
+
+Experiments show that the `Summary` typically needs to keep less than 100 samples to provide that precision, even if you have hundreds of millions of observations.
+
+There are a few special cases:
+
+* You can set an allowed error of `0`, but then the `Summary` will keep all observations in memory.
+* You can track the minimum value with `.quantile(0, 0)`. This special case will not use additional memory even though the allowed error is `0`.
+* You can track the maximum value with `.quantile(1, 0)`. This special case will not use additional memory even though the allowed error is `0`.
+
+Typically, you don't want to have a `Summary` representing the entire runtime of the application, but you want to look at a reasonable time interval. `Summary` metrics implement a configurable sliding time window:
+
+```java
+Summary requestLatency = Summary.build()
+ .name("requests_latency_seconds")
+ .help("Request latency in seconds.")
+ .maxAgeSeconds(10 * 60)
+ .ageBuckets(5)
+ // ...
+ .register();
```
+The default is a time window of 10 minutes and 5 age buckets, i.e. the time window is 10 minutes wide, and * we slide it forward every 2 minutes.
+
### Histogram
-Histograms track the size and number of events in buckets.
-This allows for aggregatable calculation of quantiles.
+Like Summaries, Histograms can be used to monitor latencies (or other things like request sizes).
+
+An overview of when to use Summaries and when to use Histograms can be found on [https://prometheus.io/docs/practices/histograms](https://prometheus.io/docs/practices/histograms).
```java
class YourClass {
diff --git a/benchmarks/pom.xml b/benchmarks/pom.xml
index 76a61bae3..e008615ae 100644
--- a/benchmarks/pom.xml
+++ b/benchmarks/pom.xml
@@ -5,7 +5,7 @@
io.prometheusparent
- 0.14.1
+ 0.15.1-SNAPSHOTbenchmarks
@@ -27,17 +27,17 @@
org.openjdk.jmhjmh-core
- 1.3.2
+ 1.34org.openjdk.jmhjmh-generator-annprocess
- 1.3.2
+ 1.34javax.annotationjavax.annotation-api
- 1.3.1
+ 1.3.2io.prometheus
diff --git a/benchmarks/src/main/java/io/prometheus/client/CKMSQuantileBenchmark.java b/benchmarks/src/main/java/io/prometheus/client/CKMSQuantileBenchmark.java
new file mode 100644
index 000000000..530810481
--- /dev/null
+++ b/benchmarks/src/main/java/io/prometheus/client/CKMSQuantileBenchmark.java
@@ -0,0 +1,138 @@
+package io.prometheus.client;
+
+import io.prometheus.client.CKMSQuantiles.Quantile;
+import org.openjdk.jmh.annotations.*;
+import org.openjdk.jmh.infra.Blackhole;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.RunnerException;
+import org.openjdk.jmh.runner.options.Options;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Random;
+import java.util.concurrent.TimeUnit;
+
+public class CKMSQuantileBenchmark {
+
+ @State(Scope.Benchmark)
+ public static class EmptyBenchmarkState {
+ @Param({"10000", "100000", "1000000"})
+ public int value;
+
+ List quantiles;
+ Random rand = new Random(0);
+
+ List shuffle;
+
+ Quantile mean = new Quantile(0.50, 0.050);
+ Quantile q90 = new Quantile(0.90, 0.010);
+ Quantile q95 = new Quantile(0.95, 0.005);
+ Quantile q99 = new Quantile(0.99, 0.001);
+
+ @Setup(Level.Trial)
+ public void setup() {
+ quantiles = new ArrayList();
+ quantiles.add(mean);
+ quantiles.add(q90);
+ quantiles.add(q95);
+ quantiles.add(q99);
+
+ shuffle = new ArrayList(value);
+ for (int i = 0; i < value; i++) {
+ shuffle.add((double) i);
+ }
+ Collections.shuffle(shuffle, rand);
+ }
+ }
+
+ @Benchmark
+ @BenchmarkMode({Mode.AverageTime})
+ @OutputTimeUnit(TimeUnit.MILLISECONDS)
+ public void ckmsQuantileInsertBenchmark(EmptyBenchmarkState state) {
+ CKMSQuantiles q = new CKMSQuantiles(state.quantiles.toArray(new Quantile[]{}));
+ for (Double l : state.shuffle) {
+ q.insert(l);
+ }
+ }
+
+ /** prefilled benchmark, means that we already have a filled and compressed samples available */
+ @State(Scope.Benchmark)
+ public static class PrefilledBenchmarkState {
+ @Param({"10000", "100000", "1000000"})
+ public int value;
+
+
+ CKMSQuantiles ckmsQuantiles;
+
+ List quantiles;
+ Random rand = new Random(0);
+
+ Quantile mean = new Quantile(0.50, 0.050);
+ Quantile q90 = new Quantile(0.90, 0.010);
+ Quantile q95 = new Quantile(0.95, 0.005);
+ Quantile q99 = new Quantile(0.99, 0.001);
+ List shuffle;
+
+ int rank = (int) (value * q95.quantile);
+
+
+ @Setup(Level.Trial)
+ public void setup() {
+ quantiles = new ArrayList();
+ quantiles.add(mean);
+ quantiles.add(q90);
+ quantiles.add(q95);
+ quantiles.add(q99);
+
+ shuffle = new ArrayList(value);
+ for (int i = 0; i < value; i++) {
+ shuffle.add((double) i);
+ }
+ Collections.shuffle(shuffle, rand);
+
+
+ ckmsQuantiles = new CKMSQuantiles(quantiles.toArray(new Quantile[]{}));
+ for (Double l : shuffle) {
+ ckmsQuantiles.insert(l);
+ }
+ // make sure we inserted all 'hanging' samples (count % 128)
+ ckmsQuantiles.get(0);
+ // compress everything so we have a similar samples size regardless of n.
+ ckmsQuantiles.compress();
+ System.out.println("Sample size is: " + ckmsQuantiles.samples.size());
+ }
+
+ }
+
+ @Benchmark
+ @BenchmarkMode({Mode.AverageTime})
+ @OutputTimeUnit(TimeUnit.NANOSECONDS)
+ public void ckmsQuantileGetBenchmark(Blackhole blackhole, PrefilledBenchmarkState state) {
+ blackhole.consume(state.ckmsQuantiles.get(state.q90.quantile));
+ }
+
+ /**
+ * benchmark for the f method.
+ */
+ @Benchmark
+ @BenchmarkMode({Mode.AverageTime})
+ @OutputTimeUnit(TimeUnit.NANOSECONDS)
+ public void ckmsQuantileF(Blackhole blackhole, PrefilledBenchmarkState state) {
+ blackhole.consume(state.ckmsQuantiles.f(state.rank));
+ }
+
+ public static void main(String[] args) throws RunnerException {
+
+ Options opt = new OptionsBuilder()
+ .include(CKMSQuantileBenchmark.class.getSimpleName())
+ .warmupIterations(5)
+ .measurementIterations(4)
+ .threads(1)
+ .forks(1)
+ .build();
+
+ new Runner(opt).run();
+ }
+}
diff --git a/integration_tests/it_common/pom.xml b/integration_tests/it_common/pom.xml
index bebfe330e..05bc4a886 100644
--- a/integration_tests/it_common/pom.xml
+++ b/integration_tests/it_common/pom.xml
@@ -5,7 +5,7 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_common
diff --git a/integration_tests/it_exemplars_otel_agent/pom.xml b/integration_tests/it_exemplars_otel_agent/pom.xml
index 69a4f6d3d..2ac1266d7 100644
--- a/integration_tests/it_exemplars_otel_agent/pom.xml
+++ b/integration_tests/it_exemplars_otel_agent/pom.xml
@@ -5,7 +5,7 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_exemplars_otel_agent
@@ -16,7 +16,7 @@
org.springframework.bootspring-boot-dependencies
- 2.4.4
+ 2.6.3pomimport
@@ -56,7 +56,7 @@
ch.qos.logbacklogback-classic
- 1.2.0
+ 1.2.10test
diff --git a/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java b/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java
index 094f04fb6..7736f3595 100644
--- a/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java
+++ b/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java
@@ -25,13 +25,13 @@
public class ExemplarsOpenTelemetryAgentIT {
private final String image = "openjdk:11-jre";
- private final String otelAgentVersion = "1.2.0";
+ private final String otelAgentVersion = "1.10.1";
private final Volume volume;
private final GenericContainer> javaContainer;
public ExemplarsOpenTelemetryAgentIT() throws IOException, URISyntaxException {
String appJar = "example-spring-boot-app.jar";
- String agentJar = "opentelemetry-javaagent-all.jar";
+ String agentJar = "opentelemetry-javaagent.jar";
String agentDownloadUrl = "https://github.com/open-telemetry/opentelemetry-java-instrumentation/" +
"releases/download/v" + otelAgentVersion + "/" + agentJar;
Downloader.downloadToTarget(agentDownloadUrl, agentJar);
diff --git a/integration_tests/it_exemplars_otel_sdk/pom.xml b/integration_tests/it_exemplars_otel_sdk/pom.xml
index adebc1d9d..aa9096d94 100644
--- a/integration_tests/it_exemplars_otel_sdk/pom.xml
+++ b/integration_tests/it_exemplars_otel_sdk/pom.xml
@@ -5,14 +5,14 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_exemplars_otel_sdkIntegration Tests - Exemplars with OpenTelemetry
- 1.2.0
+ 1.10.1
diff --git a/integration_tests/it_java_versions/pom.xml b/integration_tests/it_java_versions/pom.xml
index 43e7f5853..3d3c86f75 100644
--- a/integration_tests/it_java_versions/pom.xml
+++ b/integration_tests/it_java_versions/pom.xml
@@ -5,7 +5,7 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_java_versions
@@ -78,8 +78,6 @@
1.61.6
-
-
1.81.8
@@ -91,6 +89,36 @@
+
+
+
+
+
+ intellij
+
+ false
+
+
+
+
+ idea.maven.embedder.version
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-compiler-plugin
+
+ 1.8
+ 1.8
+
+
+
+
+
+
+
The Apache Software License, Version 2.0
diff --git a/integration_tests/it_log4j2/pom.xml b/integration_tests/it_log4j2/pom.xml
index 8b792d908..15f296931 100644
--- a/integration_tests/it_log4j2/pom.xml
+++ b/integration_tests/it_log4j2/pom.xml
@@ -5,14 +5,14 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_log4j2Integration Tests - log4j2
- 2.17.0
+ 2.17.1
@@ -97,8 +97,6 @@
1.61.6
-
-
1.81.8
@@ -110,6 +108,36 @@
+
+
+
+
+
+ intellij
+
+ false
+
+
+
+
+ idea.maven.embedder.version
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-compiler-plugin
+
+ 1.8
+ 1.8
+
+
+
+
+
+
+
The Apache Software License, Version 2.0
diff --git a/integration_tests/it_pushgateway/pom.xml b/integration_tests/it_pushgateway/pom.xml
index a64f0efff..f199d5601 100644
--- a/integration_tests/it_pushgateway/pom.xml
+++ b/integration_tests/it_pushgateway/pom.xml
@@ -5,7 +5,7 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_pushgateway
@@ -30,13 +30,13 @@
com.squareup.okhttp3okhttp
- 4.9.1
+ 4.9.3testch.qos.logbacklogback-classic
- 1.2.0
+ 1.2.10test
@@ -75,8 +75,6 @@
1.61.6
-
-
1.81.8
@@ -88,6 +86,36 @@
+
+
+
+
+
+ intellij
+
+ false
+
+
+
+
+ idea.maven.embedder.version
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-compiler-plugin
+
+ 1.8
+ 1.8
+
+
+
+
+
+
+
The Apache Software License, Version 2.0
diff --git a/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml b/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml
index bb9a40da1..072b567c9 100644
--- a/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml
+++ b/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml
@@ -5,7 +5,7 @@
io.prometheusintegration_tests
- 0.14.1
+ 0.15.1-SNAPSHOTit_servlet_jakarta_exporter_webxml
@@ -35,7 +35,7 @@
com.squareup.okhttp3okhttp
- 4.9.1
+ 4.9.3org.testcontainers
@@ -45,7 +45,7 @@
ch.qos.logbacklogback-classic
- 1.2.0
+ 1.2.10jakarta.servlet
diff --git a/integration_tests/pom.xml b/integration_tests/pom.xml
index 280040fb3..eeacb74b8 100644
--- a/integration_tests/pom.xml
+++ b/integration_tests/pom.xml
@@ -5,7 +5,7 @@
io.prometheusparent
- 0.14.1
+ 0.15.1-SNAPSHOTintegration_tests
@@ -56,7 +56,7 @@
org.testcontainerstestcontainers
- 1.15.2
+ 1.16.3test
@@ -68,7 +68,7 @@
ch.qos.logbacklogback-classic
- 1.2.0
+ 1.2.10test
diff --git a/pom.xml b/pom.xml
index 85d3c4ea3..1e4fb9d8b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -5,7 +5,7 @@
io.prometheusparent
- 0.14.1
+ 0.15.1-SNAPSHOTPrometheus Java Suitehttp://github.com/prometheus/client_java
@@ -25,7 +25,7 @@
scm:git:git@github.com:prometheus/client_java.gitscm:git:git@github.com:prometheus/client_java.gitgit@github.com:prometheus/client_java.git
- parent-0.14.1
+ HEAD
@@ -173,8 +173,8 @@
- maven-release-pluginorg.apache.maven.plugins
+ maven-release-plugintruefalse
@@ -183,8 +183,8 @@
- maven-deploy-pluginorg.apache.maven.plugins
+ maven-deploy-pluginorg.apache.felix
diff --git a/simpleclient/pom.xml b/simpleclient/pom.xml
index 3cc8b7845..c0067869a 100644
--- a/simpleclient/pom.xml
+++ b/simpleclient/pom.xml
@@ -5,7 +5,7 @@
io.prometheusparent
- 0.14.1
+ 0.15.1-SNAPSHOTsimpleclient
@@ -54,5 +54,11 @@
4.13.2test
+
+ org.apache.commons
+ commons-math3
+ 3.6.1
+ test
+
diff --git a/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java b/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java
index 1ffb65382..78126efc8 100644
--- a/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java
+++ b/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java
@@ -1,15 +1,9 @@
package io.prometheus.client;
-// Copied from https://raw.githubusercontent.com/Netflix/ocelli/master/ocelli-core/src/main/java/netflix/ocelli/stats/CKMSQuantiles.java
+// The original implementation was copied from
+// https://raw.githubusercontent.com/Netflix/ocelli/master/ocelli-core/src/main/java/netflix/ocelli/stats/CKMSQuantiles.java
// Revision d0357b8bf5c17a173ce94d6b26823775b3f999f6 from Jan 21, 2015.
-//
-// This is the original code except for the following modifications:
-//
-// - Changed the type of the observed values from int to double.
-// - Removed the Quantiles interface and corresponding @Override annotations.
-// - Changed the package name.
-// - Make get() return NaN when no sample was observed.
-// - Make class package private
+// However, it has been heavily refactored in the meantime.
/*
Copyright 2012 Andrew Wang (andrew@umbrant.com)
@@ -28,266 +22,275 @@
*/
import java.util.Arrays;
+import java.util.Iterator;
import java.util.LinkedList;
import java.util.ListIterator;
/**
- * Implementation of the Cormode, Korn, Muthukrishnan, and Srivastava algorithm
- * for streaming calculation of targeted high-percentile epsilon-approximate
- * quantiles.
- *
- * This is a generalization of the earlier work by Greenwald and Khanna (GK),
- * which essentially allows different error bounds on the targeted quantiles,
- * which allows for far more efficient calculation of high-percentiles.
- *
- *
- * See: Cormode, Korn, Muthukrishnan, and Srivastava
- * "Effective Computation of Biased Quantiles over Data Streams" in ICDE 2005
- *
- * Greenwald and Khanna,
- * "Space-efficient online computation of quantile summaries" in SIGMOD 2001
- *
+ * Algorithm solving the "Targeted Quantile Problem" as described in
+ * "Effective Computation of Biased Quantiles over Data Streams"
+ * by Cormode, Korn, Muthukrishnan, and Srivastava.
+ *
*/
-class CKMSQuantiles {
- /**
- * Total number of items in stream.
- */
- private int count = 0;
+final class CKMSQuantiles {
+
+ final Quantile[] quantiles;
/**
- * Used for tracking incremental compression.
+ * Total number of observations (not including those that are still in the buffer).
*/
- private int compressIdx = 0;
+ int n = 0;
/**
- * Current list of sampled items, maintained in sorted order with error
- * bounds.
+ * List of sampled observations, ordered by Sample.value.
*/
- protected LinkedList sample;
+ final LinkedList samples = new LinkedList();
/**
- * Buffers incoming items to be inserted in batch.
+ * Compress is called every compressInterval inserts.
+ * Note that the buffer is flushed whenever get() is called, so we
+ * cannot just wait until the buffer is full before we call compress.
*/
- private double[] buffer = new double[500];
-
- private int bufferCount = 0;
+ private final int compressInterval = 128;
+ private int insertsSinceLastCompress = 0;
/**
- * Array of Quantiles that we care about, along with desired error.
+ * Note that the buffer size could as well be less than the compressInterval.
+ * However, the buffer size should not be greater than the compressInterval,
+ * because the compressInterval is not respected in flush(), so if you want
+ * to compress more often than calling flush() that won't work.
*/
- private final Quantile quantiles[];
+ private final double[] buffer = new double[compressInterval];
+ private int bufferPos = 0;
- public CKMSQuantiles(Quantile[] quantiles) {
+ public CKMSQuantiles(Quantile... quantiles) {
+ if (quantiles.length == 0) {
+ throw new IllegalArgumentException("quantiles cannot be empty");
+ }
this.quantiles = quantiles;
- this.sample = new LinkedList();
}
/**
- * Add a new value from the stream.
- *
- * @param value
+ * Add an observed value
*/
public void insert(double value) {
- buffer[bufferCount] = value;
- bufferCount++;
+ buffer[bufferPos++] = value;
+
+ if (bufferPos == buffer.length) {
+ flush();
+ }
- if (bufferCount == buffer.length) {
- insertBatch();
+ if (++insertsSinceLastCompress == compressInterval) {
compress();
+ insertsSinceLastCompress = 0;
}
}
+ private void flush() {
+ Arrays.sort(buffer, 0, bufferPos);
+ insertBatch(buffer, bufferPos);
+ bufferPos = 0;
+ }
+
/**
- * Get the estimated value at the specified quantile.
- *
- * @param q
- * Queried quantile, e.g. 0.50 or 0.99.
- * @return Estimated value at that quantile.
+ * Inserts the elements from index 0 to index toIndex from the sortedBuffer.
*/
- public double get(double q) {
- // clear the buffer
- insertBatch();
- compress();
-
- if (sample.size() == 0) {
- return Double.NaN;
+ void insertBatch(double[] sortedBuffer, int toIndex) {
+ if (toIndex == 0) {
+ return;
}
-
- int rankMin = 0;
- int desired = (int) (q * count);
-
- ListIterator it = sample.listIterator();
- Item prev, cur;
- cur = it.next();
- while (it.hasNext()) {
- prev = cur;
- cur = it.next();
-
- rankMin += prev.g;
-
- if (rankMin + cur.g + cur.delta > desired
- + (allowableError(desired) / 2)) {
- return prev.value;
+ ListIterator iterator = samples.listIterator();
+ int i = 0; // position in buffer
+ int r = 0; // sum of g's left of the current sample
+ while (iterator.hasNext() && i < toIndex) {
+ Sample item = iterator.next();
+ while (i < toIndex) {
+ if (sortedBuffer[i] > item.value) {
+ break;
+ }
+ insertBefore(iterator, sortedBuffer[i], r);
+ r++; // new item with g=1 was inserted before, so increment r
+ i++;
+ n++;
}
+ r += item.g;
}
+ while (i < toIndex) {
+ samples.add(new Sample(sortedBuffer[i], 0));
+ i++;
+ n++;
+ }
+ }
- // edge case of wanting max value
- return sample.getLast().value;
+ private void insertBefore(ListIterator iterator, double value, int r) {
+ if (!iterator.hasPrevious()) {
+ samples.addFirst(new Sample(value, 0));
+ } else {
+ iterator.previous();
+ iterator.add(new Sample(value, f(r) - 1));
+ iterator.next();
+ }
}
/**
- * Specifies the allowable error for this rank, depending on which quantiles
- * are being targeted.
- *
- * This is the f(r_i, n) function from the CKMS paper. It's basically how
- * wide the range of this rank can be.
- *
- * @param rank
- * the index in the list of samples
+ * Get the estimated value at the specified quantile.
*/
- private double allowableError(int rank) {
- // NOTE: according to CKMS, this should be count, not size, but this
- // leads
- // to error larger than the error bounds. Leaving it like this is
- // essentially a HACK, and blows up memory, but does "work".
- // int size = count;
- int size = sample.size();
- double minError = size + 1;
+ public double get(double q) {
+ flush();
- for (Quantile q : quantiles) {
- double error;
- if (rank <= q.quantile * size) {
- error = q.u * (size - rank);
- } else {
- error = q.v * rank;
- }
- if (error < minError) {
- minError = error;
- }
+ if (samples.size() == 0) {
+ return Double.NaN;
}
- return minError;
- }
-
- private boolean insertBatch() {
- if (bufferCount == 0) {
- return false;
+ if (q == 0.0) {
+ return samples.getFirst().value;
}
- Arrays.sort(buffer, 0, bufferCount);
-
- // Base case: no samples
- int start = 0;
- if (sample.size() == 0) {
- Item newItem = new Item(buffer[0], 1, 0);
- sample.add(newItem);
- start++;
- count++;
+ if (q == 1.0) {
+ return samples.getLast().value;
}
- ListIterator it = sample.listIterator();
- Item item = it.next();
-
- for (int i = start; i < bufferCount; i++) {
- double v = buffer[i];
- while (it.nextIndex() < sample.size() && item.value < v) {
- item = it.next();
+ int r = 0; // sum of g's left of the current sample
+ int desiredRank = (int) Math.ceil(q * n);
+ int upperBound = desiredRank + f(desiredRank) / 2;
+
+ ListIterator iterator = samples.listIterator();
+ while (iterator.hasNext()) {
+ Sample sample = iterator.next();
+ if (r + sample.g + sample.delta > upperBound) {
+ iterator.previous(); // roll back the item.next() above
+ if (iterator.hasPrevious()) {
+ Sample result = iterator.previous();
+ return result.value;
+ } else {
+ return sample.value;
+ }
}
+ r += sample.g;
+ }
+ return samples.getLast().value;
+ }
- // If we found that bigger item, back up so we insert ourselves
- // before it
- if (item.value > v) {
- it.previous();
+ /**
+ * Error function, as in definition 5 of the paper.
+ */
+ int f(int r) {
+ int minResult = Integer.MAX_VALUE;
+ for (Quantile q : quantiles) {
+ if (q.quantile == 0 || q.quantile == 1) {
+ continue;
}
-
- // We use different indexes for the edge comparisons, because of the
- // above
- // if statement that adjusts the iterator
- int delta;
- if (it.previousIndex() == 0 || it.nextIndex() == sample.size()) {
- delta = 0;
- }
- else {
- delta = ((int) Math.floor(allowableError(it.nextIndex()))) - 1;
+ int result;
+ // We had a numerical error here with the following example:
+ // quantile = 0.95, epsilon = 0.01, (n-r) = 30.
+ // The expected result of (2*0.01*30)/(1-0.95) is 12. The actual result is 11.99999999999999.
+ // To avoid running into these types of error we add 0.00000000001 before rounding down.
+ if (r >= q.quantile * n) {
+ result = (int) (q.v * r + 0.00000000001);
+ } else {
+ result = (int) (q.u * (n - r) + 0.00000000001);
+ }
+ if (result < minResult) {
+ minResult = result;
}
-
- Item newItem = new Item(v, 1, delta);
- it.add(newItem);
- count++;
- item = newItem;
}
-
- bufferCount = 0;
- return true;
+ return Math.max(minResult, 1);
}
/**
- * Try to remove extraneous items from the set of sampled items. This checks
- * if an item is unnecessary based on the desired error bounds, and merges
- * it with the adjacent item if it is.
+ * Merge pairs of consecutive samples if this doesn't violate the error function.
*/
- private void compress() {
- if (sample.size() < 2) {
+ void compress() {
+ if (samples.size() < 3) {
return;
}
-
- ListIterator it = sample.listIterator();
- int removed = 0;
-
- Item prev = null;
- Item next = it.next();
-
- while (it.hasNext()) {
- prev = next;
- next = it.next();
-
- if (prev.g + next.g + next.delta <= allowableError(it.previousIndex())) {
- next.g += prev.g;
- // Remove prev. it.remove() kills the last thing returned.
- it.previous();
- it.previous();
- it.remove();
- // it.next() is now equal to next, skip it back forward again
- it.next();
- removed++;
+ Iterator descendingIterator = samples.descendingIterator();
+ int r = n; // n is equal to the sum of the g's of all samples
+
+ Sample right;
+ Sample left = descendingIterator.next();
+ r -= left.g;
+
+ while (descendingIterator.hasNext()) {
+ right = left;
+ left = descendingIterator.next();
+ r = r - left.g;
+ if (left == samples.getFirst()) {
+ // The min sample must never be merged.
+ break;
+ }
+ if (left.g + right.g + right.delta < f(r)) {
+ right.g += left.g;
+ descendingIterator.remove();
+ left = right;
}
}
}
- private class Item {
- public final double value;
- public int g;
- public final int delta;
+ static class Sample {
+
+ /**
+ * Observed value.
+ */
+ final double value;
+
+ /**
+ * Difference between the lowest possible rank of this sample and its predecessor.
+ * This always starts with 1, but will be updated when compress() merges Samples.
+ */
+ int g = 1;
+
+ /**
+ * Difference between the greatest possible rank of this sample and the lowest possible rank of this sample.
+ */
+ final int delta;
- public Item(double value, int lower_delta, int delta) {
+ Sample(double value, int delta) {
this.value = value;
- this.g = lower_delta;
this.delta = delta;
}
@Override
public String toString() {
- return String.format("I{val=%.3f, g=%d, del=%d}", value, g, delta);
+ return String.format("Sample{val=%.3f, g=%d, delta=%d}", value, g, delta);
}
}
- public static class Quantile {
- public final double quantile;
- public final double error;
- public final double u;
- public final double v;
+ static class Quantile {
+
+ /**
+ * Quantile. Must be between 0 and 1.
+ */
+ final double quantile;
+
+ /**
+ * Allowed error. Must be between 0 and 1.
+ */
+ final double epsilon;
+
+ /**
+ * Helper used in the error function f(), see definition 5 in the paper.
+ */
+ final double u;
+
+ /**
+ * Helper used in the error function f(), see definition 5 in the paper.
+ */
+ final double v;
+
+ Quantile(double quantile, double epsilon) {
+ if (quantile < 0.0 || quantile > 1.0) throw new IllegalArgumentException("Quantile must be between 0 and 1");
+ if (epsilon < 0.0 || epsilon > 1.0) throw new IllegalArgumentException("Epsilon must be between 0 and 1");
- public Quantile(double quantile, double error) {
this.quantile = quantile;
- this.error = error;
- u = 2.0 * error / (1.0 - quantile);
- v = 2.0 * error / quantile;
+ this.epsilon = epsilon;
+ u = 2.0 * epsilon / (1.0 - quantile); // if quantile == 1 this will be Double.NaN
+ v = 2.0 * epsilon / quantile; // if quantile == 0 this will be Double.NaN
}
@Override
public String toString() {
- return String.format("Q{q=%.3f, eps=%.3f}", quantile, error);
+ return String.format("Quantile{q=%.3f, epsilon=%.3f}", quantile, epsilon);
}
}
-
}
diff --git a/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java b/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java
index ec321ba0a..5c5bf7c37 100644
--- a/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java
+++ b/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java
@@ -11,7 +11,7 @@
*
* This class handles common initialization and label logic for the standard metrics.
* You should never subclass this class.
- *
+ *
*
Initialization
* After calling build() on a subclass, {@link Builder#name(String) name},
* {@link SimpleCollector.Builder#help(String) help},
diff --git a/simpleclient/src/main/java/io/prometheus/client/Summary.java b/simpleclient/src/main/java/io/prometheus/client/Summary.java
index 34b123ba0..8836cbe7d 100644
--- a/simpleclient/src/main/java/io/prometheus/client/Summary.java
+++ b/simpleclient/src/main/java/io/prometheus/client/Summary.java
@@ -13,70 +13,91 @@
import java.util.concurrent.TimeUnit;
/**
- * Summary metric, to track the size of events.
+ * {@link Summary} metrics and {@link Histogram} metrics can both be used to monitor distributions like latencies or request sizes.
*
- * This would allow you to track request rate, average latency and average request size.
*
+ * The {@link Summary} class provides different utility methods for observing values, like {@link #observe(double)},
+ * {@link #startTimer()} and {@link Timer#observeDuration()}, {@link #time(Callable)}, etc.
*
- * How to add custom quantiles:
+ * By default, {@link Summary} metrics provide the {@code count} and the {@code sum}. For example, if you measure
+ * latencies of a REST service, the {@code count} will tell you how often the REST service was called,
+ * and the {@code sum} will tell you the total aggregated response time.
+ * You can calculate the average response time using a Prometheus query dividing {@code sum / count}.
+ *
+ * In addition to {@code count} and {@code sum}, you can configure a Summary to provide quantiles:
+ *
*
*
- * The quantiles are calculated over a sliding window of time. There are two options to configure this time window:
+ * As an example, a 0.95 quantile of 120ms tells you that 95% of the calls were faster than 120ms, and 5% of the calls were slower than 120ms.
+ *
+ * Tracking exact quantiles require a large amount of memory, because all observations need to be stored in a sorted list. Therefore, we allow an error to significantly reduce memory usage.
+ *
+ * In the example, the allowed error of 0.005 means that you will not get the exact 0.95 quantile, but anything between the 0.945 quantile and the 0.955 quantile.
+ *
+ * Experiments show that the {@link Summary} typically needs to keep less than 100 samples to provide that precision, even if you have hundreds of millions of observations.
+ *
+ * There are a few special cases:
+ *
*
- *
maxAgeSeconds(long): Set the duration of the time window is, i.e. how long observations are kept before they are discarded.
- * Default is 10 minutes.
- *
ageBuckets(int): Set the number of buckets used to implement the sliding time window. If your time window is 10 minutes, and you have ageBuckets=5,
- * buckets will be switched every 2 minutes. The value is a trade-off between resources (memory and cpu for maintaining the bucket)
- * and how smooth the time window is moved. Default value is 5.
+ *
You can set an allowed error of 0, but then the {@link Summary} will keep all observations in memory.
+ *
You can track the minimum value with {@code .quantile(0.0, 0.0)}.
+ * This special case will not use additional memory even though the allowed error is 0.
+ *
You can track the maximum value with {@code .quantile(1.0, 0.0)}.
+ * This special case will not use additional memory even though the allowed error is 0.
*
*
- * See https://prometheus.io/docs/practices/histograms/ for more info on quantiles.
+ * Typically, you don't want to have a {@link Summary} representing the entire runtime of the application,
+ * but you want to look at a reasonable time interval. {@link Summary} metrics implement a configurable sliding
+ * time window:
+ *
+ *