diff --git a/OTEL_EXEMPLARS.md b/OTEL_EXEMPLARS.md index ea7377251..1034d52be 100644 --- a/OTEL_EXEMPLARS.md +++ b/OTEL_EXEMPLARS.md @@ -9,8 +9,8 @@ If you want to see this in action, you can run the example from the `ExemplarsCl ``` ./mvnw package cd integration_tests/it_exemplars_otel_agent/target/ -curl -LO https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.2.0/opentelemetry-javaagent-all.jar -java -Dotel.traces.exporter=logging -Dotel.metrics.exporter=none -javaagent:./opentelemetry-javaagent-all.jar -jar ./example-spring-boot-app.jar +curl -LO https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.10.1/opentelemetry-javaagent.jar +java -Dotel.traces.exporter=logging -Dotel.metrics.exporter=none -javaagent:./opentelemetry-javaagent.jar -jar ./example-spring-boot-app.jar ``` Now you have a Spring REST service running on [http://localhost:8080/hello](http://localhost:8080/hello) that is instrumented with the OpenTelemetry Java agent. diff --git a/README.md b/README.md index f6fe7a172..b6dd47033 100644 --- a/README.md +++ b/README.md @@ -46,25 +46,25 @@ version can be found on in the maven repository for io.prometheus simpleclient - 0.12.0 + 0.15.0 io.prometheus simpleclient_hotspot - 0.12.0 + 0.15.0 io.prometheus simpleclient_httpserver - 0.12.0 + 0.15.0 io.prometheus simpleclient_pushgateway - 0.12.0 + 0.15.0 ``` @@ -130,57 +130,86 @@ when using this approach ensure the value you are reporting accounts for concurr ### Summary -Summaries track the size and number of events. +Summaries and Histograms can both be used to monitor distributions, like latencies or request sizes. + +An overview of when to use Summaries and when to use Histograms can be found on [https://prometheus.io/docs/practices/histograms](https://prometheus.io/docs/practices/histograms). + +The following example shows how to measure latencies and request sizes: ```java class YourClass { - static final Summary receivedBytes = Summary.build() - .name("requests_size_bytes").help("Request size in bytes.").register(); - static final Summary requestLatency = Summary.build() - .name("requests_latency_seconds").help("Request latency in seconds.").register(); - void processRequest(Request req) { + private static final Summary requestLatency = Summary.build() + .name("requests_latency_seconds") + .help("request latency in seconds") + .register(); + + private static final Summary receivedBytes = Summary.build() + .name("requests_size_bytes") + .help("request size in bytes") + .register(); + + public void processRequest(Request req) { Summary.Timer requestTimer = requestLatency.startTimer(); try { // Your code here. } finally { - receivedBytes.observe(req.size()); requestTimer.observeDuration(); + receivedBytes.observe(req.size()); } } } ``` -There are utilities for timing code and support for [quantiles](https://prometheus.io/docs/practices/histograms/#quantiles). -Essentially quantiles aren't aggregatable and add some client overhead for the calculation. +The `Summary` class provides different utility methods for observing values, like `observe(double)`, `startTimer(); timer.observeDuration()`, `time(Callable)`, etc. + +By default, `Summary` metrics provide the `count` and the `sum`. For example, if you measure latencies of a REST service, the `count` will tell you how often the REST service was called, and the `sum` will tell you the total aggregated response time. You can calculate the average response time using a Prometheus query dividing `sum / count`. + +In addition to `count` and `sum`, you can configure a Summary to provide quantiles: ```java -class YourClass { - static final Summary requestLatency = Summary.build() - .quantile(0.5, 0.05) // Add 50th percentile (= median) with 5% tolerated error - .quantile(0.9, 0.01) // Add 90th percentile with 1% tolerated error - .name("requests_latency_seconds").help("Request latency in seconds.").register(); +Summary requestLatency = Summary.build() + .name("requests_latency_seconds") + .help("Request latency in seconds.") + .quantile(0.5, 0.01) // 0.5 quantile (median) with 0.01 allowed error + .quantile(0.95, 0.005) // 0.95 quantile with 0.005 allowed error + // ... + .register(); +``` - void processRequest(Request req) { - requestLatency.time(new Runnable() { - public abstract void run() { - // Your code here. - } - }); +As an example, a `0.95` quantile of `120ms` tells you that `95%` of the calls were faster than `120ms`, and `5%` of the calls were slower than `120ms`. +Tracking exact quantiles require a large amount of memory, because all observations need to be stored in a sorted list. Therefore, we allow an error to significantly reduce memory usage. - // Or the Java 8 lambda equivalent - requestLatency.time(() -> { - // Your code here. - }); - } -} +In the example, the allowed error of `0.005` means that you will not get the exact `0.95` quantile, but anything between the `0.945` quantile and the `0.955` quantile. + +Experiments show that the `Summary` typically needs to keep less than 100 samples to provide that precision, even if you have hundreds of millions of observations. + +There are a few special cases: + +* You can set an allowed error of `0`, but then the `Summary` will keep all observations in memory. +* You can track the minimum value with `.quantile(0, 0)`. This special case will not use additional memory even though the allowed error is `0`. +* You can track the maximum value with `.quantile(1, 0)`. This special case will not use additional memory even though the allowed error is `0`. + +Typically, you don't want to have a `Summary` representing the entire runtime of the application, but you want to look at a reasonable time interval. `Summary` metrics implement a configurable sliding time window: + +```java +Summary requestLatency = Summary.build() + .name("requests_latency_seconds") + .help("Request latency in seconds.") + .maxAgeSeconds(10 * 60) + .ageBuckets(5) + // ... + .register(); ``` +The default is a time window of 10 minutes and 5 age buckets, i.e. the time window is 10 minutes wide, and * we slide it forward every 2 minutes. + ### Histogram -Histograms track the size and number of events in buckets. -This allows for aggregatable calculation of quantiles. +Like Summaries, Histograms can be used to monitor latencies (or other things like request sizes). + +An overview of when to use Summaries and when to use Histograms can be found on [https://prometheus.io/docs/practices/histograms](https://prometheus.io/docs/practices/histograms). ```java class YourClass { diff --git a/benchmarks/pom.xml b/benchmarks/pom.xml index 76a61bae3..e008615ae 100644 --- a/benchmarks/pom.xml +++ b/benchmarks/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT benchmarks @@ -27,17 +27,17 @@ org.openjdk.jmh jmh-core - 1.3.2 + 1.34 org.openjdk.jmh jmh-generator-annprocess - 1.3.2 + 1.34 javax.annotation javax.annotation-api - 1.3.1 + 1.3.2 io.prometheus diff --git a/benchmarks/src/main/java/io/prometheus/client/CKMSQuantileBenchmark.java b/benchmarks/src/main/java/io/prometheus/client/CKMSQuantileBenchmark.java new file mode 100644 index 000000000..530810481 --- /dev/null +++ b/benchmarks/src/main/java/io/prometheus/client/CKMSQuantileBenchmark.java @@ -0,0 +1,138 @@ +package io.prometheus.client; + +import io.prometheus.client.CKMSQuantiles.Quantile; +import org.openjdk.jmh.annotations.*; +import org.openjdk.jmh.infra.Blackhole; +import org.openjdk.jmh.runner.Runner; +import org.openjdk.jmh.runner.RunnerException; +import org.openjdk.jmh.runner.options.Options; +import org.openjdk.jmh.runner.options.OptionsBuilder; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Random; +import java.util.concurrent.TimeUnit; + +public class CKMSQuantileBenchmark { + + @State(Scope.Benchmark) + public static class EmptyBenchmarkState { + @Param({"10000", "100000", "1000000"}) + public int value; + + List quantiles; + Random rand = new Random(0); + + List shuffle; + + Quantile mean = new Quantile(0.50, 0.050); + Quantile q90 = new Quantile(0.90, 0.010); + Quantile q95 = new Quantile(0.95, 0.005); + Quantile q99 = new Quantile(0.99, 0.001); + + @Setup(Level.Trial) + public void setup() { + quantiles = new ArrayList(); + quantiles.add(mean); + quantiles.add(q90); + quantiles.add(q95); + quantiles.add(q99); + + shuffle = new ArrayList(value); + for (int i = 0; i < value; i++) { + shuffle.add((double) i); + } + Collections.shuffle(shuffle, rand); + } + } + + @Benchmark + @BenchmarkMode({Mode.AverageTime}) + @OutputTimeUnit(TimeUnit.MILLISECONDS) + public void ckmsQuantileInsertBenchmark(EmptyBenchmarkState state) { + CKMSQuantiles q = new CKMSQuantiles(state.quantiles.toArray(new Quantile[]{})); + for (Double l : state.shuffle) { + q.insert(l); + } + } + + /** prefilled benchmark, means that we already have a filled and compressed samples available */ + @State(Scope.Benchmark) + public static class PrefilledBenchmarkState { + @Param({"10000", "100000", "1000000"}) + public int value; + + + CKMSQuantiles ckmsQuantiles; + + List quantiles; + Random rand = new Random(0); + + Quantile mean = new Quantile(0.50, 0.050); + Quantile q90 = new Quantile(0.90, 0.010); + Quantile q95 = new Quantile(0.95, 0.005); + Quantile q99 = new Quantile(0.99, 0.001); + List shuffle; + + int rank = (int) (value * q95.quantile); + + + @Setup(Level.Trial) + public void setup() { + quantiles = new ArrayList(); + quantiles.add(mean); + quantiles.add(q90); + quantiles.add(q95); + quantiles.add(q99); + + shuffle = new ArrayList(value); + for (int i = 0; i < value; i++) { + shuffle.add((double) i); + } + Collections.shuffle(shuffle, rand); + + + ckmsQuantiles = new CKMSQuantiles(quantiles.toArray(new Quantile[]{})); + for (Double l : shuffle) { + ckmsQuantiles.insert(l); + } + // make sure we inserted all 'hanging' samples (count % 128) + ckmsQuantiles.get(0); + // compress everything so we have a similar samples size regardless of n. + ckmsQuantiles.compress(); + System.out.println("Sample size is: " + ckmsQuantiles.samples.size()); + } + + } + + @Benchmark + @BenchmarkMode({Mode.AverageTime}) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + public void ckmsQuantileGetBenchmark(Blackhole blackhole, PrefilledBenchmarkState state) { + blackhole.consume(state.ckmsQuantiles.get(state.q90.quantile)); + } + + /** + * benchmark for the f method. + */ + @Benchmark + @BenchmarkMode({Mode.AverageTime}) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + public void ckmsQuantileF(Blackhole blackhole, PrefilledBenchmarkState state) { + blackhole.consume(state.ckmsQuantiles.f(state.rank)); + } + + public static void main(String[] args) throws RunnerException { + + Options opt = new OptionsBuilder() + .include(CKMSQuantileBenchmark.class.getSimpleName()) + .warmupIterations(5) + .measurementIterations(4) + .threads(1) + .forks(1) + .build(); + + new Runner(opt).run(); + } +} diff --git a/integration_tests/it_common/pom.xml b/integration_tests/it_common/pom.xml index bebfe330e..05bc4a886 100644 --- a/integration_tests/it_common/pom.xml +++ b/integration_tests/it_common/pom.xml @@ -5,7 +5,7 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_common diff --git a/integration_tests/it_exemplars_otel_agent/pom.xml b/integration_tests/it_exemplars_otel_agent/pom.xml index 69a4f6d3d..2ac1266d7 100644 --- a/integration_tests/it_exemplars_otel_agent/pom.xml +++ b/integration_tests/it_exemplars_otel_agent/pom.xml @@ -5,7 +5,7 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_exemplars_otel_agent @@ -16,7 +16,7 @@ org.springframework.boot spring-boot-dependencies - 2.4.4 + 2.6.3 pom import @@ -56,7 +56,7 @@ ch.qos.logback logback-classic - 1.2.0 + 1.2.10 test diff --git a/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java b/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java index 094f04fb6..7736f3595 100644 --- a/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java +++ b/integration_tests/it_exemplars_otel_agent/src/test/java/io/prometheus/client/it/exemplars_otel_agent/ExemplarsOpenTelemetryAgentIT.java @@ -25,13 +25,13 @@ public class ExemplarsOpenTelemetryAgentIT { private final String image = "openjdk:11-jre"; - private final String otelAgentVersion = "1.2.0"; + private final String otelAgentVersion = "1.10.1"; private final Volume volume; private final GenericContainer javaContainer; public ExemplarsOpenTelemetryAgentIT() throws IOException, URISyntaxException { String appJar = "example-spring-boot-app.jar"; - String agentJar = "opentelemetry-javaagent-all.jar"; + String agentJar = "opentelemetry-javaagent.jar"; String agentDownloadUrl = "https://github.com/open-telemetry/opentelemetry-java-instrumentation/" + "releases/download/v" + otelAgentVersion + "/" + agentJar; Downloader.downloadToTarget(agentDownloadUrl, agentJar); diff --git a/integration_tests/it_exemplars_otel_sdk/pom.xml b/integration_tests/it_exemplars_otel_sdk/pom.xml index adebc1d9d..aa9096d94 100644 --- a/integration_tests/it_exemplars_otel_sdk/pom.xml +++ b/integration_tests/it_exemplars_otel_sdk/pom.xml @@ -5,14 +5,14 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_exemplars_otel_sdk Integration Tests - Exemplars with OpenTelemetry - 1.2.0 + 1.10.1 diff --git a/integration_tests/it_java_versions/pom.xml b/integration_tests/it_java_versions/pom.xml index 43e7f5853..3d3c86f75 100644 --- a/integration_tests/it_java_versions/pom.xml +++ b/integration_tests/it_java_versions/pom.xml @@ -5,7 +5,7 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_java_versions @@ -78,8 +78,6 @@ 1.6 1.6 - - 1.8 1.8 @@ -91,6 +89,36 @@ + + + + + + intellij + + false + + + + + idea.maven.embedder.version + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 1.8 + 1.8 + + + + + + + The Apache Software License, Version 2.0 diff --git a/integration_tests/it_log4j2/pom.xml b/integration_tests/it_log4j2/pom.xml index 8b792d908..15f296931 100644 --- a/integration_tests/it_log4j2/pom.xml +++ b/integration_tests/it_log4j2/pom.xml @@ -5,14 +5,14 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_log4j2 Integration Tests - log4j2 - 2.17.0 + 2.17.1 @@ -97,8 +97,6 @@ 1.6 1.6 - - 1.8 1.8 @@ -110,6 +108,36 @@ + + + + + + intellij + + false + + + + + idea.maven.embedder.version + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 1.8 + 1.8 + + + + + + + The Apache Software License, Version 2.0 diff --git a/integration_tests/it_pushgateway/pom.xml b/integration_tests/it_pushgateway/pom.xml index a64f0efff..f199d5601 100644 --- a/integration_tests/it_pushgateway/pom.xml +++ b/integration_tests/it_pushgateway/pom.xml @@ -5,7 +5,7 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_pushgateway @@ -30,13 +30,13 @@ com.squareup.okhttp3 okhttp - 4.9.1 + 4.9.3 test ch.qos.logback logback-classic - 1.2.0 + 1.2.10 test @@ -75,8 +75,6 @@ 1.6 1.6 - - 1.8 1.8 @@ -88,6 +86,36 @@ + + + + + + intellij + + false + + + + + idea.maven.embedder.version + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 1.8 + 1.8 + + + + + + + The Apache Software License, Version 2.0 diff --git a/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml b/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml index bb9a40da1..072b567c9 100644 --- a/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml +++ b/integration_tests/it_servlet_jakarta_exporter_webxml/pom.xml @@ -5,7 +5,7 @@ io.prometheus integration_tests - 0.14.1 + 0.15.1-SNAPSHOT it_servlet_jakarta_exporter_webxml @@ -35,7 +35,7 @@ com.squareup.okhttp3 okhttp - 4.9.1 + 4.9.3 org.testcontainers @@ -45,7 +45,7 @@ ch.qos.logback logback-classic - 1.2.0 + 1.2.10 jakarta.servlet diff --git a/integration_tests/pom.xml b/integration_tests/pom.xml index 280040fb3..eeacb74b8 100644 --- a/integration_tests/pom.xml +++ b/integration_tests/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT integration_tests @@ -56,7 +56,7 @@ org.testcontainers testcontainers - 1.15.2 + 1.16.3 test @@ -68,7 +68,7 @@ ch.qos.logback logback-classic - 1.2.0 + 1.2.10 test diff --git a/pom.xml b/pom.xml index 85d3c4ea3..1e4fb9d8b 100644 --- a/pom.xml +++ b/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT Prometheus Java Suite http://github.com/prometheus/client_java @@ -25,7 +25,7 @@ scm:git:git@github.com:prometheus/client_java.git scm:git:git@github.com:prometheus/client_java.git git@github.com:prometheus/client_java.git - parent-0.14.1 + HEAD @@ -173,8 +173,8 @@ - maven-release-plugin org.apache.maven.plugins + maven-release-plugin true false @@ -183,8 +183,8 @@ - maven-deploy-plugin org.apache.maven.plugins + maven-deploy-plugin org.apache.felix diff --git a/simpleclient/pom.xml b/simpleclient/pom.xml index 3cc8b7845..c0067869a 100644 --- a/simpleclient/pom.xml +++ b/simpleclient/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient @@ -54,5 +54,11 @@ 4.13.2 test + + org.apache.commons + commons-math3 + 3.6.1 + test + diff --git a/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java b/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java index 1ffb65382..78126efc8 100644 --- a/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java +++ b/simpleclient/src/main/java/io/prometheus/client/CKMSQuantiles.java @@ -1,15 +1,9 @@ package io.prometheus.client; -// Copied from https://raw.githubusercontent.com/Netflix/ocelli/master/ocelli-core/src/main/java/netflix/ocelli/stats/CKMSQuantiles.java +// The original implementation was copied from +// https://raw.githubusercontent.com/Netflix/ocelli/master/ocelli-core/src/main/java/netflix/ocelli/stats/CKMSQuantiles.java // Revision d0357b8bf5c17a173ce94d6b26823775b3f999f6 from Jan 21, 2015. -// -// This is the original code except for the following modifications: -// -// - Changed the type of the observed values from int to double. -// - Removed the Quantiles interface and corresponding @Override annotations. -// - Changed the package name. -// - Make get() return NaN when no sample was observed. -// - Make class package private +// However, it has been heavily refactored in the meantime. /* Copyright 2012 Andrew Wang (andrew@umbrant.com) @@ -28,266 +22,275 @@ */ import java.util.Arrays; +import java.util.Iterator; import java.util.LinkedList; import java.util.ListIterator; /** - * Implementation of the Cormode, Korn, Muthukrishnan, and Srivastava algorithm - * for streaming calculation of targeted high-percentile epsilon-approximate - * quantiles. - * - * This is a generalization of the earlier work by Greenwald and Khanna (GK), - * which essentially allows different error bounds on the targeted quantiles, - * which allows for far more efficient calculation of high-percentiles. - * - * - * See: Cormode, Korn, Muthukrishnan, and Srivastava - * "Effective Computation of Biased Quantiles over Data Streams" in ICDE 2005 - * - * Greenwald and Khanna, - * "Space-efficient online computation of quantile summaries" in SIGMOD 2001 - * + * Algorithm solving the "Targeted Quantile Problem" as described in + * "Effective Computation of Biased Quantiles over Data Streams" + * by Cormode, Korn, Muthukrishnan, and Srivastava. + * */ -class CKMSQuantiles { - /** - * Total number of items in stream. - */ - private int count = 0; +final class CKMSQuantiles { + + final Quantile[] quantiles; /** - * Used for tracking incremental compression. + * Total number of observations (not including those that are still in the buffer). */ - private int compressIdx = 0; + int n = 0; /** - * Current list of sampled items, maintained in sorted order with error - * bounds. + * List of sampled observations, ordered by Sample.value. */ - protected LinkedList sample; + final LinkedList samples = new LinkedList(); /** - * Buffers incoming items to be inserted in batch. + * Compress is called every compressInterval inserts. + * Note that the buffer is flushed whenever get() is called, so we + * cannot just wait until the buffer is full before we call compress. */ - private double[] buffer = new double[500]; - - private int bufferCount = 0; + private final int compressInterval = 128; + private int insertsSinceLastCompress = 0; /** - * Array of Quantiles that we care about, along with desired error. + * Note that the buffer size could as well be less than the compressInterval. + * However, the buffer size should not be greater than the compressInterval, + * because the compressInterval is not respected in flush(), so if you want + * to compress more often than calling flush() that won't work. */ - private final Quantile quantiles[]; + private final double[] buffer = new double[compressInterval]; + private int bufferPos = 0; - public CKMSQuantiles(Quantile[] quantiles) { + public CKMSQuantiles(Quantile... quantiles) { + if (quantiles.length == 0) { + throw new IllegalArgumentException("quantiles cannot be empty"); + } this.quantiles = quantiles; - this.sample = new LinkedList(); } /** - * Add a new value from the stream. - * - * @param value + * Add an observed value */ public void insert(double value) { - buffer[bufferCount] = value; - bufferCount++; + buffer[bufferPos++] = value; + + if (bufferPos == buffer.length) { + flush(); + } - if (bufferCount == buffer.length) { - insertBatch(); + if (++insertsSinceLastCompress == compressInterval) { compress(); + insertsSinceLastCompress = 0; } } + private void flush() { + Arrays.sort(buffer, 0, bufferPos); + insertBatch(buffer, bufferPos); + bufferPos = 0; + } + /** - * Get the estimated value at the specified quantile. - * - * @param q - * Queried quantile, e.g. 0.50 or 0.99. - * @return Estimated value at that quantile. + * Inserts the elements from index 0 to index toIndex from the sortedBuffer. */ - public double get(double q) { - // clear the buffer - insertBatch(); - compress(); - - if (sample.size() == 0) { - return Double.NaN; + void insertBatch(double[] sortedBuffer, int toIndex) { + if (toIndex == 0) { + return; } - - int rankMin = 0; - int desired = (int) (q * count); - - ListIterator it = sample.listIterator(); - Item prev, cur; - cur = it.next(); - while (it.hasNext()) { - prev = cur; - cur = it.next(); - - rankMin += prev.g; - - if (rankMin + cur.g + cur.delta > desired - + (allowableError(desired) / 2)) { - return prev.value; + ListIterator iterator = samples.listIterator(); + int i = 0; // position in buffer + int r = 0; // sum of g's left of the current sample + while (iterator.hasNext() && i < toIndex) { + Sample item = iterator.next(); + while (i < toIndex) { + if (sortedBuffer[i] > item.value) { + break; + } + insertBefore(iterator, sortedBuffer[i], r); + r++; // new item with g=1 was inserted before, so increment r + i++; + n++; } + r += item.g; } + while (i < toIndex) { + samples.add(new Sample(sortedBuffer[i], 0)); + i++; + n++; + } + } - // edge case of wanting max value - return sample.getLast().value; + private void insertBefore(ListIterator iterator, double value, int r) { + if (!iterator.hasPrevious()) { + samples.addFirst(new Sample(value, 0)); + } else { + iterator.previous(); + iterator.add(new Sample(value, f(r) - 1)); + iterator.next(); + } } /** - * Specifies the allowable error for this rank, depending on which quantiles - * are being targeted. - * - * This is the f(r_i, n) function from the CKMS paper. It's basically how - * wide the range of this rank can be. - * - * @param rank - * the index in the list of samples + * Get the estimated value at the specified quantile. */ - private double allowableError(int rank) { - // NOTE: according to CKMS, this should be count, not size, but this - // leads - // to error larger than the error bounds. Leaving it like this is - // essentially a HACK, and blows up memory, but does "work". - // int size = count; - int size = sample.size(); - double minError = size + 1; + public double get(double q) { + flush(); - for (Quantile q : quantiles) { - double error; - if (rank <= q.quantile * size) { - error = q.u * (size - rank); - } else { - error = q.v * rank; - } - if (error < minError) { - minError = error; - } + if (samples.size() == 0) { + return Double.NaN; } - return minError; - } - - private boolean insertBatch() { - if (bufferCount == 0) { - return false; + if (q == 0.0) { + return samples.getFirst().value; } - Arrays.sort(buffer, 0, bufferCount); - - // Base case: no samples - int start = 0; - if (sample.size() == 0) { - Item newItem = new Item(buffer[0], 1, 0); - sample.add(newItem); - start++; - count++; + if (q == 1.0) { + return samples.getLast().value; } - ListIterator it = sample.listIterator(); - Item item = it.next(); - - for (int i = start; i < bufferCount; i++) { - double v = buffer[i]; - while (it.nextIndex() < sample.size() && item.value < v) { - item = it.next(); + int r = 0; // sum of g's left of the current sample + int desiredRank = (int) Math.ceil(q * n); + int upperBound = desiredRank + f(desiredRank) / 2; + + ListIterator iterator = samples.listIterator(); + while (iterator.hasNext()) { + Sample sample = iterator.next(); + if (r + sample.g + sample.delta > upperBound) { + iterator.previous(); // roll back the item.next() above + if (iterator.hasPrevious()) { + Sample result = iterator.previous(); + return result.value; + } else { + return sample.value; + } } + r += sample.g; + } + return samples.getLast().value; + } - // If we found that bigger item, back up so we insert ourselves - // before it - if (item.value > v) { - it.previous(); + /** + * Error function, as in definition 5 of the paper. + */ + int f(int r) { + int minResult = Integer.MAX_VALUE; + for (Quantile q : quantiles) { + if (q.quantile == 0 || q.quantile == 1) { + continue; } - - // We use different indexes for the edge comparisons, because of the - // above - // if statement that adjusts the iterator - int delta; - if (it.previousIndex() == 0 || it.nextIndex() == sample.size()) { - delta = 0; - } - else { - delta = ((int) Math.floor(allowableError(it.nextIndex()))) - 1; + int result; + // We had a numerical error here with the following example: + // quantile = 0.95, epsilon = 0.01, (n-r) = 30. + // The expected result of (2*0.01*30)/(1-0.95) is 12. The actual result is 11.99999999999999. + // To avoid running into these types of error we add 0.00000000001 before rounding down. + if (r >= q.quantile * n) { + result = (int) (q.v * r + 0.00000000001); + } else { + result = (int) (q.u * (n - r) + 0.00000000001); + } + if (result < minResult) { + minResult = result; } - - Item newItem = new Item(v, 1, delta); - it.add(newItem); - count++; - item = newItem; } - - bufferCount = 0; - return true; + return Math.max(minResult, 1); } /** - * Try to remove extraneous items from the set of sampled items. This checks - * if an item is unnecessary based on the desired error bounds, and merges - * it with the adjacent item if it is. + * Merge pairs of consecutive samples if this doesn't violate the error function. */ - private void compress() { - if (sample.size() < 2) { + void compress() { + if (samples.size() < 3) { return; } - - ListIterator it = sample.listIterator(); - int removed = 0; - - Item prev = null; - Item next = it.next(); - - while (it.hasNext()) { - prev = next; - next = it.next(); - - if (prev.g + next.g + next.delta <= allowableError(it.previousIndex())) { - next.g += prev.g; - // Remove prev. it.remove() kills the last thing returned. - it.previous(); - it.previous(); - it.remove(); - // it.next() is now equal to next, skip it back forward again - it.next(); - removed++; + Iterator descendingIterator = samples.descendingIterator(); + int r = n; // n is equal to the sum of the g's of all samples + + Sample right; + Sample left = descendingIterator.next(); + r -= left.g; + + while (descendingIterator.hasNext()) { + right = left; + left = descendingIterator.next(); + r = r - left.g; + if (left == samples.getFirst()) { + // The min sample must never be merged. + break; + } + if (left.g + right.g + right.delta < f(r)) { + right.g += left.g; + descendingIterator.remove(); + left = right; } } } - private class Item { - public final double value; - public int g; - public final int delta; + static class Sample { + + /** + * Observed value. + */ + final double value; + + /** + * Difference between the lowest possible rank of this sample and its predecessor. + * This always starts with 1, but will be updated when compress() merges Samples. + */ + int g = 1; + + /** + * Difference between the greatest possible rank of this sample and the lowest possible rank of this sample. + */ + final int delta; - public Item(double value, int lower_delta, int delta) { + Sample(double value, int delta) { this.value = value; - this.g = lower_delta; this.delta = delta; } @Override public String toString() { - return String.format("I{val=%.3f, g=%d, del=%d}", value, g, delta); + return String.format("Sample{val=%.3f, g=%d, delta=%d}", value, g, delta); } } - public static class Quantile { - public final double quantile; - public final double error; - public final double u; - public final double v; + static class Quantile { + + /** + * Quantile. Must be between 0 and 1. + */ + final double quantile; + + /** + * Allowed error. Must be between 0 and 1. + */ + final double epsilon; + + /** + * Helper used in the error function f(), see definition 5 in the paper. + */ + final double u; + + /** + * Helper used in the error function f(), see definition 5 in the paper. + */ + final double v; + + Quantile(double quantile, double epsilon) { + if (quantile < 0.0 || quantile > 1.0) throw new IllegalArgumentException("Quantile must be between 0 and 1"); + if (epsilon < 0.0 || epsilon > 1.0) throw new IllegalArgumentException("Epsilon must be between 0 and 1"); - public Quantile(double quantile, double error) { this.quantile = quantile; - this.error = error; - u = 2.0 * error / (1.0 - quantile); - v = 2.0 * error / quantile; + this.epsilon = epsilon; + u = 2.0 * epsilon / (1.0 - quantile); // if quantile == 1 this will be Double.NaN + v = 2.0 * epsilon / quantile; // if quantile == 0 this will be Double.NaN } @Override public String toString() { - return String.format("Q{q=%.3f, eps=%.3f}", quantile, error); + return String.format("Quantile{q=%.3f, epsilon=%.3f}", quantile, epsilon); } } - } diff --git a/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java b/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java index ec321ba0a..5c5bf7c37 100644 --- a/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java +++ b/simpleclient/src/main/java/io/prometheus/client/SimpleCollector.java @@ -11,7 +11,7 @@ *

* This class handles common initialization and label logic for the standard metrics. * You should never subclass this class. - *

+ * *

Initialization

* After calling build() on a subclass, {@link Builder#name(String) name}, * {@link SimpleCollector.Builder#help(String) help}, diff --git a/simpleclient/src/main/java/io/prometheus/client/Summary.java b/simpleclient/src/main/java/io/prometheus/client/Summary.java index 34b123ba0..8836cbe7d 100644 --- a/simpleclient/src/main/java/io/prometheus/client/Summary.java +++ b/simpleclient/src/main/java/io/prometheus/client/Summary.java @@ -13,70 +13,91 @@ import java.util.concurrent.TimeUnit; /** - * Summary metric, to track the size of events. + * {@link Summary} metrics and {@link Histogram} metrics can both be used to monitor distributions like latencies or request sizes. *

- * Example of uses for Summaries include: - *

    - *
  • Response latency
  • - *
  • Request size
  • - *
- * + * An overview of when to use Summaries and when to use Histograms can be found on https://prometheus.io/docs/practices/histograms. *

- * Example Summaries: + * The following example shows how to measure latencies and request sizes: + * *

- * {@code
- *   class YourClass {
- *     static final Summary receivedBytes = Summary.build()
- *         .name("requests_size_bytes").help("Request size in bytes.").register();
- *     static final Summary requestLatency = Summary.build()
- *         .name("requests_latency_seconds").help("Request latency in seconds.").register();
+ * class YourClass {
  *
- *     void processRequest(Request req) {
- *        Summary.Timer requestTimer = requestLatency.startTimer();
- *        try {
- *          // Your code here.
- *        } finally {
- *          receivedBytes.observe(req.size());
- *          requestTimer.observeDuration();
- *        }
- *     }
+ *   private static final Summary requestLatency = Summary.build()
+ *       .name("requests_latency_seconds")
+ *       .help("request latency in seconds")
+ *       .register();
+ *
+ *   private static final Summary receivedBytes = Summary.build()
+ *       .name("requests_size_bytes")
+ *       .help("request size in bytes")
+ *       .register();
  *
- *     // Or if using Java 8 and lambdas.
- *     void processRequestLambda(Request req) {
+ *   public void processRequest(Request req) {
+ *     Summary.Timer requestTimer = requestLatency.startTimer();
+ *     try {
+ *       // Your code here.
+ *     } finally {
+ *       requestTimer.observeDuration();
  *       receivedBytes.observe(req.size());
- *       requestLatency.time(() -> {
- *         // Your code here.
- *       });
  *     }
- * }
+ *   }
  * }
  * 
- * This would allow you to track request rate, average latency and average request size. * + * The {@link Summary} class provides different utility methods for observing values, like {@link #observe(double)}, + * {@link #startTimer()} and {@link Timer#observeDuration()}, {@link #time(Callable)}, etc. *

- * How to add custom quantiles: + * By default, {@link Summary} metrics provide the {@code count} and the {@code sum}. For example, if you measure + * latencies of a REST service, the {@code count} will tell you how often the REST service was called, + * and the {@code sum} will tell you the total aggregated response time. + * You can calculate the average response time using a Prometheus query dividing {@code sum / count}. + *

+ * In addition to {@code count} and {@code sum}, you can configure a Summary to provide quantiles: + * *

- * {@code
- *     static final Summary myMetric = Summary.build()
- *             .quantile(0.5, 0.05)   // Add 50th percentile (= median) with 5% tolerated error
- *             .quantile(0.9, 0.01)   // Add 90th percentile with 1% tolerated error
- *             .quantile(0.99, 0.001) // Add 99th percentile with 0.1% tolerated error
- *             .name("requests_size_bytes")
- *             .help("Request size in bytes.")
- *             .register();
- * }
+ * Summary requestLatency = Summary.build()
+ *     .name("requests_latency_seconds")
+ *     .help("Request latency in seconds.")
+ *     .quantile(0.5, 0.01)    // 0.5 quantile (median) with 0.01 allowed error
+ *     .quantile(0.95, 0.005)  // 0.95 quantile with 0.005 allowed error
+ *     // ...
+ *     .register();
  * 
* - * The quantiles are calculated over a sliding window of time. There are two options to configure this time window: + * As an example, a 0.95 quantile of 120ms tells you that 95% of the calls were faster than 120ms, and 5% of the calls were slower than 120ms. + *

+ * Tracking exact quantiles require a large amount of memory, because all observations need to be stored in a sorted list. Therefore, we allow an error to significantly reduce memory usage. + *

+ * In the example, the allowed error of 0.005 means that you will not get the exact 0.95 quantile, but anything between the 0.945 quantile and the 0.955 quantile. + *

+ * Experiments show that the {@link Summary} typically needs to keep less than 100 samples to provide that precision, even if you have hundreds of millions of observations. + *

+ * There are a few special cases: + * *

    - *
  • maxAgeSeconds(long): Set the duration of the time window is, i.e. how long observations are kept before they are discarded. - * Default is 10 minutes. - *
  • ageBuckets(int): Set the number of buckets used to implement the sliding time window. If your time window is 10 minutes, and you have ageBuckets=5, - * buckets will be switched every 2 minutes. The value is a trade-off between resources (memory and cpu for maintaining the bucket) - * and how smooth the time window is moved. Default value is 5. + *
  • You can set an allowed error of 0, but then the {@link Summary} will keep all observations in memory.
  • + *
  • You can track the minimum value with {@code .quantile(0.0, 0.0)}. + * This special case will not use additional memory even though the allowed error is 0.
  • + *
  • You can track the maximum value with {@code .quantile(1.0, 0.0)}. + * This special case will not use additional memory even though the allowed error is 0.
  • *
* - * See https://prometheus.io/docs/practices/histograms/ for more info on quantiles. + * Typically, you don't want to have a {@link Summary} representing the entire runtime of the application, + * but you want to look at a reasonable time interval. {@link Summary} metrics implement a configurable sliding + * time window: + * + *
+ * Summary requestLatency = Summary.build()
+ *     .name("requests_latency_seconds")
+ *     .help("Request latency in seconds.")
+ *     .maxAgeSeconds(10 * 60)
+ *     .ageBuckets(5)
+ *     // ...
+ *     .register();
+ * 
+ * + * The default is a time window of 10 minutes and 5 age buckets, i.e. the time window is 10 minutes wide, and + * we slide it forward every 2 minutes. */ public class Summary extends SimpleCollector implements Counter.Describable { @@ -98,6 +119,10 @@ public static class Builder extends SimpleCollector.Builder { private long maxAgeSeconds = TimeUnit.MINUTES.toSeconds(10); private int ageBuckets = 5; + /** + * The class JavaDoc for {@link Summary} has more information on {@link #quantile(double, double)}. + * @see Summary + */ public Builder quantile(double quantile, double error) { if (quantile < 0.0 || quantile > 1.0) { throw new IllegalArgumentException("Quantile " + quantile + " invalid: Expected number between 0.0 and 1.0."); @@ -109,6 +134,10 @@ public Builder quantile(double quantile, double error) { return this; } + /** + * The class JavaDoc for {@link Summary} has more information on {@link #maxAgeSeconds(long)} + * @see Summary + */ public Builder maxAgeSeconds(long maxAgeSeconds) { if (maxAgeSeconds <= 0) { throw new IllegalArgumentException("maxAgeSeconds cannot be " + maxAgeSeconds); @@ -117,6 +146,10 @@ public Builder maxAgeSeconds(long maxAgeSeconds) { return this; } + /** + * The class JavaDoc for {@link Summary} has more information on {@link #ageBuckets(int)} + * @see Summary + */ public Builder ageBuckets(int ageBuckets) { if (ageBuckets <= 0) { throw new IllegalArgumentException("ageBuckets cannot be " + ageBuckets); diff --git a/simpleclient/src/test/java/io/prometheus/client/CKMSQuantilesTest.java b/simpleclient/src/test/java/io/prometheus/client/CKMSQuantilesTest.java new file mode 100644 index 000000000..f00371fec --- /dev/null +++ b/simpleclient/src/test/java/io/prometheus/client/CKMSQuantilesTest.java @@ -0,0 +1,340 @@ +package io.prometheus.client; + +import io.prometheus.client.CKMSQuantiles.Quantile; +import org.apache.commons.math3.distribution.NormalDistribution; +import org.apache.commons.math3.random.JDKRandomGenerator; +import org.apache.commons.math3.random.RandomGenerator; +import org.junit.Test; + +import java.util.*; + +import static org.junit.Assert.*; + +public class CKMSQuantilesTest { + + private final Quantile qMin = new Quantile(0.0, 0.00); + private final Quantile q50 = new Quantile(0.5, 0.01); + private final Quantile q95 = new Quantile(0.95, 0.005); + private final Quantile q99 = new Quantile(0.99, 0.001); + private final Quantile qMax = new Quantile(1.0, 0.00); + + @Test + public void testGetOnEmptyValues() { + CKMSQuantiles ckms = new CKMSQuantiles(q50, q95, q99); + assertTrue(Double.isNaN(ckms.get(q95.quantile))); + } + + @Test + public void testGet() { + Random random = new Random(0); + CKMSQuantiles ckms = new CKMSQuantiles(q50, q95, q99); + List input = shuffledValues(100, random); + for (double value : input) { + ckms.insert(value); + } + validateResults(ckms); + } + + @Test + public void testBatchInsert() { + Random random = new Random(1); + testInsertBatch(1, 1, 100, random); + testInsertBatch(1, 10, 100, random); + testInsertBatch(2, 10, 100, random); + testInsertBatch(2, 110, 100, random); // compress never called, because compress interval > number of inserts + testInsertBatch(3, 10, 100, random); + testInsertBatch(10, 10, 100, random); + testInsertBatch(128, 128, 1, random); + testInsertBatch(128, 128, 1000, random); + testInsertBatch(128, 128, 10*1000, random); + testInsertBatch(128, 128, 100*1000, random); + testInsertBatch(128, 128, 1000*1000, random); + } + + private void testInsertBatch(int batchSize, int compressInterval, int totalNumber, Random random) { + System.out.println("testInsertBatch(batchSize=" + batchSize + ", compressInterval=" + compressInterval + ", totalNumber=" + totalNumber + ")"); + CKMSQuantiles ckms = new CKMSQuantiles(q50, q95); + int insertsSinceCompress = 0; + List input = shuffledValues(totalNumber, random); + for (int i=0; i= compressInterval) { + ckms.compress(); + validateSamples(ckms); // after each compress the samples should still be valid + insertsSinceCompress=0; + } + } + validateResults(ckms); + } + + @Test + public void testGetWithAMillionElements() { + Random random = new Random(2); + List input = shuffledValues(1000*1000, random); + CKMSQuantiles ckms = new CKMSQuantiles(q50, q95, q99); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + assertTrue("sample size should be way below 1_000_000", ckms.samples.size() < 1000); + } + + @Test + public void testMin() { + Random random = new Random(3); + List input = shuffledValues(1000, random); + CKMSQuantiles ckms = new CKMSQuantiles(qMin); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + ckms.compress(); + assertEquals(2, ckms.samples.size()); + } + + @Test + public void testMax() { + Random random = new Random(4); + List input = shuffledValues(1000, random); + CKMSQuantiles ckms = new CKMSQuantiles(qMax); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + ckms.compress(); + assertEquals(2, ckms.samples.size()); + } + + @Test + public void testMinMax() { + Random random = new Random(5); + List input = shuffledValues(1000, random); + CKMSQuantiles ckms = new CKMSQuantiles(qMin, qMax); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + ckms.compress(); + assertEquals(2, ckms.samples.size()); + } + + @Test + public void testMinAndOthers() { + Random random = new Random(6); + List input = shuffledValues(1000, random); + CKMSQuantiles ckms = new CKMSQuantiles(q95, qMin); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + assertTrue(ckms.samples.size() < 200); // should be a lot less than input.size() + } + + @Test + public void testMaxAndOthers() { + Random random = new Random(7); + List input = shuffledValues(10000, random); + CKMSQuantiles ckms = new CKMSQuantiles(q50, q95, qMax); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + assertTrue(ckms.samples.size() < 200); // should be a lot less than input.size() + } + + @Test + public void testMinMaxAndOthers() { + Random random = new Random(8); + List input = shuffledValues(10000, random); + CKMSQuantiles ckms = new CKMSQuantiles(qMin, q50, q95, q99, qMax); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + assertTrue(ckms.samples.size() < 200); // should be a lot less than input.size() + } + + @Test + public void testExactQuantile() { + Random random = new Random(9); + List input = shuffledValues(10000, random); + CKMSQuantiles ckms = new CKMSQuantiles(new Quantile(0.95, 0)); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + // With epsilon == 0 we need to keep all inputs in samples. + assertEquals(input.size(), ckms.samples.size()); + } + + @Test + public void testExactAndOthers() { + Random random = new Random(10); + List input = shuffledValues(10000, random); + CKMSQuantiles ckms = new CKMSQuantiles(q50, new Quantile(0.95, 0), q99); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + // With epsilon == 0 we need to keep all inputs in samples. + assertEquals(input.size(), ckms.samples.size()); + } + + @Test + public void testExactAndMin() { + Random random = new Random(11); + List input = shuffledValues(10000, random); + CKMSQuantiles ckms = new CKMSQuantiles(qMin, q50, new Quantile(0.95, 0)); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + // With epsilon == 0 we need to keep all inputs in samples. + assertEquals(input.size(), ckms.samples.size()); + } + + @Test + public void testMaxEpsilon() { + Random random = new Random(12); + List input = shuffledValues(10000, random); + // epsilon == 1 basically gives you random results, but it should still not throw an exception. + CKMSQuantiles ckms = new CKMSQuantiles(new Quantile(0.95, 1)); + for (double v : input) { + ckms.insert(v); + } + validateResults(ckms); + } + + @Test + public void testGetGaussian() { + RandomGenerator rand = new JDKRandomGenerator(); + rand.setSeed(0); + + double mean = 0.0; + double stddev = 1.0; + NormalDistribution normalDistribution = new NormalDistribution(rand, mean, stddev, NormalDistribution.DEFAULT_INVERSE_ABSOLUTE_ACCURACY); + + List quantiles = new ArrayList(); + quantiles.add(new Quantile(0.10, 0.001)); + quantiles.add(new Quantile(0.50, 0.01)); + quantiles.add(new Quantile(0.90, 0.001)); + quantiles.add(new Quantile(0.95, 0.001)); + quantiles.add(new Quantile(0.99, 0.001)); + + CKMSQuantiles ckms = new CKMSQuantiles(quantiles.toArray(new Quantile[]{})); + + final int elemCount = 1000*1000; + double[] shuffle = normalDistribution.sample(elemCount); + + // insert a million samples + for (double v : shuffle) { + ckms.insert(v); + } + + // give the actual values for the quantiles we test + double p10 = normalDistribution.inverseCumulativeProbability(0.1); + double p90 = normalDistribution.inverseCumulativeProbability(0.9); + double p95 = normalDistribution.inverseCumulativeProbability(0.95); + double p99 = normalDistribution.inverseCumulativeProbability(0.99); + + //ε-approximate quantiles relaxes the requirement + //to finding an item with rank between (φ−ε)n and (φ+ε)n. + assertEquals(p10, ckms.get(0.1), errorBoundsNormalDistribution(0.1, 0.001, normalDistribution)); + assertEquals(mean, ckms.get(0.5), errorBoundsNormalDistribution(0.5, 0.01, normalDistribution)); + assertEquals(p90, ckms.get(0.9), errorBoundsNormalDistribution(0.9, 0.001, normalDistribution)); + assertEquals(p95, ckms.get(0.95), errorBoundsNormalDistribution(0.95, 0.001, normalDistribution)); + assertEquals(p99, ckms.get(0.99), errorBoundsNormalDistribution(0.99, 0.001, normalDistribution)); + + assertTrue("sample size should be below 1000", ckms.samples.size() < 1000); + } + + double errorBoundsNormalDistribution(double p, double epsilon, NormalDistribution nd) { + //(φ+ε)n + double upperBound = nd.inverseCumulativeProbability(p + epsilon); + //(φ−ε)n + double lowerBound = nd.inverseCumulativeProbability(p - epsilon); + // subtract and divide by 2, assuming that the increase is linear in this small epsilon. + return Math.abs(upperBound - lowerBound) / 2; + } + + @Test + public void testIllegalArgumentException() { + try { + new Quantile(-1, 0); + } catch (IllegalArgumentException e) { + assertEquals("Quantile must be between 0 and 1", e.getMessage()); + } catch (Exception e) { + fail("Wrong exception thrown" + e); + } + try { + new Quantile(0.95, 2); + } catch (IllegalArgumentException e) { + assertEquals("Epsilon must be between 0 and 1", e.getMessage()); + } catch (Exception e) { + fail("Wrong exception thrown" + e); + } + } + + private List shuffledValues(int n, Random random) { + List result = new ArrayList(n); + for (int i=0; i= lowerBound && actual <= upperBound; + if (!ok) { + for (CKMSQuantiles.Sample sample : ckms.samples) { + System.err.println(sample); + } + } + String errorMessage = q + ": " + actual + " not in [" + lowerBound + ", " + upperBound + "], n=" + ckms.n + ", " + q.quantile + "*" + ckms.n + "=" + (q.quantile*ckms.n); + assertTrue(errorMessage, ok); + } + } +} \ No newline at end of file diff --git a/simpleclient_bom/pom.xml b/simpleclient_bom/pom.xml index c8bf490fb..d79c63d0b 100644 --- a/simpleclient_bom/pom.xml +++ b/simpleclient_bom/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_bom diff --git a/simpleclient_caffeine/pom.xml b/simpleclient_caffeine/pom.xml index f273ec83b..90005fa7c 100644 --- a/simpleclient_caffeine/pom.xml +++ b/simpleclient_caffeine/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_caffeine @@ -36,12 +36,12 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT com.github.ben-manes.caffeine caffeine - 2.7.0 + 2.9.3 @@ -55,13 +55,13 @@ org.mockito mockito-core - 2.28.2 + 4.3.1 test org.assertj assertj-core - 2.6.0 + 3.22.0 test diff --git a/simpleclient_common/pom.xml b/simpleclient_common/pom.xml index 6fdd64144..fac75ceef 100644 --- a/simpleclient_common/pom.xml +++ b/simpleclient_common/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_common @@ -36,7 +36,7 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT diff --git a/simpleclient_common/src/test/java/io/prometheus/client/exporter/common/ExemplarTest.java b/simpleclient_common/src/test/java/io/prometheus/client/exporter/common/ExemplarTest.java index 469446cda..2ab2821f9 100644 --- a/simpleclient_common/src/test/java/io/prometheus/client/exporter/common/ExemplarTest.java +++ b/simpleclient_common/src/test/java/io/prometheus/client/exporter/common/ExemplarTest.java @@ -310,15 +310,15 @@ public void testSummaryNoLabels() throws IOException { .help("help") .quantile(0.5, 0.01) .register(registry); - for (int i=1; i<=11; i++) { // median is 5 + for (int i=1; i<=11; i++) { // median is 6 noLabelsDefaultExemplar.observe(i); } // Summaries don't have Exemplars according to the OpenMetrics spec. - assertOpenMetrics100Format("no_labels{quantile=\"0.5\"} 5.0\n"); + assertOpenMetrics100Format("no_labels{quantile=\"0.5\"} 6.0\n"); assertOpenMetrics100Format("no_labels_count 11.0\n"); assertOpenMetrics100Format("no_labels_sum 66.0\n"); - assert004Format("no_labels{quantile=\"0.5\",} 5.0\n"); + assert004Format("no_labels{quantile=\"0.5\",} 6.0\n"); assert004Format("no_labels_count 11.0\n"); assert004Format("no_labels_sum 66.0\n"); } @@ -331,15 +331,15 @@ public void testSummaryLabels() throws IOException { .labelNames("label") .quantile(0.5, 0.01) .register(registry); - for (int i=1; i<=11; i++) { // median is 5 + for (int i=1; i<=11; i++) { // median is 6 labelsNoExemplar.labels("test").observe(i); } // Summaries don't have Exemplars according to the OpenMetrics spec. - assertOpenMetrics100Format("labels{label=\"test\",quantile=\"0.5\"} 5.0\n"); + assertOpenMetrics100Format("labels{label=\"test\",quantile=\"0.5\"} 6.0\n"); assertOpenMetrics100Format("labels_count{label=\"test\"} 11.0\n"); assertOpenMetrics100Format("labels_sum{label=\"test\"} 66.0\n"); - assert004Format("labels{label=\"test\",quantile=\"0.5\",} 5.0\n"); + assert004Format("labels{label=\"test\",quantile=\"0.5\",} 6.0\n"); assert004Format("labels_count{label=\"test\",} 11.0\n"); assert004Format("labels_sum{label=\"test\",} 66.0\n"); } diff --git a/simpleclient_dropwizard/pom.xml b/simpleclient_dropwizard/pom.xml index 9993b45a8..b05def52f 100644 --- a/simpleclient_dropwizard/pom.xml +++ b/simpleclient_dropwizard/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_dropwizard @@ -34,12 +34,13 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.dropwizard.metrics metrics-core - 3.1.2 + 4.2.8 + provided @@ -51,7 +52,7 @@ org.assertj assertj-core - 2.9.1 + 3.22.0 test diff --git a/simpleclient_graphite_bridge/pom.xml b/simpleclient_graphite_bridge/pom.xml index 4f28d5aca..a5858ed20 100644 --- a/simpleclient_graphite_bridge/pom.xml +++ b/simpleclient_graphite_bridge/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_graphite_bridge @@ -37,7 +37,7 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT diff --git a/simpleclient_guava/pom.xml b/simpleclient_guava/pom.xml index 75bde610e..e4d7be62b 100644 --- a/simpleclient_guava/pom.xml +++ b/simpleclient_guava/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_guava @@ -36,7 +36,7 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT com.google.guava @@ -55,13 +55,13 @@ org.mockito mockito-core - 2.28.2 + 4.3.1 test org.assertj assertj-core - 2.6.0 + 3.22.0 test diff --git a/simpleclient_hibernate/pom.xml b/simpleclient_hibernate/pom.xml index c4f4fd704..585b71eba 100644 --- a/simpleclient_hibernate/pom.xml +++ b/simpleclient_hibernate/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_hibernate @@ -37,14 +37,14 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT org.hibernate hibernate-core - 5.2.0.Final + 5.6.5.Final provided @@ -58,7 +58,7 @@ org.mockito mockito-core - 2.18.0 + 4.3.1 test diff --git a/simpleclient_hotspot/pom.xml b/simpleclient_hotspot/pom.xml index 2e7543df1..d079df396 100644 --- a/simpleclient_hotspot/pom.xml +++ b/simpleclient_hotspot/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_hotspot @@ -36,14 +36,14 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_servlet - 0.14.1 + 0.15.1-SNAPSHOT test @@ -62,7 +62,7 @@ org.eclipse.jetty jetty-servlet - 8.1.7.v20120910 + 8.2.0.v20160908 test diff --git a/simpleclient_httpserver/pom.xml b/simpleclient_httpserver/pom.xml index 86fb104d2..76fa866d0 100644 --- a/simpleclient_httpserver/pom.xml +++ b/simpleclient_httpserver/pom.xml @@ -1,11 +1,23 @@ 4.0.0 + + + + org.apache.maven.plugins + maven-compiler-plugin + + 8 + 8 + + + + io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_httpserver @@ -36,12 +48,12 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT @@ -53,13 +65,13 @@ org.assertj assertj-core - 2.6.0 + 3.22.0 test javax.xml.bind jaxb-api - 2.3.0 + 2.4.0-b180830.0359 test diff --git a/simpleclient_httpserver/src/main/java/io/prometheus/client/exporter/HTTPServer.java b/simpleclient_httpserver/src/main/java/io/prometheus/client/exporter/HTTPServer.java index f56bc8860..e930c4e3e 100644 --- a/simpleclient_httpserver/src/main/java/io/prometheus/client/exporter/HTTPServer.java +++ b/simpleclient_httpserver/src/main/java/io/prometheus/client/exporter/HTTPServer.java @@ -72,6 +72,9 @@ public static class HTTPMetricHandler implements HttpHandler { private final LocalByteArray response = new LocalByteArray(); private final Supplier> sampleNameFilterSupplier; private final static String HEALTHY_RESPONSE = "Exporter is Healthy."; + private Integer metricRequests = 0; + private Integer healthRequests = 0; + private Integer reqCountRequests = 0; public HTTPMetricHandler(CollectorRegistry registry) { this(registry, null); @@ -90,8 +93,14 @@ public void handle(HttpExchange t) throws IOException { response.reset(); OutputStreamWriter osw = new OutputStreamWriter(response, Charset.forName("UTF-8")); if ("/-/healthy".equals(contextPath)) { + healthRequests++; osw.write(HEALTHY_RESPONSE); + } else if("/-/req-count".equals(contextPath)) { + reqCountRequests++; + Integer totalReqs = metricRequests + healthRequests + reqCountRequests; + osw.write(totalReqs.toString()); } else { + metricRequests++; String contentType = TextFormat.chooseContentType(t.getRequestHeaders().getFirst("Accept")); t.getResponseHeaders().set("Content-Type", contentType); Predicate filter = sampleNameFilterSupplier == null ? null : sampleNameFilterSupplier.get(); @@ -116,7 +125,9 @@ public void handle(HttpExchange t) throws IOException { } } else { long contentLength = response.size(); - t.getResponseHeaders().set("Content-Length", String.valueOf(contentLength)); + if (contentLength > 0) { + t.getResponseHeaders().set("Content-Length", String.valueOf(contentLength)); + } if (t.getRequestMethod().equals("HEAD")) { contentLength = -1; } @@ -132,11 +143,8 @@ protected static boolean shouldUseCompression(HttpExchange exchange) { if (encodingHeaders == null) return false; for (String encodingHeader : encodingHeaders) { - String[] encodings = encodingHeader.split(","); - for (String encoding : encodings) { - if (encoding.trim().equalsIgnoreCase("gzip")) { - return true; - } + if (encodingHeader.equals("gzip")) { + return true; } } return false; @@ -432,6 +440,10 @@ private HTTPServer(HttpServer httpServer, CollectorRegistry registry, boolean da if (authenticator != null) { mContext.setAuthenticator(authenticator); } + mContext = server.createContext("/-/req-count", mHandler); + if (authenticator != null) { + mContext.setAuthenticator(authenticator); + } executorService = Executors.newFixedThreadPool(5, NamedDaemonThreadFactory.defaultThreadFactory(daemon)); server.setExecutor(executorService); start(daemon); diff --git a/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/HttpRequest.java b/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/HttpRequest.java new file mode 100644 index 000000000..a08dcfeba --- /dev/null +++ b/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/HttpRequest.java @@ -0,0 +1,302 @@ +package io.prometheus.client.exporter; + +import javax.net.ssl.HostnameVerifier; +import javax.net.ssl.HttpsURLConnection; +import javax.net.ssl.SSLContext; +import javax.net.ssl.TrustManager; +import javax.xml.bind.DatatypeConverter; +import java.io.*; +import java.net.HttpURLConnection; +import java.net.URL; +import java.net.URLConnection; +import java.security.GeneralSecurityException; +import java.security.KeyManagementException; +import java.security.NoSuchAlgorithmException; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Scanner; +import java.util.Set; +import java.util.zip.GZIPInputStream; + +/** + * Class to perform HTTP testing + */ +public class HttpRequest { + + enum METHOD { GET, HEAD } + + private final Configuration configuration; + + /** + * Constructor + * + * @param configuration configuration + */ + private HttpRequest(Configuration configuration) { + this.configuration = configuration; + } + + /** + * Method to execute an HTTP request + * + * @return HttpResponse + * @throws IOException + */ + public HttpResponse execute() throws IOException { + if (configuration.url.toLowerCase().startsWith("https://") && (configuration.trustManagers != null)) { + try { + SSLContext sslContext = SSLContext.getInstance("SSL"); + sslContext.init(null, configuration.trustManagers, new java.security.SecureRandom()); + HttpsURLConnection.setDefaultSSLSocketFactory(sslContext.getSocketFactory()); + } catch (GeneralSecurityException e) { + throw new IOException(e); + } + + if (configuration.hostnameVerifier != null) { + HttpsURLConnection.setDefaultHostnameVerifier(configuration.hostnameVerifier); + } + } + + URLConnection urlConnection = new URL(configuration.url).openConnection(); + ((HttpURLConnection) urlConnection).setRequestMethod(configuration.method.toString()); + + Set>> entries = configuration.headers.entrySet(); + for (Map.Entry> entry : entries) { + for (String value : entry.getValue()) { + urlConnection.addRequestProperty(entry.getKey(), value); + } + } + + urlConnection.setUseCaches(false); + urlConnection.setDoInput(true); + urlConnection.setDoOutput(true); + urlConnection.connect(); + + Scanner scanner = new Scanner(urlConnection.getInputStream(), "UTF-8").useDelimiter("\\A"); + + return new HttpResponse( + ((HttpURLConnection) urlConnection).getResponseCode(), + urlConnection.getHeaderFields(), + urlConnection.getContentLength(), scanner.hasNext() ? scanner.next() : ""); + } + + /** + * Method to execute an HTTP request and decompresses it + * + * @return HttpResponse + * @throws IOException + */ + public HttpResponse executeAndDecompress() throws IOException { + if (configuration.url.toLowerCase().startsWith("https://") && (configuration.trustManagers != null)) { + try { + SSLContext sslContext = SSLContext.getInstance("SSL"); + sslContext.init(null, configuration.trustManagers, new java.security.SecureRandom()); + HttpsURLConnection.setDefaultSSLSocketFactory(sslContext.getSocketFactory()); + } catch (GeneralSecurityException e) { + throw new IOException(e); + } + + if (configuration.hostnameVerifier != null) { + HttpsURLConnection.setDefaultHostnameVerifier(configuration.hostnameVerifier); + } + } + + URLConnection urlConnection = new URL(configuration.url).openConnection(); + ((HttpURLConnection) urlConnection).setRequestMethod(configuration.method.toString()); + + Set>> entries = configuration.headers.entrySet(); + for (Map.Entry> entry : entries) { + for (String value : entry.getValue()) { + urlConnection.addRequestProperty(entry.getKey(), value); + } + } + + urlConnection.setUseCaches(false); + urlConnection.setDoInput(true); + urlConnection.setDoOutput(true); + urlConnection.connect(); + + InputStream stream = urlConnection.getInputStream(); + String body = this.decompress(stream); + + return new HttpResponse( + ((HttpURLConnection) urlConnection).getResponseCode(), + urlConnection.getHeaderFields(), + urlConnection.getContentLength(), body); + } + + private String decompress(InputStream stream) throws IOException { + GZIPInputStream gis = new GZIPInputStream(stream); + BufferedReader br = new BufferedReader(new InputStreamReader(gis, "UTF-8")); + StringBuilder sb = new StringBuilder(); + String line; + while((line = br.readLine()) != null) { + sb.append(line); + } + br.close(); + gis.close(); + return sb.toString(); + } + + /** + * Class to build an HttpRequest + */ + static class Builder { + + private final Configuration configuration; + + /** + * Constructor + */ + public Builder() { + configuration = new Configuration(); + } + + /** + * Method to set the HTTP request method + * + * @param method + * @return Builder + */ + public Builder withMethod(METHOD method) { + configuration.method = method; + return this; + } + + /** + * Method to set the HTTP request URL + * + * @param url + * @return Builder + */ + public Builder withURL(String url) { + configuration.url = url; + return this; + } + + /** + * Method to add an HTTP request header + * + * @param name + * @param value + * @return Builder + */ + public Builder withHeader(String name, String value) { + configuration.addHeader(name, value); + return this; + } + + /** + * Method to set the HTTP request "Authorization" header + * + * @param username + * @param password + * @return Builder + */ + public Builder withAuthorization(String username, String password) { + configuration.setHeader("Authorization", encodeCredentials(username, password)); + return this; + } + + /** + * Method to set the HTTP request trust managers when using an SSL URL + * + * @param trustManagers + * @return Builder + */ + public Builder withTrustManagers(TrustManager[] trustManagers) { + configuration.trustManagers = trustManagers; + return this; + } + + /** + * Method to set the HTTP request hostname verifier when using an SSL URL + * + * @param hostnameVerifier + * @return Builder + */ + public Builder withHostnameVerifier(HostnameVerifier hostnameVerifier) { + configuration.hostnameVerifier = hostnameVerifier; + return this; + } + + /** + * Method to build the HttpRequest + * + * @return HttpRequest + */ + public HttpRequest build() { + return new HttpRequest(configuration); + } + } + + /** + * Class used for Builder configuration + */ + private static class Configuration { + + public METHOD method; + public String url; + public Map> headers; + public TrustManager[] trustManagers; + public HostnameVerifier hostnameVerifier; + + /** + * Constructor + */ + Configuration() { + method = METHOD.GET; + headers = new HashMap>(); + } + + /** + * Method to add (append) an HTTP request header + * + * @param name + * @param value + * @return Configuration + */ + void addHeader(String name, String value) { + name = name.toLowerCase(); + List values = headers.get(name); + if (values == null) { + values = new LinkedList(); + headers.put(name, values); + } + + values.add(value); + } + + /** + * Method to set (overwrite) an HTTP request header, removing all previous header values + * + * @param name + * @param value + * @return Configuration + */ + void setHeader(String name, String value) { + List values = new LinkedList(); + values.add(value); + headers.put(name, values); + } + } + + /** + * Method to encode "Authorization" credentials + * + * @param username + * @param password + * @return String + */ + private final static String encodeCredentials(String username, String password) { + // Per RFC4648 table 2. We support Java 6, and java.util.Base64 was only added in Java 8, + try { + byte[] credentialsBytes = (username + ":" + password).getBytes("UTF-8"); + return "Basic " + DatatypeConverter.printBase64Binary(credentialsBytes); + } catch (UnsupportedEncodingException e) { + throw new IllegalArgumentException(e); + } + } +} diff --git a/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/HttpResponse.java b/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/HttpResponse.java new file mode 100644 index 000000000..9ad49be41 --- /dev/null +++ b/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/HttpResponse.java @@ -0,0 +1,109 @@ +package io.prometheus.client.exporter; + +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * Class to perform HTTP testing + */ +class HttpResponse { + + private int responseCode; + private Map> headers; + private long contentLength = -1; + private String body; + + /** + * Constructor + * + * @param responseCode + * @param headers + * @param contentLength + * @param body + */ + HttpResponse(int responseCode, Map> headers, long contentLength, String body) throws IOException { + this.responseCode = responseCode; + this.body = body; + this.contentLength = contentLength; + this.headers = new HashMap>(); + + Set>> headerSet = headers.entrySet(); + for (String header : headers.keySet()) { + if (header != null) { + List values = headers.get(header); + this.headers.put(header.toLowerCase(), values); + } + } + + if (getHeader("content-length") != null && getHeader("transfer-encoding") != null) { + throw new IOException("Invalid HTTP response, should only contain Connect-Length or Transfer-Encoding"); + } + } + + /** + * Method to get the HTTP response code + * + * @return int + */ + public int getResponseCode() { + return this.responseCode; + } + + /** + * Method to get a list of HTTP response headers values + * + * @param name + * @return List + */ + public List getHeaderList(String name) { + return headers.get(name.toLowerCase()); + } + + /** + * Method to get the first HTTP response header value + * + * @param name + * @return String + */ + public String getHeader(String name) { + String value = null; + + List valueList = getHeaderList(name); + if (valueList != null && (valueList.size() >= 0)) { + value = valueList.get(0); + } + + return value; + } + + /** + * Method to get the first HTTP response header value as a Long. + * Returns null of the header doesn't exist + * + * @param name + * @return Long + */ + public Long getHeaderAsLong(String name) { + String value = getHeader(name); + if (value != null) { + try { + return Long.valueOf(value); + } catch (Exception e) { + } + } + + return null; + } + + /** + * Method to get the HTTP response body + * + * @return String + */ + public String getBody() { + return body; + } +} diff --git a/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/TestHTTPServer.java b/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/TestHTTPServer.java index bb8f7fa13..11c15703d 100644 --- a/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/TestHTTPServer.java +++ b/simpleclient_httpserver/src/test/java/io/prometheus/client/exporter/TestHTTPServer.java @@ -5,30 +5,14 @@ import com.sun.net.httpserver.HttpServer; import com.sun.net.httpserver.HttpsConfigurator; import com.sun.net.httpserver.HttpsParameters; -import io.prometheus.client.Gauge; import io.prometheus.client.CollectorRegistry; - -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.UnsupportedEncodingException; -import java.net.HttpURLConnection; -import java.net.InetSocketAddress; -import java.net.URL; -import java.net.URLConnection; -import java.security.GeneralSecurityException; -import java.security.KeyStore; -import java.security.cert.X509Certificate; -import java.util.Scanner;; -import java.util.zip.GZIPInputStream; - +import io.prometheus.client.Gauge; import io.prometheus.client.SampleNameFilter; import org.junit.Assert; import org.junit.Before; -import org.junit.Test;; +import org.junit.Test; import javax.net.ssl.HostnameVerifier; -import javax.net.ssl.HttpsURLConnection; import javax.net.ssl.KeyManagerFactory; import javax.net.ssl.SSLContext; import javax.net.ssl.SSLEngine; @@ -38,6 +22,19 @@ import javax.net.ssl.TrustManagerFactory; import javax.net.ssl.X509TrustManager; import javax.xml.bind.DatatypeConverter; +import java.io.*; +import java.net.InetSocketAddress; +import java.nio.charset.Charset; +import java.nio.charset.StandardCharsets; +import java.security.GeneralSecurityException; +import java.security.KeyStore; +import java.security.cert.X509Certificate; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import java.util.stream.IntStream; +import java.util.zip.GZIPInputStream; +import java.util.zip.GZIPOutputStream; import static org.assertj.core.api.Java6Assertions.assertThat; @@ -66,45 +63,45 @@ public class TestHTTPServer { HTTPS_CONFIGURATOR = createHttpsConfigurator(SSL_CONTEXT); } - private final static TrustManager[] TRUST_MANAGERS = new TrustManager[]{ + /** + * TrustManager[] that trusts all certificates + */ + private final static TrustManager[] TRUST_ALL_CERTS_TRUST_MANAGERS = new TrustManager[]{ new X509TrustManager() { - public java.security.cert.X509Certificate[] getAcceptedIssuers() { + public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; } - public void checkClientTrusted(java.security.cert.X509Certificate[] certs, String authType) { + public void checkClientTrusted(X509Certificate[] certs, String authType) { } - public void checkServerTrusted(java.security.cert.X509Certificate[] certs, String authType) { + public void checkServerTrusted(X509Certificate[] certs, String authType) { } } }; - private final static HostnameVerifier HOSTNAME_VERIFIER = new HostnameVerifier() { + /** + * HostnameVerifier that accepts any hostname + */ + private final static HostnameVerifier TRUST_ALL_HOSTS_HOSTNAME_VERIFIER = new HostnameVerifier() { @Override public boolean verify(String hostname, SSLSession session) { return true; } }; - final static Authenticator createAuthenticator(String realm, final String validUsername, final String validPassword) { - return new BasicAuthenticator(realm) { - @Override - public boolean checkCredentials(String username, String password) { - return validUsername.equals(username) && validPassword.equals(password); - } - }; + HttpRequest.Builder createHttpRequestBuilder(HTTPServer httpServer, String urlPath) { + return new HttpRequest.Builder().withURL("http://localhost:" + httpServer.getPort() + urlPath); } - class Response { - - public long contentLength; - public String body; + HttpRequest.Builder createHttpRequestBuilderWithSSL(HTTPServer httpServer, String urlPath) { + return new HttpRequest.Builder().withURL("https://localhost:" + httpServer.getPort() + urlPath) + .withTrustManagers(TRUST_ALL_CERTS_TRUST_MANAGERS) + .withHostnameVerifier(TRUST_ALL_HOSTS_HOSTNAME_VERIFIER); + } - public Response(long contentLength, String body) { - this.contentLength = contentLength; - this.body = body; - } + HttpRequest.Builder createHttpRequestBuilder(HttpServer httpServer, String urlPath) { + return new HttpRequest.Builder().withURL("http://localhost:" + httpServer.getAddress().getPort() + urlPath); } @Before @@ -115,316 +112,287 @@ public void init() throws IOException { Gauge.build("c", "a help").register(registry); } - Response request(String requestMethod, HTTPServer s, String context, String suffix) throws IOException { - String url = "http://localhost:" + s.server.getAddress().getPort() + context + suffix; - URLConnection connection = new URL(url).openConnection(); - ((HttpURLConnection)connection).setRequestMethod(requestMethod); - connection.setDoOutput(true); - connection.connect(); - Scanner scanner = new Scanner(connection.getInputStream(), "UTF-8").useDelimiter("\\A"); - return new Response(connection.getContentLength(), scanner.hasNext() ? scanner.next() : ""); - } - - Response request(HTTPServer s, String context, String suffix) throws IOException { - return request("GET", s, context, suffix); - } - - Response request(HTTPServer s, String suffix) throws IOException { - return request(s, "/metrics", suffix); - } - - Response requestWithCompression(HTTPServer s, String suffix) throws IOException { - return requestWithCompression(s, "/metrics", suffix); - } - - Response requestWithCompression(HTTPServer s, String context, String suffix) throws IOException { - String url = "http://localhost:" + s.server.getAddress().getPort() + context + suffix; - URLConnection connection = new URL(url).openConnection(); - connection.setDoOutput(true); - connection.setDoInput(true); - connection.setRequestProperty("Accept-Encoding", "gzip, deflate"); - connection.connect(); - GZIPInputStream gzs = new GZIPInputStream(connection.getInputStream()); - Scanner scanner = new Scanner(gzs).useDelimiter("\\A"); - return new Response(connection.getContentLength(), scanner.hasNext() ? scanner.next() : ""); - } - - Response requestWithAccept(HTTPServer s, String accept) throws IOException { - String url = "http://localhost:" + s.server.getAddress().getPort(); - URLConnection connection = new URL(url).openConnection(); - connection.setDoOutput(true); - connection.setDoInput(true); - connection.setRequestProperty("Accept", accept); - Scanner scanner = new Scanner(connection.getInputStream(), "UTF-8").useDelimiter("\\A"); - return new Response(connection.getContentLength(), scanner.hasNext() ? scanner.next() : ""); - } - - Response requestWithCredentials(HTTPServer httpServer, String context, String suffix, String username, String password) throws IOException { - String url = "http://localhost:" + httpServer.server.getAddress().getPort() + context + suffix; - URLConnection connection = new URL(url).openConnection(); - connection.setDoOutput(true); - if (username != null && password != null) { - connection.setRequestProperty("Authorization", encodeCredentials(username, password)); - } - connection.connect(); - Scanner s = new Scanner(connection.getInputStream(), "UTF-8").useDelimiter("\\A"); - return new Response(connection.getContentLength(), s.hasNext() ? s.next() : ""); - } - - Response requestWithSSL(String requestMethod, String username, String password, HTTPServer s, String context, String suffix) throws GeneralSecurityException, IOException { - String url = "https://localhost:" + s.server.getAddress().getPort() + context + suffix; - - SSLContext sc = SSLContext.getInstance("SSL"); - sc.init(null, TRUST_MANAGERS, new java.security.SecureRandom()); - HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory()); - HttpsURLConnection.setDefaultHostnameVerifier(HOSTNAME_VERIFIER); - - URLConnection connection = new URL(url).openConnection(); - ((HttpURLConnection)connection).setRequestMethod(requestMethod); - - if (username != null && password != null) { - connection.setRequestProperty("Authorization", encodeCredentials(username, password)); - } - - connection.setDoOutput(true); - connection.connect(); - Scanner scanner = new Scanner(connection.getInputStream(), "UTF-8").useDelimiter("\\A"); - return new Response(connection.getContentLength(), scanner.hasNext() ? scanner.next() : ""); - } - - Response request(HttpServer httpServer, String context, String suffix) throws IOException { - String url = "http://localhost:" + httpServer.getAddress().getPort() + context + suffix; - URLConnection connection = new URL(url).openConnection(); - connection.setDoOutput(true); - connection.connect(); - Scanner s = new Scanner(connection.getInputStream(), "UTF-8").useDelimiter("\\A"); - return new Response(connection.getContentLength(), s.hasNext() ? s.next() : ""); - } - - String encodeCredentials(String username, String password) { - // Per RFC4648 table 2. We support Java 6, and java.util.Base64 was only added in Java 8, - try { - byte[] credentialsBytes = (username + ":" + password).getBytes("UTF-8"); - return "Basic " + DatatypeConverter.printBase64Binary(credentialsBytes); - } catch (UnsupportedEncodingException e) { - throw new IllegalArgumentException(e); - } - } - @Test(expected = IllegalArgumentException.class) public void testRefuseUsingUnbound() throws IOException { CollectorRegistry registry = new CollectorRegistry(); - HTTPServer s = new HTTPServer(HttpServer.create(), registry, true); - s.close(); + HTTPServer httpServer = new HTTPServer(HttpServer.create(), registry, true); + httpServer.close(); } @Test public void testSimpleRequest() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = request(s, "").body; - assertThat(response).contains("a 0.0"); - assertThat(response).contains("b 0.0"); - assertThat(response).contains("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics").build().execute().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).contains("b 0.0"); + assertThat(body).contains("c 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testBadParams() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = request(s, "?x").body; - assertThat(response).contains("a 0.0"); - assertThat(response).contains("b 0.0"); - assertThat(response).contains("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics?x").build().execute().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).contains("b 0.0"); + assertThat(body).contains("c 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testSingleName() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = request(s, "?name[]=a").body; - assertThat(response).contains("a 0.0"); - assertThat(response).doesNotContain("b 0.0"); - assertThat(response).doesNotContain("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics?name[]=a").build().execute().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).doesNotContain("b 0.0"); + assertThat(body).doesNotContain("c 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testMultiName() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = request(s, "?name[]=a&name[]=b").body; - assertThat(response).contains("a 0.0"); - assertThat(response).contains("b 0.0"); - assertThat(response).doesNotContain("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b").build().execute().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).contains("b 0.0"); + assertThat(body).doesNotContain("c 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testSampleNameFilter() throws IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() + .withRegistry(registry) + .withSampleNameFilter(new SampleNameFilter.Builder() + .nameMustNotStartWith("a") + .build()) + .build(); + + try { + String body = createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b").build().execute().getBody(); + assertThat(body).doesNotContain("a 0.0"); + assertThat(body).contains("b 0.0"); + assertThat(body).doesNotContain("c 0.0"); + } finally { + httpServer.close(); + } + } + + @Test + public void testSampleNameFilterEmptyBody() throws IOException { + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withSampleNameFilter(new SampleNameFilter.Builder() .nameMustNotStartWith("a") + .nameMustNotStartWith("b") .build()) .build(); + try { - String response = request(s, "?name[]=a&name[]=b").body; - assertThat(response).doesNotContain("a 0.0"); - assertThat(response).contains("b 0.0"); - assertThat(response).doesNotContain("c 0.0"); + HttpResponse httpResponse = createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b").build().execute(); + assertThat(httpResponse.getBody()).isEmpty(); } finally { - s.close(); + httpServer.close(); } } @Test public void testDecoding() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = request(s, "?n%61me[]=%61").body; - assertThat(response).contains("a 0.0"); - assertThat(response).doesNotContain("b 0.0"); - assertThat(response).doesNotContain("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics?n%61me[]=%61").build().execute().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).doesNotContain("b 0.0"); + assertThat(body).doesNotContain("c 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testGzipCompression() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = requestWithCompression(s, "").body; - assertThat(response).contains("a 0.0"); - assertThat(response).contains("b 0.0"); - assertThat(response).contains("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics") + .withHeader("Accept-Encoding", "gzip") + .build().executeAndDecompress().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).contains("b 0.0"); + assertThat(body).contains("c 0.0"); + + body = createHttpRequestBuilder(httpServer, "/metrics") + .withHeader("Accept-Encoding", "gzip") + .build().execute().getBody(); + assertThat(body).doesNotContain("a 0.0"); + assertThat(body).doesNotContain("b 0.0"); + assertThat(body).doesNotContain("c 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testOpenMetrics() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = requestWithAccept(s, "application/openmetrics-text; version=0.0.1,text/plain;version=0.0.4;q=0.5,*/*;q=0.1").body; - assertThat(response).contains("# EOF"); + String body = createHttpRequestBuilder(httpServer, "/metrics") + .withHeader("Accept", "application/openmetrics-text; version=0.0.1,text/plain;version=0.0.4;q=0.5,*/*;q=0.1") + .build().execute().getBody(); + assertThat(body).contains("# EOF"); } finally { - s.close(); + httpServer.close(); } } @Test public void testHealth() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = request(s, "/-/healthy", "").body; - assertThat(response).contains("Exporter is Healthy"); + String body = createHttpRequestBuilder(httpServer, "/-/healthy").build().execute().getBody(); + assertThat(body).contains("Exporter is Healthy"); } finally { - s.close(); + httpServer.close(); + } + } + + @Test + public void testTotalRequests() throws IOException { + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + + try { + String body = createHttpRequestBuilder(httpServer, "/-/req-count").build().execute().getBody(); + assertThat(body).contains("1"); + String body2 = createHttpRequestBuilder(httpServer, "/-/req-count").build().execute().getBody(); + assertThat(body2).contains("2"); + } finally { + httpServer.close(); } } @Test public void testHealthGzipCompression() throws IOException { - HTTPServer s = new HTTPServer(new InetSocketAddress(0), registry); + HTTPServer httpServer = new HTTPServer(new InetSocketAddress(0), registry); + try { - String response = requestWithCompression(s, "/-/healthy", "").body; - assertThat(response).contains("Exporter is Healthy"); + String body = createHttpRequestBuilder(httpServer, "/-/healthy") + .withHeader("Accept", "gzip") + .withHeader("Accept", "deflate") + .build().execute().getBody(); + assertThat(body).contains("Exporter is Healthy"); } finally { - s.close(); + httpServer.close(); } } @Test public void testBasicAuthSuccess() throws IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withAuthenticator(createAuthenticator("/", "user", "secret")) .build(); + try { - String response = requestWithCredentials(s, "/metrics","?name[]=a&name[]=b", "user", "secret").body; - assertThat(response).contains("a 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b") + .withAuthorization("user", "secret") + .build().execute().getBody(); + assertThat(body).contains("a 0.0"); } finally { - s.close(); + httpServer.close(); } } @Test public void testBasicAuthCredentialsMissing() throws IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withAuthenticator(createAuthenticator("/", "user", "secret")) .build(); + try { - request(s, "/metrics", "?name[]=a&name[]=b"); + createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b").build().execute().getBody(); Assert.fail("expected IOException with HTTP 401"); } catch (IOException e) { Assert.assertTrue(e.getMessage().contains("401")); } finally { - s.close(); + httpServer.close(); } } @Test public void testBasicAuthWrongCredentials() throws IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) - .withAuthenticator(createAuthenticator("/", "user", "wrong")) + .withAuthenticator(createAuthenticator("/", "user", "secret")) .build(); + try { - request(s, "/metrics", "?name[]=a&name[]=b"); + createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b") + .withAuthorization("user", "wrong") + .build().execute().getBody(); Assert.fail("expected IOException with HTTP 401"); } catch (IOException e) { Assert.assertTrue(e.getMessage().contains("401")); } finally { - s.close(); + httpServer.close(); } } @Test public void testHEADRequest() throws IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .build(); - try { - Response response = request("HEAD", s, "/metrics", "?name[]=a&name[]=b"); - Assert.assertNotNull(response); - Assert.assertTrue(response.contentLength == 74); - Assert.assertTrue("".equals(response.body)); + try { + HttpResponse httpResponse = createHttpRequestBuilder(httpServer, "/metrics?name[]=a&name[]=b") + .withMethod(HttpRequest.METHOD.HEAD) + .build().execute(); + Assert.assertNotNull(httpResponse); + Assert.assertNotNull(httpResponse.getHeaderAsLong("content-length")); + Assert.assertTrue(httpResponse.getHeaderAsLong("content-length") == 74); + assertThat(httpResponse.getBody()).isEmpty(); } finally { - s.close(); + httpServer.close(); } } @Test public void testHEADRequestWithSSL() throws GeneralSecurityException, IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withHttpsConfigurator(HTTPS_CONFIGURATOR) .build(); try { - Response response = requestWithSSL( - "HEAD", null, null, s, "/metrics", "?name[]=a&name[]=b"); - - Assert.assertNotNull(response); - Assert.assertTrue(response.contentLength == 74); - Assert.assertTrue("".equals(response.body)); + HttpResponse httpResponse = createHttpRequestBuilderWithSSL(httpServer, "/metrics?name[]=a&name[]=b") + .withMethod(HttpRequest.METHOD.HEAD) + .build().execute(); + Assert.assertNotNull(httpResponse); + Assert.assertNotNull(httpResponse.getHeaderAsLong("content-length")); + Assert.assertTrue(httpResponse.getHeaderAsLong("content-length") == 74); + assertThat(httpResponse.getBody()).isEmpty(); } finally { - s.close(); + httpServer.close(); } } @@ -436,10 +404,10 @@ public void testSimpleRequestHttpServerWithHTTPMetricHandler() throws IOExceptio httpServer.start(); try { - String response = request(httpServer, "/metrics", null).body; - assertThat(response).contains("a 0.0"); - assertThat(response).contains("b 0.0"); - assertThat(response).contains("c 0.0"); + String body = createHttpRequestBuilder(httpServer, "/metrics").build().execute().getBody(); + assertThat(body).contains("a 0.0"); + assertThat(body).contains("b 0.0"); + assertThat(body).contains("c 0.0"); } finally { httpServer.stop(0); } @@ -447,59 +415,84 @@ public void testSimpleRequestHttpServerWithHTTPMetricHandler() throws IOExceptio @Test public void testHEADRequestWithSSLAndBasicAuthSuccess() throws GeneralSecurityException, IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withHttpsConfigurator(HTTPS_CONFIGURATOR) .withAuthenticator(createAuthenticator("/", "user", "secret")) .build(); try { - Response response = requestWithSSL( - "HEAD", "user", "secret", s, "/metrics", "?name[]=a&name[]=b"); - - Assert.assertNotNull(response); - Assert.assertTrue(response.contentLength == 74); - Assert.assertTrue("".equals(response.body)); + HttpResponse httpResponse = createHttpRequestBuilderWithSSL(httpServer, "/metrics?name[]=a&name[]=b") + .withMethod(HttpRequest.METHOD.HEAD) + .withAuthorization("user", "secret") + .build().execute(); + Assert.assertNotNull(httpResponse); + Assert.assertNotNull(httpResponse.getHeaderAsLong("content-length")); + Assert.assertTrue(httpResponse.getHeaderAsLong("content-length") == 74); + assertThat(httpResponse.getBody()).isEmpty(); } finally { - s.close(); + httpServer.close(); } } @Test public void testHEADRequestWithSSLAndBasicAuthCredentialsMissing() throws GeneralSecurityException, IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withHttpsConfigurator(HTTPS_CONFIGURATOR) .withAuthenticator(createAuthenticator("/", "user", "secret")) .build(); try { - Response response = requestWithSSL("HEAD", null, null, s, "/metrics", "?name[]=a&name[]=b"); + createHttpRequestBuilderWithSSL(httpServer, "/metrics?name[]=a&name[]=b") + .withMethod(HttpRequest.METHOD.HEAD) + .build().execute(); Assert.fail("expected IOException with HTTP 401"); } catch (IOException e) { Assert.assertTrue(e.getMessage().contains("401")); } finally { - s.close(); + httpServer.close(); } } @Test public void testHEADRequestWithSSLAndBasicAuthWrongCredentials() throws GeneralSecurityException, IOException { - HTTPServer s = new HTTPServer.Builder() + HTTPServer httpServer = new HTTPServer.Builder() .withRegistry(registry) .withHttpsConfigurator(HTTPS_CONFIGURATOR) .withAuthenticator(createAuthenticator("/", "user", "secret")) .build(); try { - Response response = requestWithSSL("HEAD", "user", "wrong", s, "/metrics", "?name[]=a&name[]=b"); + createHttpRequestBuilderWithSSL(httpServer, "/metrics?name[]=a&name[]=b") + .withMethod(HttpRequest.METHOD.HEAD) + .withAuthorization("user", "wrong") + .build().execute(); Assert.fail("expected IOException with HTTP 401"); } catch (IOException e) { Assert.assertTrue(e.getMessage().contains("401")); } finally { - s.close(); + httpServer.close(); + } + } + + /** + * Encodes authorization credentials + * + * @param username + * @param password + * @return String + */ + private final static String encodeCredentials(String username, String password) { + // Per RFC4648 table 2. We support Java 6, and java.util.Base64 was only added in Java 8, + try { + byte[] credentialsBytes = (username + ":" + password).getBytes("UTF-8"); + return "Basic " + DatatypeConverter.printBase64Binary(credentialsBytes); + } catch (UnsupportedEncodingException e) { + throw new IllegalArgumentException(e); } } + /** * Create an SSLContext * @@ -511,7 +504,7 @@ public void testHEADRequestWithSSLAndBasicAuthWrongCredentials() throws GeneralS * @throws GeneralSecurityException * @throws IOException */ - public static SSLContext createSSLContext(String sslContextType, String keyStoreType, String keyStorePath, String keyStorePassword) + private final static SSLContext createSSLContext(String sslContextType, String keyStoreType, String keyStorePath, String keyStorePassword) throws GeneralSecurityException, IOException { SSLContext sslContext = null; FileInputStream fileInputStream = null; @@ -550,7 +543,25 @@ public static SSLContext createSSLContext(String sslContextType, String keyStore } /** + * Creates an Authenticator * + * @param realm + * @param validUsername + * @param validPassword + * @return Authenticator + */ + private final static Authenticator createAuthenticator(String realm, final String validUsername, final String validPassword) { + return new BasicAuthenticator(realm) { + @Override + public boolean checkCredentials(String username, String password) { + return validUsername.equals(username) && validPassword.equals(password); + } + }; + } + + /** + * Creates an HttpsConfiguration + * * @param sslContext * @return HttpsConfigurator */ diff --git a/simpleclient_jetty/pom.xml b/simpleclient_jetty/pom.xml index abf8cc29b..4ccae9591 100644 --- a/simpleclient_jetty/pom.xml +++ b/simpleclient_jetty/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_jetty @@ -38,7 +38,7 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT org.eclipse.jetty @@ -60,7 +60,7 @@ org.hamcrest hamcrest-all - 1.1 + 1.3 test diff --git a/simpleclient_jetty/src/main/java/io/prometheus/client/jetty/JettyStatisticsCollector.java b/simpleclient_jetty/src/main/java/io/prometheus/client/jetty/JettyStatisticsCollector.java index 8907ba151..7dddb00aa 100644 --- a/simpleclient_jetty/src/main/java/io/prometheus/client/jetty/JettyStatisticsCollector.java +++ b/simpleclient_jetty/src/main/java/io/prometheus/client/jetty/JettyStatisticsCollector.java @@ -9,8 +9,7 @@ /** * Collect metrics from jetty's org.eclipse.jetty.server.handler.StatisticsHandler. - *

- *

{@code
+ * 
  * Server server = new Server(8080);
  *
  * ServletContextHandler context = new ServletContextHandler();
@@ -29,7 +28,7 @@
  * server.setHandler(handlers);
  *
  * server.start();
- * }
+ *
*/ public class JettyStatisticsCollector extends Collector { private final StatisticsHandler statisticsHandler; diff --git a/simpleclient_jetty_jdk8/pom.xml b/simpleclient_jetty_jdk8/pom.xml index c9a1313e3..310d2d198 100644 --- a/simpleclient_jetty_jdk8/pom.xml +++ b/simpleclient_jetty_jdk8/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_jetty_jdk8 @@ -47,7 +47,7 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT org.eclipse.jetty @@ -69,7 +69,7 @@ org.hamcrest hamcrest-all - 1.1 + 1.3 test diff --git a/simpleclient_log4j/pom.xml b/simpleclient_log4j/pom.xml index eda703aa7..dfaec19d4 100644 --- a/simpleclient_log4j/pom.xml +++ b/simpleclient_log4j/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_log4j @@ -36,14 +36,20 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT - log4j - log4j - 1.2.17 + org.apache.logging.log4j + log4j-core + 2.17.1 + provided + + + org.apache.logging.log4j + log4j-1.2-api + 2.17.1 + provided - junit @@ -55,7 +61,7 @@ org.mockito mockito-core - 2.28.2 + 4.3.1 test diff --git a/simpleclient_log4j2/pom.xml b/simpleclient_log4j2/pom.xml index 4d2e64dab..1006522c8 100644 --- a/simpleclient_log4j2/pom.xml +++ b/simpleclient_log4j2/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_log4j2 @@ -36,12 +36,12 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT org.apache.logging.log4j log4j-core - 2.17.0 + 2.17.1 provided @@ -55,7 +55,7 @@ org.mockito mockito-core - 2.28.2 + 4.3.1 test diff --git a/simpleclient_logback/pom.xml b/simpleclient_logback/pom.xml index c245328a7..6642ec681 100644 --- a/simpleclient_logback/pom.xml +++ b/simpleclient_logback/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_logback @@ -36,12 +36,12 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT ch.qos.logback logback-classic - 1.2.0 + 1.2.10 @@ -55,7 +55,7 @@ org.mockito mockito-core - 2.28.2 + 4.3.1 test diff --git a/simpleclient_pushgateway/pom.xml b/simpleclient_pushgateway/pom.xml index ba10fb802..4971eb0ab 100644 --- a/simpleclient_pushgateway/pom.xml +++ b/simpleclient_pushgateway/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_pushgateway @@ -37,17 +37,17 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT javax.xml.bind jaxb-api - 2.3.0 + 2.4.0-b180830.0359 provided diff --git a/simpleclient_servlet/pom.xml b/simpleclient_servlet/pom.xml index 8deb19ac3..9aab58f58 100644 --- a/simpleclient_servlet/pom.xml +++ b/simpleclient_servlet/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_servlet @@ -40,22 +40,22 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_servlet_common - 0.14.1 + 0.15.1-SNAPSHOT javax.servlet javax.servlet-api - 3.0.1 + 3.1.0 provided @@ -68,19 +68,19 @@ org.assertj assertj-core - 2.6.0 + 3.22.0 test org.eclipse.jetty jetty-servlet - 8.1.7.v20120910 + 8.2.0.v20160908 test org.mockito mockito-core - 2.28.2 + 4.3.1 test diff --git a/simpleclient_servlet/src/main/java/io/prometheus/client/exporter/MetricsServlet.java b/simpleclient_servlet/src/main/java/io/prometheus/client/exporter/MetricsServlet.java index 76a6d5cee..3cd6af877 100644 --- a/simpleclient_servlet/src/main/java/io/prometheus/client/exporter/MetricsServlet.java +++ b/simpleclient_servlet/src/main/java/io/prometheus/client/exporter/MetricsServlet.java @@ -41,6 +41,7 @@ public MetricsServlet(CollectorRegistry registry, Predicate sampleNameFi @Override public void init(ServletConfig servletConfig) throws ServletException { try { + super.init(servletConfig); exporter.init(Adapter.wrap(servletConfig)); } catch (ServletConfigurationException e) { throw new ServletException(e); diff --git a/simpleclient_servlet_common/pom.xml b/simpleclient_servlet_common/pom.xml index debb21ac4..15578ea90 100644 --- a/simpleclient_servlet_common/pom.xml +++ b/simpleclient_servlet_common/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_servlet_common @@ -40,12 +40,12 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT @@ -57,7 +57,7 @@ org.assertj assertj-core - 2.6.0 + 3.22.0 test diff --git a/simpleclient_servlet_jakarta/pom.xml b/simpleclient_servlet_jakarta/pom.xml index 1707a0b51..766cfd7f1 100644 --- a/simpleclient_servlet_jakarta/pom.xml +++ b/simpleclient_servlet_jakarta/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_servlet_jakarta @@ -40,17 +40,17 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_servlet_common - 0.14.1 + 0.15.1-SNAPSHOT jakarta.servlet @@ -68,19 +68,19 @@ org.assertj assertj-core - 2.6.0 + 3.22.0 test org.eclipse.jetty jetty-servlet - 11.0.2 + 11.0.7 test org.mockito mockito-core - 2.28.2 + 4.3.1 test diff --git a/simpleclient_servlet_jakarta/src/main/java/io/prometheus/client/servlet/jakarta/exporter/MetricsServlet.java b/simpleclient_servlet_jakarta/src/main/java/io/prometheus/client/servlet/jakarta/exporter/MetricsServlet.java index bf8646af7..4617ebd48 100644 --- a/simpleclient_servlet_jakarta/src/main/java/io/prometheus/client/servlet/jakarta/exporter/MetricsServlet.java +++ b/simpleclient_servlet_jakarta/src/main/java/io/prometheus/client/servlet/jakarta/exporter/MetricsServlet.java @@ -40,6 +40,7 @@ public MetricsServlet(CollectorRegistry registry, Predicate sampleNameFi @Override public void init(ServletConfig servletConfig) throws ServletException { try { + super.init(servletConfig); exporter.init(wrap(servletConfig)); } catch (ServletConfigurationException e) { throw new ServletException(e); diff --git a/simpleclient_spring_boot/pom.xml b/simpleclient_spring_boot/pom.xml index 7a340dbb5..7c69ce2b2 100644 --- a/simpleclient_spring_boot/pom.xml +++ b/simpleclient_spring_boot/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_spring_boot @@ -51,32 +51,32 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_spring_web - 0.14.1 + 0.15.1-SNAPSHOT org.springframework.boot spring-boot-actuator - 1.5.4.RELEASE + 1.5.22.RELEASE org.springframework.boot spring-boot-starter-aop - 1.5.4.RELEASE + 1.5.22.RELEASE org.apache.commons commons-lang3 - 3.4 + 3.12.0 @@ -95,13 +95,13 @@ org.springframework.boot spring-boot-starter-test - 1.5.4.RELEASE + 1.5.22.RELEASE test org.springframework.boot spring-boot-starter-web - 1.5.4.RELEASE + 1.5.22.RELEASE test diff --git a/simpleclient_spring_web/pom.xml b/simpleclient_spring_web/pom.xml index a8bfe77c7..f0ff5280b 100644 --- a/simpleclient_spring_web/pom.xml +++ b/simpleclient_spring_web/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_spring_web @@ -51,37 +51,37 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT org.springframework spring-web - 4.3.9.RELEASE + 4.3.30.RELEASE org.springframework spring-aop - 4.3.9.RELEASE + 4.3.30.RELEASE org.springframework spring-context - 4.3.9.RELEASE + 4.3.30.RELEASE org.aspectj aspectjweaver - 1.8.6 + 1.9.7 org.apache.commons commons-lang3 - 3.4 + 3.12.0 @@ -94,7 +94,7 @@ org.springframework spring-test - 4.2.3.RELEASE + 4.3.30.RELEASE test diff --git a/simpleclient_tracer/pom.xml b/simpleclient_tracer/pom.xml index 14d573067..dd646182e 100644 --- a/simpleclient_tracer/pom.xml +++ b/simpleclient_tracer/pom.xml @@ -5,7 +5,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_tracer @@ -18,7 +18,7 @@ io.opentelemetry opentelemetry-api - 1.0.1 + 1.10.1 diff --git a/simpleclient_tracer/simpleclient_tracer_common/pom.xml b/simpleclient_tracer/simpleclient_tracer_common/pom.xml index bf4498212..acc02c7ce 100644 --- a/simpleclient_tracer/simpleclient_tracer_common/pom.xml +++ b/simpleclient_tracer/simpleclient_tracer_common/pom.xml @@ -5,7 +5,7 @@ io.prometheus simpleclient_tracer - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_tracer_common diff --git a/simpleclient_tracer/simpleclient_tracer_otel/pom.xml b/simpleclient_tracer/simpleclient_tracer_otel/pom.xml index 335213475..f60633366 100644 --- a/simpleclient_tracer/simpleclient_tracer_otel/pom.xml +++ b/simpleclient_tracer/simpleclient_tracer_otel/pom.xml @@ -5,7 +5,7 @@ io.prometheus simpleclient_tracer - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_tracer_otel diff --git a/simpleclient_tracer/simpleclient_tracer_otel_agent/pom.xml b/simpleclient_tracer/simpleclient_tracer_otel_agent/pom.xml index 82284541d..9ed206650 100644 --- a/simpleclient_tracer/simpleclient_tracer_otel_agent/pom.xml +++ b/simpleclient_tracer/simpleclient_tracer_otel_agent/pom.xml @@ -5,7 +5,7 @@ io.prometheus simpleclient_tracer - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_tracer_otel_agent diff --git a/simpleclient_vertx/pom.xml b/simpleclient_vertx/pom.xml index ca298114c..31619a4b3 100644 --- a/simpleclient_vertx/pom.xml +++ b/simpleclient_vertx/pom.xml @@ -17,7 +17,7 @@ io.prometheus parent - 0.14.1 + 0.15.1-SNAPSHOT simpleclient_vertx @@ -52,17 +52,17 @@ io.prometheus simpleclient - 0.14.1 + 0.15.1-SNAPSHOT io.prometheus simpleclient_common - 0.14.1 + 0.15.1-SNAPSHOT io.vertx vertx-web - 3.3.2 + 3.5.4 provided @@ -75,7 +75,7 @@ org.assertj assertj-core - 2.6.0 + 3.22.0 test