opentelemetry collector推送数据到prometheus失败

Question

Java将Counter指标推送到opentelemetry-collector-contrib容器, 但无法同步到prometheus, opentelemetry-collector可以看到推送的指标值, prometheus中没有任何数据

这个简单的demo 参考的是 https://github.com/prometheus/client_java/tree/main/examples/example-exporter-opentelemetry

配置如下

1.docker启动 prometheus

docker run -d --restart always --name prometheus_test \
        -p 19090:9090 \
        -v /path/prometheus.yaml:/etc/prometheus/prometheus.yaml \
        prom/prometheus:v2.47.0

prometheus.yaml文件内容为空

2.docker启动 opentelemetry-collector

docker run -d --name otel-collector-contrib-test \
        -p 4317:4317 -p 55678:55678 \
        -v /path/otelcol-config.yaml:/config.yaml \
        otel/opentelemetry-collector-contrib

otelcol-config.yaml:

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:

exporters:
  prometheusremotewrite:
    endpoint: 'http://prometheus容器ip:19090/api/v1/write'
  logging:
    verbosity: detailed

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, prometheusremotewrite]

3. java代码

package org.example;

import io.prometheus.metrics.core.metrics.Counter;
import io.prometheus.metrics.exporter.opentelemetry.OpenTelemetryExporter;

/**
 * Hello world!
 *
 */
public class App 
{
    public static void main( String[] args ) throws InterruptedException {
        String[] LABELS = {"appid", "method", "result"};
        Counter counter = Counter.builder()
                .name("test_rpc_requests_total")
                .help("接口请求总数")
                .labelNames(LABELS)
                .register();

        OpenTelemetryExporter.builder()
                .endpoint("http://opentelemetry collect容器ip:4317")
                .intervalSeconds(10) // ridiculously short interval for demo purposes
                .buildAndStart();

        while (true) {
            Thread.sleep(1000);
            counter.labelValues("appid", "method1", "success").inc();
        }
    }
}

maven依赖:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>openTelemetryDemo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>openTelemetryDemo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>io.prometheus</groupId>
      <artifactId>prometheus-metrics-core</artifactId>
      <version>1.2.0-SNAPSHOT</version>
    </dependency>
    <dependency>
      <groupId>io.prometheus</groupId>
      <artifactId>prometheus-metrics-exporter-opentelemetry</artifactId>
      <version>1.2.0-SNAPSHOT</version>
    </dependency>
  </dependencies>
</project>

Answer 1

你启动Prometheus 的配置少了关键的一行 --web.enable-remote-write-receiver，启动collector的配置也少了一行 --config=file:/config.yaml。
根据你的配置我本地复现出来了相同的现象，参考 官方配置 https://github.com/prometheus/client_java/blob/main/examples/example-exporter-opentelemetry/docker-compose.yaml