I am trying to get some custom application metrics captured in golang using the prometheus client library to show up in Prometheus.
I have the following working:
-
I have a go application which is exposing metrics on localhost:8080/metrics as described in this article:
https://godoc.org/github.com/prometheus/client_golang/prometheus
-
I have a kubernates minikube running which has Prometheus, Grafana and AlertManager running using the operator from this article:
https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus
I created a docker image for my go app, when I run it and go to localhost:8080/metrics I can see the prometheus metrics showing up in a browser.
I use the following pod.yaml to deploy my docker image to a pod in k8s
apiVersion: v1 kind: Pod metadata: name: my-app-pod labels: zone: prod version: v1 annotations: prometheus.io/scrape: 'true' prometheus.io/port: '8080' spec: containers: - name: my-container image: name/my-app:latest imagePullPolicy: IfNotPresent ports: - containerPort: 8080
- If I connect to my pod using:
kubectl exec -it my-app-pod -- /bin/bash
then do wget on "localhost:8080/metrics", I can see my metrics
So far so good, here is where I am hitting a wall. I could have multiple pods running this same image. I want to expose all the images to prometheus as targets. How do I configure my pods so that they show up in prometheus so I can report on my custom metrics?
Thanks for any help offered!