Я использую Prometheus в качестве службы:
admin1@POC-k8s-master:/tmp$ sudo systemctl status prometheus
[sudo] password for admin1:
● prometheus.service - Prometheus Monitoring
Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-01-09 04:25:33 EST; 1h 55min ago
Main PID: 8812 (prometheus)
Tasks: 12
Memory: 97.8M
CPU: 20.163s
CGroup: /system.slice/prometheus.service
└─8812 /usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/prometheus/consoles --web.console.libraries=/
Jan 09 04:25:34 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:34.1310759Z caller=main.go:222 host_details="(Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 POC-
Jan 09 04:25:34 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:34.1312489Z caller=main.go:223 fd_limits="(soft=1024, hard=4096)"
Jan 09 04:25:34 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:34.1339119Z caller=main.go:504 msg="Starting TSDB ..."
Jan 09 04:25:34 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:34.13421Z caller=web.go:382 component=web msg="Start listening for connections" address=0.0.0.0:9090
Jan 09 04:25:35 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:35.1504977Z caller=main.go:514 msg="TSDB started"
Jan 09 04:25:35 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:35.1506174Z caller=main.go:588 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
Jan 09 04:25:35 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T09:25:35.1509515Z caller=main.go:491 msg="Server is ready to receive web requests."
Jan 09 06:00:01 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T11:00:01.4377765Z caller=compact.go:393 component=tsdb msg="compact blocks" count=1 mint=1578556800000 maxt=1578564000000
Jan 09 06:00:02 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T11:00:02.1968892Z caller=head.go:348 component=tsdb msg="head GC completed" duration=2.7095ms
Jan 09 06:00:02 POC-k8s-master prometheus[8812]: level=info ts=2020-01-09T11:00:02.4665533Z caller=head.go:357 component=tsdb msg="WAL truncation completed" duration=269.4061ms
Графана как объект K8s:
admin1@POC-k8s-master:~$ kubectl get all --all-namespaces | grep grafana
grafana pod/grafana-79465c4ffc-l4bxg 1/1 Running 4 21d
grafana service/grafana ClusterIP 10.97.79.21 <none> 80/TCP 21d
grafana service/grafana-ext NodePort 10.109.58.242 <none> 3000:32767/TCP 21d
grafana deployment.apps/grafana 1/1 1 1 21d
grafana replicaset.apps/grafana-79465c4ffc 1 1 1 21d
Снимок экрана без точек данных:
Снимок экрана для показателей экспортера узлов: 
Снимок экрана для Prometheus:
Снимок экрана для источника данных Prometheus, работающего в графане:
Пожалуйста, помогите мне решить эту проблему!
Обновление: admin1@POC-k8s-master: ~ $ kubectl logs grafana-79465c4ff c -l4bxg -n grafana -f
t=2020-01-02T11:01:13+0000 lvl=info msg="Starting Grafana" logger=server version=5.0.4 commit=7dc36ae compiled=2018-03-28T11:52:41+0000
t=2020-01-02T11:01:13+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini
t=2020-01-02T11:01:13+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from command line" logger=settings arg="default.paths.data=/var/lib/grafana"
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from command line" logger=settings arg="default.paths.logs=/var/log/grafana"
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from command line" logger=settings arg="default.paths.plugins=/var/lib/grafana/plugins"
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from command line" logger=settings arg="default.paths.provisioning=/etc/grafana/provisioning"
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from command line" logger=settings arg="default.log.mode=console"
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_SECURITY_ADMIN_USER=admin"
t=2020-01-02T11:01:13+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_SECURITY_ADMIN_PASSWORD=*********"
t=2020-01-02T11:01:13+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana
t=2020-01-02T11:01:13+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana/data
t=2020-01-02T11:01:13+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana
t=2020-01-02T11:01:13+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins
t=2020-01-02T11:01:13+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning
t=2020-01-02T11:01:13+0000 lvl=info msg="App mode production" logger=settings
t=2020-01-02T11:01:13+0000 lvl=info msg="Initializing DB" logger=sqlstore dbtype=sqlite3
t=2020-01-02T11:01:13+0000 lvl=info msg="Starting DB migration" logger=migrator
t=2020-01-02T11:01:13+0000 lvl=info msg="Executing migration" logger=migrator id="copy data account to org"
t=2020-01-02T11:01:13+0000 lvl=info msg="Skipping migration condition not fulfilled" logger=migrator id="copy data account to org"
t=2020-01-02T11:01:13+0000 lvl=info msg="Executing migration" logger=migrator id="copy data account_user to org_user"
t=2020-01-02T11:01:13+0000 lvl=info msg="Skipping migration condition not fulfilled" logger=migrator id="copy data account_user to org_user"
t=2020-01-02T11:01:13+0000 lvl=info msg="Starting plugin search" logger=plugins
t=2020-01-02T11:01:13+0000 lvl=info msg="Initializing Alerting" logger=alerting.engine
t=2020-01-02T11:01:13+0000 lvl=info msg="Initializing CleanUpService" logger=cleanup
t=2020-01-02T11:01:14+0000 lvl=info msg="Initializing Stream Manager"
t=2020-01-02T11:01:14+0000 lvl=info msg="Initializing HTTP Server" logger=http.server address=0.0.0.0:3000 protocol=http subUrl= socket=