Tuesday, June 23, 2026

Priorities

Work Log

Switch Grafana Data Source to the new Prometheus instance — DA-1350

Situation

The Infrastructure team has switched the data source for service metrics to a new prometheus instance. This decision was made to enhance the reliability of the observability platform. However, the downside is that the Grafana dashboards and alerts require the new data source.

Task

I need to change the Grafana dashboards and alerts for the data team services to the new version with name vm-prometheus-{env}. Otherwise, The Grafana dashboard will not show any metrics and multiple alerts will trigger because of no data state.

Action

I modified the data source in the Grafana dashboards for each data team service. Additionally, I updated the alert rules to utilize the new data source.

Result

As a result, the Grafana dashboards for the data team service now display metrics from the new data source. Furthermore, the alerts are now reading from the new data source.

Blockers

N/A

Carry-overs

Reflection

N/A