GitSync Production Manual

General Metrics

The following dashboard is for the skia-public instances: https://grafana2.skia.org/d/Onp7_5FWk/gitsync

The following dashboard is for the skia-corp instances: https://skia-mon.corp.goog/d/Wi0Yu5FZk/gitsync

Some things to look for:

• Do goroutines or memory increase continuously (e.g leaks)?
• Have any repos taken more than a few seconds to sync?
• Is there an elevated error rate?

General Logs

Logs for GitSync instances in skia-public/skia-corp are in the usual GKE container grouping, for example: https://console.cloud.google.com/logs/viewer?project=skia-public&resource=container&logName=projects%2Fskia-public%2Flogs%2Fgitsync2

Alerts

Items below here should include target links from alerts.

GitSyncStalled

This alert means we haven't successfully synced a repo in over 5 minutes. This could be due to failure to communicate with the Gitiles server, or because of a problem with GitSync itself. Check the logs for details.

Key metrics: liveness_last_successful_git_sync_s

GitSyncErrorRate

The log error rate is elevated. There are a number of possible causes; check the logs and verify that things are working as expected.

Key metrics: rate(num_log_lines{level=“ERROR”,app=~“gitsync.*”}[30m])