Monitoring (Graphite)

Graphite is a monitoring tool for servers and services. We are using it to monitor the runtime performance and behavior of the SkFiddle.com and the new SkPerf services, and maybe other services in the future.

This document describes the setup procedure for the Graphite server and the process for loading data into the server.

Full Server Setup

Do once

$ ./vm_create_instance.sh
$ ./vm_setup_instance.sh

Make sure to ‘set daemon 2’ in /etc/monit/monitrc so that monit runs every 2 seconds.

After the setup has completed once, do the following

sudo su www-data
cd /home/www-data/graphite/store/
rm -rf whisper
ln -s /mnt/graphite-data/whisper whisper

If the data disk doesn't exist you will need to create it and attach it using the following:

sudo mkdir -p /mnt/graphite-data sudo /usr/share/google/safe_format_and_mount -m “mkfs.ext4 -F”
/dev/disk/by-id/google-skia-monitoring-data-$ZONE_TAG /mnt/graphite-data

Do on update

$ ./vm_push_update.sh

Notes

To SSH into the instance:

gcutil --project=google.com:skia-buildbots ssh --ssh_user=default skia-monitoring-b

If you need to modify the constants for the vm_XXX.sh scripts they are specified in compute_engine_scripts/buildbot/vm_config.sh.