DESIGN

Overview

Provides interactive dashboard for Skia performance data.

Code Locations

The code for the server along with VM instance setup scripts is kept in:

Architecture

This is the general flow of data for the Skia performance application. The frontend is available at http://skiaperf.com.

                                                       +-------------+                           |             |                           |   Browser   |                           |             |                           |             |                           |             |                           +----------^--+                                      |                         +--------------------+----+-----+         |   GCE Instance|skia+perf+b    |         |               |               |         |   +-----------+----------+    |         |   |     Squid3           |    |         |   |                      |    |         |   +--------^-------------+    |         |            |                  |         | +----------+-------------+    |         | |    Perf (Go)      |    |         | | ^    ^                 |    |         | +------------------------+    |         |   |    |                      |         |   |    |                      |         |   |    | +------------------+ |         |   |    | |Tile Pipeline (Go)| |         |   |    | |            ^     | |         |   |    | +--+---------------+ |         |   |    |    |         |       |         +-------------------------------+             |    |    |         |           +---------+-+  |    | +-------+--+        |   MySQL   |  |    | | BigQuery |        |           |  |    | |          |        |           |  |    | |          |        |           |  |    | |          |        |           |  |    | |          |        |           |  |    | |          |        +-----------+  |    | +----------+                       |    |                                  +-+----v---+                              |   Tile   |                              |   Repo   |                              |          |                              |          |                              |          |                              |          |                              +----------+                                                         Perf is a Go application that serves the HTML, CSS, JS and the JSON representations that the JS needs. It loads test results in the form of ‘tiles’ from the Tile Repo. It combines that data with data about commits and annotations from the MySQL data base and serves that the UI.

The Tile Pipeline is a separate application that periodically queries for fresh data from BigQuery and then writes Tiles into the Tile Repo. Note that when ingestion moves out of prod and into the same server we can do the tile updates immediately after ingestion is done.

Tile Repo will be represented internally as an interface, the first implemetation will be as files on the local disk, with a directory tree that contains gzipped JSON files called tiles. Note that we may alternatively use Go native gob encoded files and just transform then into JSON when serving them to the UI.

Each tile contains exactly 16 points of every trace for a dataset. The one exception being the last.gz tile, which may contain less that 16 points; see below for an explanation of that. The Tile Repo directory structure is:

$TILE_REPO_ROOT/<dataset>/<scale>/<tilenumber>.gz

Where:

  • dataset = {skps|micro}
  • scale = 0..5 The scale factor of 4^N, so points in the /0/ directory represent 1:1 with test results, while tiles in the /1/ directory have every fourth commit with data, and /2/ has every 16th commit with data.
  • tilenumber = The number of the tile, at the given scale, starting at BOT (Beginning of Time).

So the data in:

/skps/0/0.gz

contains the data for the first 16 commits from the BOT that have test data.

/micro/3/6.gz

contains the 16 commits per trace of every 64th commit that has data and is the 7th data set in that order. So it contains 16 points per trace, and each point falls between 6 * 64 * 16=6144 and 7 * 64 * 16=7168, i.e it is a slice of 16 commits that represent a range of 1024 commits.

A manifest file will be available that give the commit timestamp ranges for each tile.

When navigating the UI users can select the tiles they are looking at (<, >) and also change the scaling factor that they are looking at (+,-).

Each /dataset/scale directory also contains a file, last.gz, that contains the most recent data, from 1 to 16 points. The Tile Pipeline process will update the ‘last.gz’ files in each directory and write new tile files as new data arrives. Last.gz will contain the id of the tile that appears just before it, that way the UI can request /skps/0/last.gz and know how to proceed from there.

URL Structure

The URL structure for retrieving Datasets is TBD.

Navigating

For each point if the user wants to zoom out, add 1 to the scale factor and divide tilenumber by two. Do the opposite to zoom in. To move forwards or backwards in time add or subtract 1 to the tile number. The actual UI mechanisms for navigating around traces are TBD, this is just a description of how the tiles are arranged.

Tile Pipeline Algorithm

Start at /0/last.gz and find the previous tilenumber.gz, open that and find the last githash. Query for all data newer than that githash. Group into tiles of 16 githashes. Put the remainder (or the last 16 if the remainder is 0) into /0/last.gz.

Now do the same for /1/, but the data comes from /0/ and not from BigQuery. I.e. /1/0.gz is just a sampling of /0/0.gz, /0/1.gz, /0/2.gz and /0/3.gz. At each level only rewrite last.gz and write out any new complete tiles as they are filled.

Perf Stats Database

The data for the performance metrics are kept in the BigQuery tables stored in the google.com:chrome-skia project. Note that this is a different project from where the data is accessed, which is by VM instances running under the google.com:skia-buildbots project. For this to work the service account email of the VM needs to be added to the permissions group of the google.com:chrome-skia project. If this isn't done then the BigQuery access will fail with a 403 error.

Logs

We use the https://github.com/golang/glog for logging, which puts Google style Error, Warning and Info logs in /tmp on the server under the ‘perf’ account.

Debugging Tips

Starting the application is done via /etc/init.d/perf which does the backgrounding itself via start-stop-daemon, which means that if the app crashes when first starting then nothing will make it to the logs. To debug the cause in that case edit /etc/init.d/perf and remove the --background flag and then run:

$ sudo /etc/init.d/perf start

And you should get stdout and stderr output.

Monitoring

Monitoring of the application is done via Graphite at http://skiamonitor.com. Both system and application level metrics are monitored.

Annotations Database

A Cloud SQL (a cloud version of MySQL) database is used to keep information on Skia git revisions and their corresponding annotations. The database will be updated when users add/edit/delete annotations via the dashboard UI.

All passwords for MySQL are stored in valentine (search “skia perf”).

To connect to the database from authorized network (including skia-perf GCE):

$ mysql -h 173.194.104.24 -u root -p

Initial setup of the database, the users, and the tables:

CREATE DATABASE skia;
USE skia;
CREATE USER 'readonly'@'%' IDENTIFIED BY <password in valentine>;
GRANT SELECT ON *.* TO 'readonly'@'%';
CREATE USER 'readwrite'@'%' IDENTIFIED BY <password in valentine>;
GRANT SELECT, DELETE, UPDATE, INSERT ON *.* TO 'readwrite'@'%';

// Table for storing annotations.
CREATE TABLE notes (
  id     INT       NOT NULL AUTO_INCREMENT PRIMARY KEY,
  type   TINYINT,
  author TEXT,
  notes  TEXT      NOT NULL
);

// Table for storing git revision information.
CREATE TABLE githash (
  githash   VARCHAR(40)   NOT NULL PRIMARY KEY,
  ts        TIMESTAMP     NOT NULL,
  gitnumber INT           NOT NULL,
  author    TEXT          NOT NULL,
  message   TEXT          NOT NULL
);

// Table for mapping revisions and annotations. This support many-to-many
// mapping.
CREATE TABLE githashnotes (
  githash VARCHAR(40)  NOT NULL,
  ts      TIMESTAMP    NOT NULL,
  id      INT          NOT NULL,

  FOREIGN KEY (githash) REFERENCES githash(githash),
  FOREIGN KEY (id) REFERENCES notes(id)
);

CREATE TABLE shortcuts (
  id      INT             NOT NULL AUTO_INCREMENT PRIMARY KEY,
  traces  MEDIUMTEXT      NOT NULL
);

Common queries that the dashboard will use:

INSERT INTO notes (type, author, notes) VALUES (1, 'bsalomon', 'Alert!');

SELECT LAST_INSERT_ID();

INSERT INTO githashnotes (ts, id) VALUES (<githash_ts>, <last_insert_id>);

The above set of commands will usually be used together to add new annotations and associate them with corresponding git commits. The commands below remove an annotation and its associations with any commit.

DELETE FROM githashnotes WHERE id = <id_to_delete>;

DELETE FROM notes WHERE id = <id_to_delete>;

Since the data size is relatively small, the dashboard server can keep a copy of all recent commit info (e.g., for constructing a “blamelist”), annotations, and their many-to-many relationship for use in the context.

Password for the database will be stored in the metadata instance. To see the current password stored in metadata and the fingerprint:

gcutil --project=google.com:skia-buildbots getinstance [skia-perf GCE instance]

To set the mysql password that perf is to use:

gcutil --project=google.com:skia-buildbots setinstancemetadata [skia-perf GCE instance] --metadata=readonly:[password-from-valentine] --metadata=readwrite:[password-from-valentine] --fingerprint=[the metadata fingerprint]

Startup and config

The server is started and stopped via:

sudo /etc/init.d/perf [start|stop|restart]

But sysv init only handles starting and stopping a program once, so we use Monit to monitor the application and restart it if it crashes. The config is in:

/etc/monit/conf.d/perf

Installation

See the README file.