Provides interactive dashboard for Skia performance data.
The code for the server along with VM instance setup scripts is kept in:
This is the general flow of data for the Skia performance application. The frontend is available at http://skiaperf.com.
+-------------+ | | | Browser | | | | | | | +----------^--+ | +--------------------+----+-----+ | GCE Instance|skia+perf+b | | | | | +-----------+----------+ | | | Squid3 | | | | | | | +--------^-------------+ | | | | | +----------+-------------+ | | | Perf (Go) | | | | ^ ^ | | | +------------------------+ | | | | | | | | | | | | +------------------+ | | | | |Tile Pipeline (Go)| | | | | | ^ | | | | | +--+---------------+ | | | | | | | +-------------------------------+ | | | | +---------+-+ | | +-------+--+ | MySQL | | | | BigQuery | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +-----------+ | | +----------+ | | +-+----v---+ | Tile | | Repo | | | | | | | | | +----------+ Perf is a Go application that serves the HTML, CSS, JS and the JSON representations that the JS needs. It loads test results in the form of ‘tiles’ from the Tile Repo. It combines that data with data about commits and annotations from the MySQL data base and serves that the UI.
The Tile Pipeline is a separate application that periodically queries for fresh data from BigQuery and then writes Tiles into the Tile Repo. Note that when ingestion moves out of prod and into the same server we can do the tile updates immediately after ingestion is done.
Tile Repo will be represented internally as an interface, the first implemetation will be as files on the local disk, with a directory tree that contains gzipped JSON files called tiles. Note that we may alternatively use Go native gob encoded files and just transform then into JSON when serving them to the UI.
Each tile contains exactly 16 points of every trace for a dataset. The one exception being the last.gz tile, which may contain less that 16 points; see below for an explanation of that. The Tile Repo directory structure is:
$TILE_REPO_ROOT/<dataset>/<scale>/<tilenumber>.gz
Where:
So the data in:
/skps/0/0.gz
contains the data for the first 16 commits from the BOT that have test data.
/micro/3/6.gz
contains the 16 commits per trace of every 64th commit that has data and is the 7th data set in that order. So it contains 16 points per trace, and each point falls between 6 * 64 * 16=6144 and 7 * 64 * 16=7168, i.e it is a slice of 16 commits that represent a range of 1024 commits.
A manifest file will be available that give the commit timestamp ranges for each tile.
When navigating the UI users can select the tiles they are looking at (<, >) and also change the scaling factor that they are looking at (+,-).
Each /dataset/scale directory also contains a file, last.gz, that contains the most recent data, from 1 to 16 points. The Tile Pipeline process will update the ‘last.gz’ files in each directory and write new tile files as new data arrives. Last.gz will contain the id of the tile that appears just before it, that way the UI can request /skps/0/last.gz and know how to proceed from there.
The URL structure for retrieving Datasets is TBD.
For each point if the user wants to zoom out, add 1 to the scale factor and divide tilenumber by two. Do the opposite to zoom in. To move forwards or backwards in time add or subtract 1 to the tile number. The actual UI mechanisms for navigating around traces are TBD, this is just a description of how the tiles are arranged.
Start at /0/last.gz and find the previous tilenumber.gz, open that and find the last githash. Query for all data newer than that githash. Group into tiles of 16 githashes. Put the remainder (or the last 16 if the remainder is 0) into /0/last.gz.
Now do the same for /1/, but the data comes from /0/ and not from BigQuery. I.e. /1/0.gz is just a sampling of /0/0.gz, /0/1.gz, /0/2.gz and /0/3.gz. At each level only rewrite last.gz and write out any new complete tiles as they are filled.
The data for the performance metrics are kept in the BigQuery tables stored in the google.com:chrome-skia project. Note that this is a different project from where the data is accessed, which is by VM instances running under the google.com:skia-buildbots project. For this to work the service account email of the VM needs to be added to the permissions group of the google.com:chrome-skia project. If this isn't done then the BigQuery access will fail with a 403 error.
We use the https://github.com/golang/glog for logging, which puts Google style Error, Warning and Info logs in /tmp on the server under the ‘perf’ account.
Starting the application is done via /etc/init.d/perf which does the backgrounding itself via start-stop-daemon, which means that if the app crashes when first starting then nothing will make it to the logs. To debug the cause in that case edit /etc/init.d/perf and remove the --background flag and then run:
$ sudo /etc/init.d/perf start
And you should get stdout and stderr output.
Monitoring of the application is done via Graphite at http://skiamonitor.com. Both system and application level metrics are monitored.
A Cloud SQL (a cloud version of MySQL) database is used to keep information on Skia git revisions and their corresponding annotations. The database will be updated when users add/edit/delete annotations via the dashboard UI.
All passwords for MySQL are stored in valentine (search “skia perf”).
To connect to the database from authorized network (including skia-perf GCE):
$ mysql -h 173.194.104.24 -u root -p
Initial setup of the database, the users, and the tables:
CREATE DATABASE skia; USE skia; CREATE USER 'readonly'@'%' IDENTIFIED BY <password in valentine>; GRANT SELECT ON *.* TO 'readonly'@'%'; CREATE USER 'readwrite'@'%' IDENTIFIED BY <password in valentine>; GRANT SELECT, DELETE, UPDATE, INSERT ON *.* TO 'readwrite'@'%'; // Table for storing annotations. CREATE TABLE notes ( id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, type TINYINT, author TEXT, notes TEXT NOT NULL ); // Table for storing git revision information. CREATE TABLE githash ( githash VARCHAR(40) NOT NULL PRIMARY KEY, ts TIMESTAMP NOT NULL, gitnumber INT NOT NULL, author TEXT NOT NULL, message TEXT NOT NULL ); // Table for mapping revisions and annotations. This support many-to-many // mapping. CREATE TABLE githashnotes ( githash VARCHAR(40) NOT NULL, ts TIMESTAMP NOT NULL, id INT NOT NULL, FOREIGN KEY (githash) REFERENCES githash(githash), FOREIGN KEY (id) REFERENCES notes(id) ); CREATE TABLE shortcuts ( id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, traces MEDIUMTEXT NOT NULL );
Common queries that the dashboard will use:
INSERT INTO notes (type, author, notes) VALUES (1, 'bsalomon', 'Alert!'); SELECT LAST_INSERT_ID(); INSERT INTO githashnotes (ts, id) VALUES (<githash_ts>, <last_insert_id>);
The above set of commands will usually be used together to add new annotations and associate them with corresponding git commits. The commands below remove an annotation and its associations with any commit.
DELETE FROM githashnotes WHERE id = <id_to_delete>; DELETE FROM notes WHERE id = <id_to_delete>;
Since the data size is relatively small, the dashboard server can keep a copy of all recent commit info (e.g., for constructing a “blamelist”), annotations, and their many-to-many relationship for use in the context.
Password for the database will be stored in the metadata instance. To see the current password stored in metadata and the fingerprint:
gcutil --project=google.com:skia-buildbots getinstance [skia-perf GCE instance]
To set the mysql password that perf is to use:
gcutil --project=google.com:skia-buildbots setinstancemetadata [skia-perf GCE instance] --metadata=readonly:[password-from-valentine] --metadata=readwrite:[password-from-valentine] --fingerprint=[the metadata fingerprint]
The server is started and stopped via:
sudo /etc/init.d/perf [start|stop|restart]
But sysv init only handles starting and stopping a program once, so we use Monit to monitor the application and restart it if it crashes. The config is in:
/etc/monit/conf.d/perf
See the README file.