Google Git
Sign in
skia/buildbot/8a7eefcae3fba7f2baabd1a50aef18031fa6d331/./perf
tree: 64011ea6b7500f5547941014fd7740b0b51a894b
  1. cockroachdb/
  2. configs/
  3. coverage/
  4. csv2days/
  5. demo/
  6. docs/
  7. go/
  8. images/
  9. integration/
  10. jupyter/
  11. lint/
  12. modules/
  13. nanostat/
  14. pages/
  15. res/
  16. samplevariance/
  17. scripts/
  18. secrets/
  19. smoke_tests/
  20. .eslintrc.js
  21. .gitignore
  22. API.md
  23. BACKUPS.md
  24. BUILD.bazel
  25. CHECKLIST.md
  26. CIDER_AGENT.md
  27. DESIGN.md
  28. FORMAT.md
  29. GEMINI.md
  30. gen_version_test.sh
  31. highlevel.excalidraw.png
  32. hot-reload.sh
  33. ingest.excalidraw.png
  34. Makefile
  35. migrate.sh
  36. mocha-setup.ts
  37. multigraph-guide.md
  38. PERFSERVER.md
  39. PERFTOOL.md
  40. PROD.md
  41. README.md
  42. reingest-android-data.sh
  43. report-page-guide.md
  44. run_local_spanner_bridge.sh
  45. run_perfserver.sh
  46. run_with_spanner.sh
  47. Spanner.md
  48. TRIAGE.md
  49. tslint.json
perf/README.md

Skia Perf

Skia Perf (also known as the Performance Dashboard) is a distributed system for monitoring and analyzing performance benchmarking metrics produced across Continuous Integration (CI) and postsubmit testing workflows. It reads performance data from databases and serves interactive dashboards to highlight how a commit impacts performance, allowing for easy exploration and annotation.

While originally developed by the Skia project to track 2D graphics performance and nanobenchmarks, Perf serves as the standard performance analysis dashboard across multiple open-source projects. The product is particularly useful for:

  • Chrome Developers who are interested in performance regressions.
  • Marketing Teams who need to track, visualize, and communicate product's performance enhancements.

Table of Contents

About

Core Capabilities

  • Continuous Metric Ingestion: Stores time-series data from automated benchmark suites run across various platforms, architectures, and hardware configurations.
  • Anomaly & Regression Detection: Algorithms monitor incoming telemetry for significant changepoints and regressions, automatically filing alerts and issue reports.
  • Interactive Data Exploration: Web interfaces for plotting metrics over time, comparing commits, viewing blame lists, and exploring cluster trends.

History

The current Perf infrastructure in this repo was originally developed for Skia. In 2023, a project began to unify it with Chrome's performance tooling, replacing a legacy Python-based system, called Chromeperf. This unification effort involves consolidating features from both platforms onto this modern Go and TypeScript stack, with the goal of eventually deprecating the older system.

Terminology

  • Benchmark: A top-level test name.
  • Test: A specific test case within a benchmark.
  • Subtest: A further breakdown of a test.
  • Bot: The device or machine that runs the tests.
  • Trace: A single line on a graph, representing measurements for a single test over time. A trace has a unique key, which is a combination of its properties (e.g., benchmark, test, subtest, bot, etc.).
  • Traceset: The set of key-value pairs that uniquely identifies a trace.
  • X-axis: Always represents commit position or timestamp.
  • Anomaly: A statistically significant change in a trace, which could be a regression or an improvement.
  • Frame: A chunk of trace data stored in the database.
  • Sheriff: A person or tool responsible for monitoring a set of tests for regressions.
  • ChromePerf: The legacy implementation of Perf.
  • Catapult: The repository for the legacy Perf implementation.

See also Chromium Infra Glossary.

Production Instances

Below is the directory of active, public production Performance Dashboard instances across various open-source projects. For the complete list, including corp instances, refer to the configurations in the perf/configs/ directory.

Project / DashboardLive URLDescription
Skia Graphicsskia-perf.luci.appPublic dashboard for the Skia 2D Graphics Library, tracking nanobenchmarks and task durations.
Chromium (Public)perf.luci.appPublic Chromium press benchmarks (Speedometer2, JetStream2, MotionMark).
AndroidXandroidx-perf.skia.orgMonitors performance and CQ tests for the Android Jetpack (AndroidX) support libraries.
ANGLE Graphicsangle-perf.luci.appPublic GPU and graphics translation layer benchmarks (angle_perftests).
Flutter Engineflutter-engine-perf.luci.appLow-level rendering speed and frame time metrics for the Flutter Engine.
Flutter Frameworkflutter-flutter-perf.luci.appCore framework-level UI performance, build speeds, and framework benchmarks.
Fuchsiafuchsia-perf.luci.appPublic telemetry, shell responsiveness, and system-level benchmarks for Fuchsia OS.
V8v8-perf.luci.appPublic JavaScript engine metrics, CQ dry-runs, and chromeperf benchmark runs.
WebRTCwebrtc-perf.luci.appAudio/video connection performance and streaming metrics for WebRTC.
Emscriptenemscripten-perf.luci.appPerformance and WebAssembly size metrics for Emscripten compiler releases.
Germaniumgermanium-evals.luci.appPublic evaluation dashboard tracking Chromium main source performance estimates.

Other Documentation

  • Skia Perf: Documentation specifically focused on the Skia instance-specific features.
  • ai_generated_doc.md: Overview of the system by Gemini.
  • API.md: How to use the HTTP/JSON API for alerts.
  • BACKUPS.md: Instructions for backing up regression and alert data.
  • CHECKLIST.md: A checklist for launching a new Perf instance.
  • DESIGN.md: The design documentation for Perf.
  • FORMAT.md: Details on the Perf JSON data format.
  • PERFSERVER.md: Documentation for the perfserver command-line tool.
  • PERFTOOL.md: Documentation for the perf-tool command-line tool.
  • PROD.md: A manual for operating Perf in a production environment.
  • Spanner.md: Information on the Spanner integration and running the emulator.
  • TRIAGE.md: Design for the regression triage page.